Documentation Review: Developer Experience Analysis

Graph-aware embeddings are a Papr embedding mode that reshapes the vector space 
so it behaves like a structured graph of meaning, not only a flat semantic similarity.

# Graph-aware embeddings and domain schemas

**The problem:** Standard vector search ranks by "semantic closeness" but can't tell if two results 
are close for the **same reason**. A code snippet about "sorting arrays" and "sorting linked lists" 
might be semantically near but use different algorithms, data structures, and APIs.

**Graph-aware embeddings** solve this by encoding **structured domain dimensions** alongside your 
base vector—things like "programming language," "operation type," "temporal context," or custom 
fields you define. Search can then filter and boost by **topical alignment**, **domain-specific 
context**, and other axes beyond flat similarity.

**Implementation:** Uses Papr's holographic pipeline (`enable_holographic`, `holographic_config`, 
`/v1/holographic/*`). This guide covers **concepts and schemas**; API request/response shapes are 
in the [API reference](/apis/index.yaml).

## About frequency bands (Hz values)

The built-in schemas show **Hz** values (0.1, 0.5, 2.0, etc.) representing the 14 standard 
**brain-inspired frequency bands** in holographic encoding. When registering a custom schema, 
your `frequency` field must use one of these allowed values (see `CustomFrequencyField` in the 
API reference for the exact enum).

**Rule of thumb:** 
- Lower Hz (0.1–2.0) → Categorical/Enum dimensions (language, domain)
- Mid Hz (4.0–14) → Descriptive/FreeText dimensions (intent, operation)
- Higher Hz (18–70) → List/MultiValue dimensions (APIs, entities)

## Do you need graph-aware embeddings?

Follow this decision tree:

1. **Is your baseline search "good enough"?**  
   → Yes: Skip graph-aware. Standard semantic + agentic graph is sufficient.  
   → No: Continue ↓

2. **Can you describe the quality gap in domain terms?**  
   Examples: "Returns Python when I need JavaScript," "Mixes bug reports with feature requests"  
   → Yes: You have a domain mismatch → Continue ↓  
   → No: Your problem might be query quality or data sparsity, not embeddings

3. **Does a built-in schema match your domain?**  
   - **cosqa** → Code search (snippet ↔ natural language)  
   - **scifact** → Scientific claims ↔ evidence  
   - **general** → Mixed content domains  
   → Yes: Start with built-in → Measure → Tune  
   → No: Continue ↓

4. **Can you define 4-14 structured dimensions for your domain?**  
   Examples: contract_type, jurisdiction, ticket_priority, user_intent  
   → Yes: Register custom schema with Papr → Co-design recommended  
   → No: Graph-aware mode requires a schema; revisit when you can articulate dimensions

## Example: Code search with vs without graph-aware

**Query:** "How do I sort a list in Python?"

### Standard semantic search (without graph-aware):
Returns mixed results:
1. Python `list.sort()` ✅
2. JavaScript `array.sort()` ❌ (different language)
3. Python sorting algorithms tutorial ⚠️ (conceptual, not code)
4. SQL `ORDER BY` ❌ (different domain)

### With graph-aware embeddings (cosqa schema):
Returns filtered results aligned on **language=Python** and **primary_operation=sorting**:
1. Python `list.sort()` ✅
2. Python `sorted()` function ✅
3. Python custom sort with `key=` ✅
4. Python `heapq.nsmallest()` for partial sorts ✅

**Why:** The schema encodes **programming_domain**, **language**, and **primary_operation** dimensions. 
H-COND scoring boosts results with high alignment on these fields.

**When to reverse:**
- **Building LLM prompts**: Models expect chronological flow (oldest → newest)
- **Timeline displays**: Users expect conversation order
- **Transcript exports**: Standard format is oldest-first

**When to keep as-is (newest-first):**
- **Recent activity feeds**: "What happened recently?"
- **Pagination UX**: Load latest messages first, older on scroll
- **Admin dashboards**: Show most recent interactions

### Full lifecycle example

```python
# 1. Create memory with webhook
memory = client.memory.add(
    content="Q4 planning: Launch new API, hire 2 engineers, $200k budget",
    enable_holographic=True,
    frequency_schema_id="general",
    webhook_url="https://api.myapp.com/papr-webhook",
    webhook_secret="my_webhook_secret_key"
)

memory_id = memory.memory_id
print(f"Created: {memory_id}, initial status: quick_saved")

# 2. Poll status if you need it immediately (webhook takes ~seconds)
import time
for i in range(5):
    status = client.memory.get_status(memory_id)
    print(f"Poll {i+1}: {status.status}")
    if status.status == "completed":
        break
    time.sleep(2)

# 3. Webhook handler (receives POST when done)
# POST https://api.myapp.com/papr-webhook
# Headers: X-Webhook-Secret, X-Webhook-Signature (HMAC)
# Body: {"event": "memory.completed", "memory_id": "...", "status": "completed", "completed_at": "..."}

### Issue 7: Holographic terminology inconsistency

**Problem:**
We use both "holographic" and "graph-aware" — relationship not always clear.

**Fix:**
Add a **terminology box** at the top of graph-aware-embeddings.md:

```markdown
> **Terminology:** We call this **graph-aware embeddings** (concept) implemented via 
> **holographic** APIs (technology). When you see `enable_holographic` or 
> `holographic_config`, that's the implementation layer. When we discuss "domain schemas" 
> and "structured dimensions," that's the conceptual layer this guide focuses on.

// Response
{
  "status": "success",
  "schema_id": "acme:legal_contracts:1.0.0",
  "domain": "legal_contracts",
  "num_frequencies": 4
}

# Use the returned schema_id when adding memories
client.memory.add(
    content="Signed NDA with Acme Corp, jurisdiction: US, expires 2027-01-01",
    enable_holographic=True,
    frequency_schema_id="acme:legal_contracts:1.0.0"  # Use registered schema
)

# And in search
results = client.memory.search(
    query="Find all active NDAs in US jurisdiction",
    holographic_config={
        "enabled": True,
        "frequency_schema_id": "acme:legal_contracts:1.0.0",
        "frequency_filters": {
            "contract_type": 0.9,  # Must be 90%+ aligned on contract type
            "jurisdiction": 0.8    # Must be 80%+ aligned on jurisdiction
        }
    }
)

- **Graph-aware embeddings (holographic APIs)** — Documented as domain-tuned vector space 
  (built-in **cosqa**, **scifact**, **general**; custom schemas via `POST /v1/holographic/domains`). 
  Implementation uses `enable_holographic`, `holographic_config`, `/v1/holographic/*` endpoints. 
  See [Graph-aware embeddings guide](/guides/graph-aware-embeddings.md); API shapes in 
  [API reference](/apis/index.yaml).

| Domain-tuned (graph-aware) retrieval | ... | Start with `frequency_schema_id="general"`, 
then move to domain-specific (`cosqa`, `scifact`) or custom. See 
[Graph-aware guide](../guides/graph-aware-embeddings.md) for decision tree |