[Models] Phase 5 — @embed directive + write-time hook + auto-HNSW + model tracking

[heskew]

Scope

Schema-level RAG (Retrieval-Augmented Generation) without app code. A developer marks a field with @embed(source: "<source-field>", model: "<logical-model-name>") and Harper auto-fires the embedding on every write, stores it on the record, and indexes it via HNSW (Hierarchical Navigable Small World) — no application-level hooks needed.

What ships:

• @embed(source, model) schema directive — new parsing branch in resources/graphql.ts.
• setEmbedAttribute(name, embedder) registration API on the Table class, mirroring the existing setComputedAttribute pattern at resources/Table.ts:3438.
• Write-time hook integration via the existing write.before / write.beforeIntermediate pre-commit callback pattern (resources/Table.ts:4582 and call sites at 1345 / 1939 / 3026 / 4368).
• Automatic HNSW indexing on the @embed-decorated field — defaults to cosine distance; explicit @indexed on the same field overrides defaults.
• Per-table embedding-model tracking metadata, with invalidation when the model changes (mirrors @computed's version-driven reindex at resources/graphql.ts:131-145).
• Replicated-write predicate: hook fires only on originating writes (!ctx.replicateFrom && !ctx.alreadyLogged) — receivers store the originator's embedding, don't re-compute.

Why this is meaningful

Three lines of schema replace what would otherwise be a custom embedding pipeline:

type Document @table @export {
  id: ID @primaryKey
  content: String
  embedding: Vector @embed(source: "content", model: "default")
}

Behavior: on write/update of content, embedding is computed (sync; see open decision below), stored in embedding, indexed via HNSW. No app-side ingest pipeline, no separate embedding service, no consistency gap between record and vector.

Implementation

Directive parsing

Add a new branch to processGraphQLSchema() in resources/graphql.ts:117 alongside the existing @computed handler:

} else if (directiveName === 'embed') {
  const args = directive.arguments;
  property.embed = {
    source: args.find(a => a.name.value === 'source')?.value.value,
    model: args.find(a => a.name.value === 'model')?.value.value,
  };
  // If model changes between deploys, this triggers invalidation; same as @computed's version tracking
  property.embedModelVersion = property.embed.model;
}

Add 'embed' to server.knownGraphQLDirectives at resources/graphql.ts:11-27 so the unknown-directive warning at line 167 stays clean.

Registration API

Mirror setComputedAttribute (existing precedent at resources/Table.ts:3438):

class Table {
  static setEmbedAttribute(attributeName: string, embedder: (record: any) => Promise<Float32Array>) {
    // Stores the embedder function on attribute metadata, like setComputedAttribute does
    // Embedder signature accepts the full record so source-field changes are visible
  }
}

At schema load, after @embed is parsed, the framework calls setEmbedAttribute(name, defaultEmbedder) where defaultEmbedder is:

async (record) => {
  const sourceValue = record[embedConfig.source];
  if (sourceValue == null) return null;
  const vectors = await scope.models.embed(sourceValue, {
    model: embedConfig.model,
    inputType: 'document',                  // @embed always produces document embeddings
  });
  return vectors[0];
}

The developer can override via setEmbedAttribute(name, customEmbedder) if they need different logic (e.g. multi-field concatenation, custom preprocessing).

Write-time hook integration

In each write path that currently wires blob pre-commit (resources/Table.ts:1345, 1939, 3026, 4368), add a pass for @embed-decorated attributes:

// pseudocode at each put/patch/create call site
if (!ctx.replicateFrom && !ctx.alreadyLogged) {
  for (const attr of this.embedAttributes) {
    write.beforeEmbed = chainCallback(write.beforeEmbed, async () => {
      const sourceValue = record[attr.embed.source];
      if (sourceValueChanged(sourceValue, existingEntry)) {
        record[attr.name] = await attr.embedder(record);
      }
    });
  }
}

The transaction processor awaits write.beforeEmbed (or chains it onto write.beforeIntermediate) before commit. Failures propagate as normal pre-commit failures.

Auto-HNSW indexing

When @embed is parsed, the framework automatically attaches @indexed(type: "HNSW") to the same attribute if no explicit @indexed directive is present. Default distance metric: cosine. Explicit @indexed parameters on the same field override the defaults entirely.

Rationale: makes the issue's "indexed via HNSW. No app code needed." statement literally true with one directive.

Model tracking + invalidation

Table metadata records the model name that produced the stored vectors. When the table is loaded and the schema's current @embed(model: ...) differs from the recorded model, vectors are marked stale and re-embedded — mirrors the @computed version-tracking pattern at resources/graphql.ts:131-145.

Two re-embed modes:

• Sync (default): block at table load until re-embed completes. Fine for small tables; blocking for large.
• Background: schedule via the existing job infrastructure at server/jobs/; mark vectors as stale until the job catches up. Configurable.

Replicated-write predicate

When a record arrives via replication, the originating node has already computed the embedding; the receiver should just store. The signal exists today: resources/replayLogs.ts:50 synthesizes Context with alreadyLogged: true; REST sets request.replicateFrom when explicitly suppressed. Hook predicate: !ctx.replicateFrom && !ctx.alreadyLogged.

Open decision (resolve in this issue or in PR review)

Default execution mode for @embed: sync-by-default vs queued-by-default.

• Sync mode (recommended default): commit blocks until the embedding callback resolves. Fits the existing write.before pattern with zero new infrastructure. Write latency includes the model round-trip.
• Queued mode: record commits without embedding, an async task computes the vector and back-fills via a second write. Requires additional plumbing on top of the existing job infrastructure (server/jobs/). Isolated, medium-sized.

Recommendation: ship sync as the default, with queued available as a per-table or per-app config opt-in. Apps that need write-latency guarantees can flip the toggle.

Decision needed before this phase implementation begins.

Files

Path	Change
resources/graphql.ts	extended — parse @embed directive; add 'embed' to knownGraphQLDirectives
resources/Table.ts	extended — setEmbedAttribute API; wire write-time hook into existing pre-commit pattern; model-tracking metadata
resources/models/embedHook.ts	new — default embedder factory + queued-mode scheduler
test/embed.test.ts	new — directive parsing, sync write, replicated-write skip, model-change invalidation

Acceptance criteria

• @embed(source, model) directive parsed from GraphQL schemas; the attribute is registered with an embedder.
• Writes to a record fire the embedder, producing a vector stored in the attribute, indexed via HNSW.
• @embed-decorated attributes are automatically HNSW-indexed when no explicit @indexed is present.
• setEmbedAttribute(name, embedder) allows component authors to override the default embedder.
• Replicated writes skip the embedder — receivers store the originator's embedding (!ctx.replicateFrom && !ctx.alreadyLogged predicate verified by test).
• Model change invalidates existing vectors and triggers re-embed (sync at table load by default; queued mode configurable).
• Per-call accounting flows through hdb_model_calls (because the embedder calls scope.models.embed(); Phase 1's writer handles it). Attribution shows the originating user / tenant.
• Sync-by-default execution mode confirmed (per open decision above).
• Documented: declaring @embed, supplying a custom embedder, configuring queued mode, model-change behavior.
• CI green (unit + integration + 3 Node versions).

Out of scope

• Custom embedding model training — fine-tuning lives at the application layer.
• Schema directive registry — @embed ships hardcoded alongside the existing directives; registry pattern is a follow-up if more model-aware directives surface.
• Cross-model migration tooling (harper kb reembed --from X --to Y) — separate concern; not blocking Add unified model-access API (scope.models) #510.
• Conversation-resource embedding wiring — that's Add ConversationResource for agent memory and conversation state #511's territory (its turns table declares its own @embed).
• Multimodal embeddings (image, audio) — text-only in v1; capability extension when backends support it.

Stacks on

• Phase 1 ([Models] Phase 1 — Foundation: scope.models interfaces, hdb_model_calls, TestBackend #628) — needs scope.models.embed() and hdb_model_calls for accounting.
• Any one backend (Phase 2 [Models] Phase 2 — ollama backend #629 or Phase 3 [Models] Phase 3 — openai backend (+ toolMode: 'return') #630) — needs something that actually produces vectors.

Branch & PR conventions

• Branch: feat/models-embed-directive
• PR base: main (after Phase 1 and at least one backend phase merge).
• Closes this issue via Closes #<self>; references Add unified model-access API (scope.models) #510 via Tracking: #510.

Smoke test

# Schema:
type Document @table @export {
  id: ID @primaryKey
  content: String
  embedding: Vector @embed(source: "content", model: "default")
}

# Configure models.embedding.default to point at a real backend (Ollama or OpenAI)
# Deploy schema, then write a record:

curl -X POST http://localhost:9926/Document/ \
  -H 'Content-Type: application/json' \
  -d '{"id": "doc1", "content": "harper is a database"}'

# Expected: 201; record persisted.
# Verify: GET /Document/doc1 → embedding is populated with a real Float32Array.
# Verify: SELECT * FROM system.hdb_model_calls WHERE method = 'embed' ORDER BY $createdtime DESC LIMIT 1 — shows the embed call.

# Vector search:
curl -X POST http://localhost:9926/Document/ \
  -H 'Content-Type: application/json' \
  -d '{"operation": "search_by_value", "search_attribute": "embedding", "search_value": <a query vector>}'

# Expected: doc1 returned, ordered by cosine similarity.

# Replicated-write skip: on a replica, the embedding written by the originating node persists as-is;
# the embedder does NOT fire again. Verify by checking that no new analytics.model_call rows appear on the replica.

Tracking

Part of #510. Sub-issue 5 of 6.

---

🤖 Generated with Claude Code

[kriszyp]

Queued mode:

I am more inclined to think the right way to minimize write latency is to move the vector embedding into a derived caching table (that follows the sync mode, but the "origin" table doesn't need to wait for it).

Anyway, the default for using sync mode, and plan for implementation sounds great, and I would expect that's overwhelming the mode that should and would be used.