[Models] Phase 2 — ollama backend

[heskew]

Scope

Ship the first real ModelBackend implementation in core: an ollama backend that talks to a local or remote Ollama HTTP API. Validates the Phase 1 interface against something non-trivial without external dependencies in CI.

What ships:

• ollama backend class implementing ModelBackend with embed, generate, generateStream.
• Config-driven backend resolution (backend: ollama, host: ..., model: ...).
• Streaming via Ollama's chunked response → AsyncIterable<GenerateChunk> per Phase 1's contract.
• AbortSignal honored from BackendOpts — propagated into the underlying fetch() so dropped-client cancellation aborts upstream.
• Per-call accounting written through hdb_model_calls via Phase 1's writer.
• Integration tests against a real local Ollama (matching the source-portability promise).

API surface

Implements the interface from Phase 1 (#628). Capability negotiation:

capabilities(): ModelCapabilities {
  return {
    embed: true,
    generate: true,
    stream: true,
    tools: false,        // Ollama tool support is evolving; not yet portable enough for v1
    adapters: false,     // No multi-LoRA serving in Ollama
  };
}

Configuration (extends Phase 1's backend registry shape):

models:
  embedding:
    default:
      backend: ollama
      host: localhost:11434
      model: <your-embedding-model>     # e.g. nomic-embed-text
  generative:
    fast:
      backend: ollama
      host: localhost:11434
      model: <your-generative-model>    # e.g. llama-3.2-3b

Multiple ollama backends can coexist (different host and model per logical name). The registry indexes them by logical name from config.

Implementation notes

• HTTP client: native fetch(). No new SDK dependency in core.
• Ollama endpoints used:
  • POST /api/embeddings for embed
  • POST /api/generate (non-streaming) and POST /api/chat (chat-shape) for generate
  • Same with stream: true for generateStream
• Token-count fields when Ollama returns them (prompt_eval_count, eval_count) map to TokenUsage.promptTokens / completionTokens / embeddingTokens.
• gpu_ms is not reported by Ollama; left undefined in the accounting record.
• inputType: 'document' | 'query' on EmbedOpts: Ollama doesn't currently consume this distinction in its embedding API, but some models (nomic-embed-text v1.5+) prepend the appropriate prefix at the application layer. Backend handles the prefix injection when the model is one that uses it; documented in code as a per-model behavior table.
• signal: AbortSignal from BackendOpts is passed straight into the fetch() call; client disconnect → fetch aborts → upstream HTTP closes.

Files

Path	Change
resources/models/backends/ollama.ts	new — OllamaBackend class
resources/models/backends/index.ts	new (or extended) — backend factory / registration
test/models/ollama.test.ts	new — integration tests against a running Ollama instance

Backend registration: when Harper boots and reads the models config, backend: ollama instantiates OllamaBackend with the provided host and model, registers under the logical name in the backend registry.

Acceptance criteria

• OllamaBackend implements ModelBackend per Phase 1's interface.
• scope.models.embed() configured with backend: ollama produces a vector from a running Ollama instance.
• scope.models.generate() produces a completion from a running Ollama instance.
• scope.models.generateStream() yields content deltas via the AsyncIterable<GenerateChunk> shape from Phase 1.
• Per-call accounting records backend: 'ollama', model, token counts (when Ollama reports them), latency, success.
• AbortSignal from BackendOpts cancels in-flight requests when a client disconnects.
• Capability negotiation correctly reports tools: false, adapters: false.
• inputType: 'document' | 'query' produces different vectors for models that distinguish (e.g. nomic-embed-text v1.5+ when configured).
• Integration tests pass against a local Ollama instance (CI provisioning to be settled in PR review — likely a container in the integration shard).
• CI green (unit + integration + 3 Node versions).

Out of scope

• Tool calls via Ollama — tools: false advertised; Ollama tool-call support is uneven across models and not portable enough for v1.
• LoRA adapter selection — adapters: false; multi-LoRA serving is FAB-503 territory via vLLM.
• Streaming cancellation beyond AbortSignal propagation (no explicit "cancel" endpoint on Ollama).
• Custom HTTP transport (mTLS, proxies, etc.) — uses default fetch() config for now; extensions ship as backend-config options later.

Stacks on

• Phase 1 ([Models] Phase 1 — Foundation: scope.models interfaces, hdb_model_calls, TestBackend #628) — needs the ModelBackend interface, backend registry, and hdb_model_calls writer.

Hard prerequisites

• Expose request.signal (AbortSignal) for Resource methods #513 (request.signal exposed on Resource methods) — Phase 1 reads ctx.signal from ALS; this phase consumes it through BackendOpts.signal.

Branch & PR conventions

• Branch: feat/models-ollama-backend
• PR base: main (after Phase 1 merges).
• Closes this issue via Closes #<self>; references Add unified model-access API (scope.models) #510 via Tracking: #510.

Smoke test

# Prerequisites:
# - Ollama running locally on :11434 with a model pulled (e.g. `ollama pull nomic-embed-text`)
# - Harper config has models.embedding.default = { backend: ollama, host: localhost:11434, model: nomic-embed-text }

# In a Resource method:
class EmbedTest extends Resource {
  async post(_target, body, _request) {
    return await scope.models.embed(body.text);
  }
}

curl -X POST http://localhost:9926/EmbedTest/ \
  -H 'Content-Type: application/json' \
  -d '{"text": "hello world"}'

# Expected: a real 768-dim (or model-appropriate) Float32Array
# Verify: SELECT * FROM system.hdb_model_calls WHERE backend = 'ollama' ORDER BY $createdtime DESC LIMIT 1
#   shows the call with method='embed', model, latency_ms, embedding_tokens, success=true.

Tracking

Part of #510. Sub-issue 2 of 6.

---

🤖 Generated with Claude Code

[heskew]

Heads-up from #628 (Phase 1 PR #638): the foundation adds models as a property on the per-component Scope instance internally, but does NOT expose it to user-deployed code. security/jsLoader.ts's getHarperExports doesn't include models (only Resource, tables, databases, etc.), and there's no scope global available to user modules.

So the canonical access pattern from a user Resource — e.g. await scope.models.embed(...) per #510's smoke test — does not work as written today. Three exposure paths to choose between when Phase 2 lands:

1. harper.models — add models to getHarperExports. Matches the existing harper.Resource / harper.tables shape.
2. Top-level models global — add to getGlobalObject alongside server/logger/resources. Simpler ergonomics, more visible API.
3. scope.models literally — introduce a scope global. Matches Add unified model-access API (scope.models) #510's smoke-test wording.

Whichever Phase 2 picks will set the pattern for #630, #631, #632, #633. The Phase 1 integration smoke test was deferred to land alongside this decision (it'd need a Resource that exercises whichever access pattern wins).

PR #638 describes the deferral in its "Deferred to Phase 2" section if more context helps.

---

🤖 Posted by Claude on Nathan's behalf