[Models] Phase 6 — anthropic + bedrock backends

[heskew]

Scope

Two more ModelBackend implementations in core: anthropic (Claude via direct API) and bedrock (AWS Bedrock — Claude, Llama, Mistral, Titan, etc. via AWS). Both validate the interface against shapes that differ meaningfully from OpenAI's, and round out the major-provider coverage for the issue's Acceptance section.

What ships:

• AnthropicBackend class implementing ModelBackend (generate, generateStream, native tool calls; no embed — Anthropic doesn't ship an embedding API).
• BedrockBackend class implementing ModelBackend (embed, generate, generateStream, with tool calls on the model families that support them).
• Config-driven backend resolution for both.
• Streaming via each SDK's chunked completion API → AsyncIterable<GenerateChunk>.
• AbortSignal propagation into SDK calls.
• Per-call accounting written through hdb_model_calls.

Implementation only — no new design surface. The interface validation work happened in Phase 3 (OpenAI); these are mostly translation-layer code per-provider.

Capability shapes

anthropic

capabilities(): ModelCapabilities {
  return {
    embed: false,                          // Anthropic doesn't currently ship an embedding API
    generate: true,
    stream: true,
    tools: true,                           // first-class tool-call support
    adapters: false,
  };
}

bedrock

capabilities(): ModelCapabilities {
  return {
    embed: true,                           // via Titan / Cohere / Voyage models on Bedrock
    generate: true,
    stream: true,
    tools: true,                           // on Claude, Llama 3.x, Mistral, etc.
    adapters: false,
  };
}

For Bedrock, tool support is per-model — the backend advertises tools: true at the capability level, and individual generate() calls raise a structured error if the specific configured model doesn't support tools. Same applies to embed — only Bedrock embedding models (Titan, Cohere, Voyage on Bedrock) work; calling embed() against a generative-only model raises a structured error.

Configuration

models:
  generative:
    claude:
      backend: anthropic
      model: <your-claude-model>            # e.g. claude-opus-4-7
      apiKey: ${ANTHROPIC_API_KEY}
    bedrock-claude:
      backend: bedrock
      model: <your-bedrock-model-id>        # e.g. anthropic.claude-opus-4-v1:0
      region: us-east-1
      # AWS credentials resolved via the standard AWS SDK chain (env vars, IAM role, profile, etc.)
  embedding:
    bedrock-titan:
      backend: bedrock
      model: amazon.titan-embed-text-v2:0
      region: us-east-1

Implementation notes

Anthropic backend

• SDK: official @anthropic-ai/sdk npm package.
• SDK pinning: lock to a specific minor version, bump per the Harper third-party trust model.
• messages array translation: Anthropic's shape differs slightly from OpenAI's (system is a top-level param, not a role; tool_use / tool_result are content blocks in messages, not separate fields). Translation handled in the backend's generate().
• Streaming: SDK's stream: true yields delta events. Translate to GenerateChunk.
• Token-count fields from result.usage (input_tokens, output_tokens) map to TokenUsage.promptTokens / completionTokens.
• Prompt caching (Anthropic-specific feature): exposed via the standard messages content blocks with cache_control markers; backend accepts opaque cache hints in opts but doesn't expose a Harper-side cache API in this phase.
• signal: AbortSignal passed into the SDK's signal option.

Bedrock backend

• SDK: @aws-sdk/client-bedrock-runtime npm package.
• Bedrock has multiple model invocation shapes per model family (Claude vs Llama vs Titan etc.). Backend dispatches on model field to the appropriate request shape — encapsulates the per-family logic so callers see only GenerateInput / GenerateOpts.
• Bedrock streaming uses the InvokeModelWithResponseStream API. Translate event-stream chunks to GenerateChunk.
• AWS auth via standard SDK chain — no API key field in the Harper config (env / IAM / profile do the work).
• Token-count fields per-family — Claude via Bedrock reports usage.input_tokens / usage.output_tokens; Llama via Bedrock reports prompt_token_count / generation_token_count. Per-family translation in the backend.

Files

Path	Change
resources/models/backends/anthropic.ts	new — AnthropicBackend class
resources/models/backends/bedrock.ts	new — BedrockBackend class with per-family request dispatch
resources/models/backends/index.ts	extended — register anthropic and bedrock factories
package.json	new deps — @anthropic-ai/sdk, @aws-sdk/client-bedrock-runtime (pinned versions)
test/models/anthropic.test.ts	new — unit + integration tests (live test behind env-gated flag)
test/models/bedrock.test.ts	new — unit + integration tests (live test behind env-gated flag)

Acceptance criteria

• AnthropicBackend implements ModelBackend per Phase 1's interface.
• BedrockBackend implements ModelBackend per Phase 1's interface; dispatches on model field to the right per-family request shape.
• scope.models.generate() produces completions from each provider.
• scope.models.generateStream() yields content deltas from each provider with correct stream framing.
• Tool calls in 'return' mode work on Anthropic and on Bedrock-Claude (and other tool-capable Bedrock models).
• scope.models.embed() produces vectors via Bedrock embedding models (Titan, etc.).
• embed: false on the anthropic backend → scope.models.embed() raises a structured "backend doesn't support embed" error (capability negotiation enforced from Phase 1).
• Per-call accounting records backend: 'anthropic' / 'bedrock', model, token counts, latency, success.
• AbortSignal from BackendOpts cancels in-flight SDK calls for both providers.
• AWS credentials resolved via the standard SDK chain for Bedrock (env / IAM / profile).
• SDK versions pinned and documented; bump cadence noted in PR description.
• Integration tests pass against real APIs behind env-gated flags (ANTHROPIC_API_KEY, AWS_PROFILE / equivalent).
• /v1/chat/completions from Phase 4 works against both backends end-to-end (an OpenAI-SDK client can hit Harper, which routes to Anthropic or Bedrock under the hood; OpenAI-shape translation in the gateway handles the request/response differences).
• CI green (unit + integration + 3 Node versions).

Out of scope

• toolMode: 'auto' orchestration — that's Add agent-loop orchestration / toolMode: 'auto' to scope.models #612, independent of these backends.
• Per-model fine-tuning or LoRA adapter selection — adapters: false on both.
• Anthropic Files / Messages Batch API (long-running operation surface) — ModelCallResult.pending is reserved in the interface but no backend in Add unified model-access API (scope.models) #510 emits it.
• Bedrock Knowledge Bases / Agents — those are application-layer features above the model-access API.
• Provider-side prompt caching as a first-class Harper feature — Anthropic's prompt caching is consumed via opaque opts passthrough; not a Harper-side cache primitive.

Stacks on

• Phase 1 ([Models] Phase 1 — Foundation: scope.models interfaces, hdb_model_calls, TestBackend #628) — needs the ModelBackend interface and registry.
• Phase 3 ([Models] Phase 3 — openai backend (+ toolMode: 'return') #630) — for the OpenAI-shape baseline that the gateway's translation layer was validated against; these backends piggyback on that validation.

Each backend (anthropic and bedrock) can ship independently in its own PR if convenient — they share Phase 1's foundation but don't depend on each other.

Branch & PR conventions

• Branch: feat/models-anthropic-bedrock-backends (single PR with both), or feat/models-anthropic-backend / feat/models-bedrock-backend if split.
• PR base: main (after Phase 1 and Phase 3 merge).
• Closes this issue via Closes #<self>; references Add unified model-access API (scope.models) #510 via Tracking: #510.

Smoke test

# Anthropic smoke:
# Prerequisites:
# - ANTHROPIC_API_KEY set in env
# - models.generative.claude = { backend: anthropic, model: claude-opus-4-7, apiKey: ${ANTHROPIC_API_KEY} }

# Direct via scope.models in a Resource:
class ChatTest extends Resource {
  async post(_target, body, _request) {
    return await scope.models.generate([{ role: 'user', content: body.q }], { model: 'claude' });
  }
}
curl -X POST http://localhost:9926/ChatTest/ -d '{"q": "say hi"}'

# Via the /v1/* gateway (Phase 4):
python3 -c '
import openai
client = openai.OpenAI(api_key="<harper-token>", base_url="http://localhost:9926/v1")
r = client.chat.completions.create(model="claude", messages=[{"role": "user", "content": "hi"}])
print(r.choices[0].message.content)
'

# Bedrock smoke:
# Prerequisites:
# - AWS credentials resolvable (env / IAM / profile)
# - models.generative.bedrock-claude = { backend: bedrock, model: anthropic.claude-opus-4-v1:0, region: us-east-1 }

curl -X POST http://localhost:9926/ChatTest/ -d '{"q": "hello via bedrock"}'

# Verify both: SELECT * FROM system.hdb_model_calls WHERE backend IN ('anthropic', 'bedrock') ORDER BY $createdtime DESC LIMIT 5

Tracking

Part of #510. Sub-issue 6 of 6.

---

🤖 Generated with Claude Code