Expand model interoperability with vendor-neutral adapters

[knisar]

Direction

The core model interface should remain vendor-neutral. BaseModel and ModelResponse define the runtime boundary, and OpenAICompatibleModel already covers public and self-hosted services that implement the compatible chat-completions protocol. The optional Anthropic adapter is now complete through #55 and #56.

Future core work should improve the neutral integration surface instead of adding a first-class cloud adapter for each platform. A provider-specific adapter belongs in core only when it addresses a concrete capability that cannot be expressed through an existing neutral path and has a separately scoped issue. Otherwise, provider-specific support should use the compatible endpoint, an application-defined BaseModel, or an external package.

The repository also ships optional PyRIT and Garak integrations. Future harness work should be filed as a focused gap in those implementations rather than as another broad adapter.

Current child work

No open child implementation is currently scoped. The adapter contract, callable adapter, and call-time option handling have shipped. Keep new integration requests focused on a concrete gap and scope them separately before implementation.

Completed work

• Add a vendor-neutral CallableModel adapter for custom inference functions #60, implemented in feat(models): add a vendor-neutral CallableModel adapter #64, added CallableModel for synchronous or asynchronous custom inference functions without a provider dependency.
• Make built-in adapter call-time options explicit and reject unsupported kwargs #63, implemented in fix(models): make adapter call-time options explicit #65, made built-in adapter call-time options explicit and rejected unsupported keyword arguments.
• docs: explain current scorer and callable contracts #69 documented the callable adapter's forwarding contract and context trust boundary alongside the built-in adapter contract.
• Add a provider-neutral model adapter contract and conformance suite #59 defined the vendor-neutral adapter contract and added reusable offline conformance tests.
• Add an optional Anthropic model adapter #55 and feat(models): add Anthropic Messages API adapter #56 added the optional Anthropic adapter.
• The OpenAI-compatible endpoint contract is covered by test(models): lock down the existing OpenAI-compatible endpoint contract #42.
• PyRIT and Garak integrations, assessor wiring, and documentation are already present.

Deferred work

External adapter discovery through Python entry points remains deferred. The contract and callable adapter have shipped through #59 and #60; scope discovery separately only if real usage demonstrates a need.

Contributing

This is an umbrella issue. Do not implement directly from it. Choose an open child issue labeled status: available, check its Development section for linked work, and open a draft pull request with Closes #NN once you have an initial commit. The linked draft is the signal that work has started, so no /claim comment or manual status change is needed.

[M4h1m4]

Hi @knisar, I want to claim the Anthropic adapter.

Plan: AnthropicModel in rai_toolkit/models/anthropic.py, mirroring OpenAICompatibleModel — same from_args() intake so it drops into the existing config-dict path, and the same ModelResponse metadata keys (model, token counts) so traces stay uniform downstream. Leaving the span to BaseModel.__init_subclass__ rather than wrapping predict myself, so it lands under rai.model.predict like everything else. The real difference from the OpenAI shape is that Anthropic takes system as a top-level parameter rather than a message role, and max_tokens is required — so the context/system assembly needs restructuring rather than reuse. Tests fully mocked, no network in CI.

One question on the provider track: openai_compatible.py already documents Ollama support via the OpenAI-compatible endpoint. Is a dedicated Ollama adapter still wanted, or is that considered covered?

Separately - I'm interested in the Garak side of the harness track once this is in, so please don't hold it on my account if someone else wants it. When I get there the design question I'd want your steer on is that Garak fuses generation and detection in a probe via primary_detector, while our catalog carries success criteria through redteam/runner.py - so it's either map onto existing Garak detectors and document the gaps, or ship a rai_toolkit detector module alongside the probes.

[knisar]

Thanks for the detailed proposal, @M4h1m4. A native Anthropic adapter is useful, and I split it into #55 with the implementation contract and local checks. Please open a linked draft pull request with Closes #55 once you have an initial commit. You do not need another /claim comment or a label change.

You are right about Ollama. The existing OpenAI-compatible adapter already covers its compatible endpoint, so a dedicated Ollama adapter is not needed. The Garak and PyRIT integrations also ship now, including assessor wiring and documentation.

One scope note: from_args() should keep adapter construction consistent, but adding Anthropic to the Streamlit or ApplicationProfile intake routes is outside #55. Let us get the provider adapter stable first and track intake integration separately if it is needed.

[M4h1m4]

Thanks @knisar for scoping #55 — the contract and the local checks are clear.

Before I started, I saw that another contributor had already opened #56 against it, so I have not begun any Anthropic work to avoid duplicate work.

Two options for where I go next, whichever is more useful to you:

• Another provider adapter. Expand model interoperability with vendor-neutral adapters #21 also names Google/Vertex and AWS Bedrock, and neither looks split out yet. If you scope one the way you did Add an optional Anthropic model adapter #55 I am happy to take it. The Anthropic shape I described carries over — Vertex has the same system/user split to work out, and Bedrock adds the request/response envelope per model family.
• Add context precision and recall scorers for RAG retrieval #26, context precision and recall. It is unclaimed, and it sits next to what I have been doing on Add a citation correctness scorer for RAG outputs #22 — same retrieval context, same parsing constraints. While working on feat(scorers): add citation correctness scorer for RAG outputs #27 I hit the fact that evaluation/pipeline.py forwards only the flattened retrieved_context string, so retrieved_ids from ModelResponse.metadata never reaches the scorer. Per-chunk metrics would want those ids, so Add context precision and recall scorers for RAG retrieval #26 may need that plumbing decided first — worth knowing before it is scoped.

[knisar]

Thanks for offering to take the next provider adapter, @M4h1m4. I revisited the direction for this part of the toolkit and want future core integration work to stay vendor-neutral.

I split the next work into #59, which defines the adapter contract and conformance tests, and #60, which adds a CallableModel for custom synchronous or asynchronous inference functions. Both are scoped and available. If either is a good fit, please check for linked work and open a draft pull request with Closes #NN once you have an initial commit. No /claim comment is needed.