From 3ebc4dd3cd5f788a35a2d23527414626dd5fb35c Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Mon, 24 Aug 2026 16:32:19 -0400 Subject: [PATCH 01/18] Add RFC-0011: Agent Registry (first draft) First draft covering the introductory material and user journeys: record-level registry for AI agents with Agent/AgentVersion entities, composition (BOM) with definitional anchors, A2A card as canonical-when-present payload, protocol-typed access bindings, and agent-centric traces and evaluations via a new trace destination. Detailed design and later sections are TBD. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 777 ++++++++++++++++++ 1 file changed, 777 insertions(+) create mode 100644 rfcs/0011-agent-registry/0011-agent-registry.md diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md new file mode 100644 index 0000000..a311f2f --- /dev/null +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -0,0 +1,777 @@ +# RFC 0011: Agent Registry + +| start_date | 2026-08-24 | +| :----------- | :--------- | +| mlflow_issue | | +| rfc_pr | | + +| Author(s) | [Bill Murdock](https://github.com/jwm4) (Red Hat) | +| :--------------------- | :-- | +| **Date Last Modified** | 2026-08-24 | +| **AI Assistant(s)** | Claude Code | + +**Table of contents** + +- [Summary](#summary) +- [Basic example](#basic-example) +- [Motivation](#motivation) + - [The problem](#the-problem) + - [User journeys](#user-journeys) + - [Out of scope](#out-of-scope) +- [Detailed design](#detailed-design) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) +- [Adoption strategy](#adoption-strategy) +- [Open questions](#open-questions) + +# Summary + +Add an Agent Registry to MLflow: a governed, record-level registry for +AI agents. The registry catalogs the agents an organization has built. +It records each agent's identity, ownership, composition, source +provenance, and lifecycle status, and it anchors the agent's traces +and evaluation results. It answers "what agents exist, who owns them, +what are they made of, and how well do they work?" + +The Agent Registry is the third registry in a series, following the +[MCP Server Registry +(RFC-0004)](https://github.com/mlflow/rfcs/blob/main/rfcs/0004-mcp-registry/0004-mcp-registry.md) +and the [Skill Registry +(RFC-0008)](https://github.com/mlflow/rfcs/blob/main/rfcs/0008-mvp-skill-registry/0008-mvp-skill-registry.md). +It completes a progression: MLflow can govern the tools an agent calls +(MCP servers), the expertise it carries (skills), and the models it +invokes (Model Registry), but the agent itself, the thing that acts, +has no registry entry. This RFC adds one. + +**The registry is record-level, not runtime-aware.** It stores what an +agent is: who built it, what components it is made of, where its +source lives, and what lifecycle state it is in. It does not store +whether the agent is running, healthy, or scaled. Runtime state +belongs to the serving platform. A platform that deploys registered +agents can join its own runtime inventory against registry records at +query time; the registry supplies the governance half of that join. +This is the same division of responsibility RFC-0004 draws between +the MCP registry and a gateway. + +The registry manages two primary entities under the `mlflow.genai` +SDK namespace, following the pattern of RFC-0004 and RFC-0008: + +- **Agent**: a named, owned entity with DNS-style naming + (`org/agent-name`), in the spirit of the namespaced names the MCP + and Skill registries use; exact alignment with RFC-0008's + `{workspace, organization, name}` coordinates is a detailed-design + point. +- **AgentVersion**: an immutable snapshot of the agent's composition, + its **bill of materials (BOM)**: skill references, MCP server + references, model references (registry models or external model + identifiers such as `gpt-4o`), and, for agents that run as + configurations of a packaged harness, a harness reference (a + proposed axis; see [Open questions](#open-questions)). Each + version also carries at least one **definitional anchor**: source + provenance (Git URI plus commit), an immutable configuration + snapshot, or an A2A Agent Card. Each change to composition is a + new version. + +BOM entries are soft references, structured values rather than +foreign keys. They resolve against the Skill Registry, MCP Server +Registry, and Model Registry when matching entries exist, and they +remain valid when they do not. This is what makes cross-registry +questions ("which agents use skill X?") answerable as registry +queries while keeping registration decoupled from registration order. + +**The BOM is a component inventory, not a complete recipe.** Its +structured axes exist because corresponding registries or identifier +conventions exist, so it is bounded by MLflow's governance surface +rather than by agent anatomy: an agent's prompt strategy or memory +configuration has no axis because nothing governs one. Three layers +share the job of describing an agent. Structured BOM references are +selective but queryable across agents. Definitional anchors (a source +commit, a configuration snapshot) are complete but opaque: they +capture everything about one agent without supporting cross-agent +queries. Free-form tags are the catch-all for facts that fit neither, +and there will always be some, because agents acquire new kinds of +parts faster than any schema anticipates. New structured axes are +expected as the governance surface grows. + +**A2A Agent Cards are the canonical interchange payload when +present.** The [A2A protocol](https://a2a-protocol.org/) defines the +Agent Card schema that much of the industry has converged on for +describing an agent's identity, capabilities, and endpoint. Following +the hybrid storage pattern of RFC-0004 (`server_json`) and RFC-0008 +(`plugin.json`), an agent version registered from an Agent Card +preserves the complete card immutably, with MLflow-managed fields +(BOM, lifecycle, tags, aliases) kept outside the canonical payload. +The card is optional: agents that do not speak A2A register with +plain metadata, and gain a card later if they adopt one. Whether the +card should instead be required or synthesized is an open question. + +**Endpoints are access bindings, not version fields.** Some agents +are reachable at a URL (A2A agents inherently; deployed agents +generally), and recording that URL lets the registry drive tracing +and evaluation for agents whose code the user cannot run. But +endpoints change independently of composition, and agent versions are +immutable. Following RFC-0004's `MCPAccessBinding` model, approved +endpoints are separate mutable binding records that target a version +or alias, created and deleted as connectivity changes without +touching version history. Where an MCP binding's protocol is always +MCP, an agent binding declares its protocol: `a2a`, `mcp` (for +agents exposed as MCP servers), or `other`. Registration accepts an +optional endpoint as a convenience that creates a binding. + +**Agents become the primary anchor for GenAI traces and +evaluations.** Today traces and evaluation runs attach to +experiments, an abstraction that fits model training but not the +agent development loop. This RFC makes an agent a trace +destination: a new `MlflowAgentLocation` joins the existing +`MlflowExperimentLocation`, usable wherever MLflow accepts a trace +destination today, and evaluation and trace-search APIs gain agent +identity alongside experiment identity. Traces and eval results +appear on the agent's registry page, filterable by version. The +change is additive: under the hood an agent owns a default +experiment, the agent location resolves to it, and experiment-based +workflows continue unchanged. + +**Relationship to other RFCs.** RFC-0004 establishes the canonical +payload and access binding patterns this RFC reuses. RFC-0008 defines +the skills and agent plugins that agent BOMs reference; note that an +*agent plugin* (a package of components installed into a harness) is +not an *agent* (an application that acts); this registry governs the +latter. [RFC-0009: Skill +Tracing](https://github.com/mlflow/rfcs/pull/37) annotates spans +within a trace with skill coordinates; this RFC associates whole +traces with an agent version. The two compose: an agent-linked trace +containing `SKILL` spans shows which parts of the agent's BOM were +exercised in a given run. +[RFC-0010: Extended Agent Plugins](https://github.com/mlflow/rfcs/pull/27) +extends plugin composition and is complementary. + +# Basic example + +The API shapes below are illustrative sketches; exact signatures +belong to the detailed design. + +## Register an agent + +```python +import mlflow +from mlflow.genai import GitSource + +mlflow.genai.register_agent( + name="acme/billing-agent", + description="Answers customer billing questions.", + source=GitSource( + url="https://github.com/acme/billing-agent.git", + ref="8f4e2a1", + ), + skills=["skills:/billing-policy/1", "skills:/refund-rules/2"], + mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], + models=["models:/acme-billing-llm/3", "gpt-4o"], +) +``` + +This creates the `Agent` (if new) and an `AgentVersion` with status +`draft`. + +## Register an agent from an A2A Agent Card + +```python +mlflow.genai.register_agent( + name="acme/travel-agent", + a2a_endpoint="https://agents.acme.internal/travel", + skills=["skills:/itinerary-planning/4"], +) +``` + +Given an endpoint, the client SDK fetches the Agent Card from the +endpoint's well-known path, stores it immutably as the canonical +payload and the version's definitional anchor (supplying +description and capability metadata), and creates an `a2a` access +binding for the endpoint. The registry `name` is always chosen by the +registrant: an Agent Card's `name` is a free-form display string, +not an identity, so it feeds presentation rather than naming. +Following RFC-0004's pattern for `server_json["title"]`, the UI +falls back to the card's name when the mutable MLflow-managed +`display_name` is unset, and to the registry `name` after that. The card is whatever the running agent serves; it +need not exist as a static file anywhere. The fetch always happens in +the client: the registry server never fetches user-supplied URLs, +consistent with RFC-0008. A caller that wants to inspect or adjust +card metadata before registering can fetch the card itself and pass +it via `a2a_card=` instead. + +## Register a harness-based agent + +```python +mlflow.genai.register_agent( + name="acme/oncall-helper", + description="On-call assistant run in OpenCode.", + harness="opencode", + harness_version="0.5.3", + skills=["skills:/runbook-triage/2"], + mcp_servers=["mcp-servers:/acme.internal/pagerduty/1.2.0"], + models=["claude-sonnet-5"], + config_snapshot="./opencode.json", +) +``` + +There is no user source repository: the agent is the harness plus its +configuration, so the configuration snapshot serves as the +definitional anchor. `config_snapshot` names a file or directory on +the caller's local disk; the client reads it and uploads the content +to MLflow artifact storage as an immutable artifact on the version. +The registry server never fetches user-supplied locations itself, +consistent with RFC-0008, which also means any redaction of secrets +must happen client-side before upload. + +## Trace and evaluate against the agent + +```python +mlflow.genai.set_active_agent("acme/billing-agent", version=3) + +with mlflow.start_span(name="answer-question"): + result = agent.run(question) + +mlflow.genai.evaluate( + data=eval_dataset, + scorers=[correctness_scorer], + agent_id="acme/billing-agent", +) +``` + +# Motivation + +## The problem + +MLflow can already govern most of the components an agent is built +from, and it can already trace what agents do. What is missing is the +agent itself as a governed entity, and that gap shows up in four +ways. + +1. **Agents have no record.** An agent is built, deployed, and + iterated on with no formal registration: no owner, no lifecycle + state, no accountability chain. Models, MCP servers, and skills + each have registry entries; the agent that composes them, the + thing that acts autonomously on behalf of the organization, has + none. When an agent misbehaves, "who is responsible for this?" is + answered by asking around. + +2. **Composition is untracked.** An agent is a composition of skills, + MCP servers, and models, each versioned independently. No record + captures which versions of which components a given build of the + agent used. When behavior changes, "something changed and the + agent broke; what was it?" requires reconstructing the composition + from memory, commit history, and luck. + +3. **Experiments do not map to agents.** MLflow traces and evaluation + runs attach to experiments. Experiments fit training workflows, + where a run is an attempt at producing a model. They fit the agent + development loop badly: a developer looking at an agent wants its + traces and eval results immediately, not by way of an experiment + wrapper created as ceremony. There is no first-class way to say + "show me this agent's traces, for this version." + +4. **Cross-registry questions require manual inspection.** The Skill + and MCP registries can say a component version is deprecated or + compromised, but nothing records which agents carry it. "Skill X + is compromised; which agents are affected?" is answered today by + inspecting every deployment individually. The registries hold the + component half of the answer; the consumer half is missing. + +These problems compound in regulated environments, where the +accountability and blast-radius gaps are commonly reported as +blockers for putting agents into production. + +## User journeys + +These journeys define the MVP scope. They are written against the +SDK, CLI, and UI surfaces the registry will offer; exact API shapes +are illustrative. + +### Register an agent + +A developer (or a CI/CD pipeline) has built an agent and wants it on +the record. + +1. Register the agent, supplying identity, provenance, and + composition: + ```python + mlflow.genai.register_agent( + name="acme/billing-agent", + description="Answers customer billing questions.", + source=GitSource( + url="https://github.com/acme/billing-agent.git", + ref="8f4e2a1", + ), + skills=["skills:/billing-policy/1"], + mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], + models=["models:/acme-billing-llm/3"], + ) + ``` + Required: name, description, and at least one definitional + anchor: source provenance, a configuration snapshot, or an A2A + Agent Card. Composition (the BOM) is required for source- and + config-anchored registrations and may be empty or partial for + card-anchored ones (see below). Optional: tags and an endpoint + (a URL plus protocol; the endpoint journey below covers + bindings). +2. MLflow creates an `AgentVersion` record with initial status + `draft`. +3. The agent appears in the registry listing for its workspace, with + its BOM entries linked to the Skill, MCP, and Model registry pages + where matching entries exist. +4. **A2A path:** an agent that has an Agent Card registers by + importing it. The UI registration form offers two modes, "import + from A2A card" and "manual"; the import mode pre-fills + descriptive and capability fields from the card, stores the card + as the immutable canonical payload, and creates an `a2a` access + binding for the card's endpoint. The registry `name` is supplied by the + registrant on this path too: the card's free-form name is + presentation, not identity, and reaches the UI through the + display-name fallback chain (RFC-0004's `title` pattern). In the SDK and CLI, the client fetches the + card from the agent's endpoint; in the UI, the card is fetched by + the browser when the endpoint permits it, or pasted, since the + registry server never fetches user-supplied URLs (consistent with + RFC-0008). The BOM is supplied alongside the card, since the card + schema does not carry component version pins. +5. **CI path:** the same call runs from a pipeline, registering a new + version on each release build with the source ref set to the + build's commit. +6. **Harness path:** an agent that runs as a configuration of a + packaged harness (Claude Code, OpenCode, Goose, and similar; see + [RFC-0009](https://github.com/mlflow/rfcs/pull/37) for the + harness/framework distinction) typically has no source repository + of its own: the agent *is* the harness plus its configuration. It + registers with a harness reference in place of user source, and + attaches the configuration that defines it: + ```python + mlflow.genai.register_agent( + name="acme/oncall-helper", + description="On-call assistant run in OpenCode.", + harness="opencode", + harness_version="0.5.3", + skills=["skills:/runbook-triage/2"], + mcp_servers=["mcp-servers:/acme.internal/pagerduty/1.2.0"], + models=["claude-sonnet-5"], + config_snapshot="./opencode.json", + ) + ``` + The configuration snapshot is read from the caller's local disk + and stored as an immutable artifact on the version. It is what + distinguishes this agent from every other installation of the + same harness: enabled tools, overridden defaults (for example, a + non-standard shell for the harness's shell tool), and behavioral + settings live there and nowhere else. + + A harness's configuration surface is not always a single file: + Claude Code, for example, spreads behavioral configuration across + a settings file, instruction files, and subagent definitions. + `config_snapshot` therefore accepts a file or a directory. What it + should capture is the configuration the registry does not + otherwise represent; content that BOM references already govern, + such as installed skill directories or MCP server definitions, + should stay out of the snapshot, because an embedded copy is + invisible to cross-registry queries and can silently drift from + the declared references. + +Composition is required wherever it is knowable, because the BOM is +the value: a registry record without composition is just a name in +a list. A registrant anchoring on source or a configuration +snapshot has the composition in front of them, so the requirement +holds there. A card-anchored registration is different: the +registrant of a vendor or partner agent cannot know which skills, +servers, and models are inside a black box, and forcing a +declaration would invite invented BOMs that pollute cross-registry +queries. For card-anchored registrations the BOM may therefore be +empty or partial, and an absent BOM is recorded as *undeclared* +composition, not as an empty dependency list. Everything else is +progressive enrichment. The endpoint URL is optional; the +trace-and-eval journey below explains which agents need one. When +provided, the URL becomes an access binding rather than a field on +the immutable version. + +The harness path is the newest part of this design and the least +settled (see [Open questions](#open-questions)). For framework-built +and custom agents, the source commit is the natural complete record +and remains the expected anchor. For harness-based agents, requiring +source would force registrations that point at the harness vendor's +repository, which identifies nothing about the specific agent, while +the configuration that actually defines it went unrecorded. +Configuration files frequently embed secrets and environment-specific +values, so the snapshot mechanism needs redaction guidance at +minimum. + +### Publish and maintain an agent's endpoint + +A platform operator deploys a registered agent and needs the +registry to say where, and how, it can be reached, without +disturbing the immutable version history. + +1. The agent is already registered (any path above) and a version + has been promoted. The platform team deploys it. +2. The operator creates an access binding for the deployment, + targeting a version or an alias and declaring the endpoint's + protocol: + ```python + mlflow.genai.create_agent_access_binding( + agent="acme/billing-agent", + target_alias="production", + endpoint_url="https://agents.acme.internal/billing", + protocol="a2a", + ) + ``` + The endpoint accepted at registration time is sugar for creating + a binding; the A2A registration path creates an `a2a` binding + automatically. +3. The agent's detail page lists its bindings. Bindings whose + protocol is self-describing (`a2a`, `mcp`) are actionable: they + are the entry points for the endpoint-driven tracing and + evaluation in the trace-and-eval journey below. An `other` + binding is a documented pointer. +4. The deployment moves to a new URL. The operator updates the + binding; no version record changes. +5. The deployment is retired. The operator deletes the binding; the + agent, its versions, and its history remain untouched. + +The protocol field is where agent bindings depart from RFC-0004, +whose bindings are always MCP and vary only by transport. Agent +endpoints speak different protocols, and the field is limited to +values that tell a caller something actionable: `a2a` and `mcp` are +self-describing (an Agent Card at the well-known path; the MCP +handshake), so URL plus protocol is enough to connect. Labels like +REST or gRPC name a transport without telling anyone how to call +the agent, so they are deliberately collapsed into `other`, which +records where an agent lives without claiming MLflow can invoke it. +This keeps the registry open to custom protocols without pretending +to understand them. As in RFC-0004, a binding that targets an alias +such as `production` follows the alias as it moves between +versions. + +### Version an agent and compare bills of materials + +A developer iterating on an agent changes its composition and needs +to see exactly what changed between two builds. + +1. The developer updates the agent: bumps a skill version, adds an + MCP server, or swaps a model. +2. They register the updated composition, producing a new + `AgentVersion`. Each version is an immutable BOM snapshot; there + is no in-place edit of composition. +3. In the UI, they open the agent's detail page, select two versions + on the Versions tab, and choose Compare. +4. The comparison shows a side-by-side BOM diff: for example, + `billing-policy` skill `1` → `2`, model `llama-3.1-70b` → + `llama-3.1-405b`, one MCP server added. When both versions carry + configuration snapshots, those diff as content: for example, v4 + switched the harness's shell configuration. +5. Changed components link to their Skill Registry and MCP Server + Registry entries, where their own version histories and changelogs + live. + +Immutable versions are what make the diff trustworthy: the comparison +reflects what was registered, not what a mutable record has drifted +into. Combined with the evaluation comparison in the next journey, +this answers the two-part question every regression investigation +starts with: what changed, and did it matter? + +### Develop against agent-centric traces and evaluations + +A developer evaluating agent quality wants traces and eval results +organized by agent and version, not by experiment. + +1. Log traces against the agent instead of an experiment: + ```python + mlflow.genai.set_active_agent("acme/billing-agent", version=3) + + with mlflow.start_span(name="answer-question"): + result = agent.run(question) + ``` + `set_active_agent` is convenience over MLflow's existing trace + destination mechanism: it wraps + `mlflow.tracing.set_destination(MlflowAgentLocation(...))`, the + way `set_experiment` establishes an experiment destination + today. The location can also be passed per span through + `start_span`'s existing `trace_destination` parameter. Framework + and harness autologgers respect the active destination, so + instrumented applications need only state which agent they are. +2. Run evaluations against the agent: + ```python + mlflow.genai.evaluate(data=eval_dataset, + scorers=[correctness_scorer], + agent_id="acme/billing-agent") + ``` +3. Open the agent's detail page. A Traces tab shows the agent's + traces, filterable by version; an Evaluations tab shows eval runs; + a summary card shows latest eval score and trace volume. +4. Compare versions: select v2 and v3, see score deltas and regressed + cases, and drill from a regressed case into its trace to identify + the cause, cross-referencing the BOM diff from the previous + journey. + +Backward compatibility is preserved by construction: an agent owns a +default experiment under the hood, and the agent location resolves +to that experiment's ID. Where MLflow already accepts a typed +destination or location (trace destinations, `search_traces` +locations), agent identity becomes a new accepted value; where it +does not (`evaluate`), `agent_id` is new, optional surface. Existing +experiment-based workflows (including model training and +fine-tuning) continue unchanged. The change is additive, not a data +model rewrite. + +No endpoint is needed for any of this when the developer has the +agent's code: the agent runs locally or in CI, autologging captures +traces during execution, and evaluation scores outputs against a test +dataset. This is why code-only registration supports the full +trace-and-eval workflow. The exception is agents whose code the user +cannot run: another team's A2A agent, a vendor agent, a partner +service. For those, the endpoint is the only execution surface, and +tracing and evaluation work by invoking the agent's access binding +with test inputs and observing responses. This track requires a +binding whose protocol MLflow can speak: `a2a` (invoked through the +card's declared interface) or `mcp` (through the MCP handshake). An +`other` binding records where the agent lives but does not by +itself tell MLflow how to call it, so it does not enable +endpoint-driven tracing or evaluation. The registry supports both +tracks; the optional endpoint exists largely for the second. + +Where [RFC-0009](https://github.com/mlflow/rfcs/pull/37) annotates +spans inside a trace with the skill that produced them, this journey +attaches the whole trace to the agent that ran. Together they give +component-level attribution within agent-level organization: from an +agent's trace list, the `SKILL` spans inside a trace show which BOM +entries were actually exercised. + +### Manage an agent's lifecycle + +An agent owner or platform team needs agents to carry an explicit, +auditable lifecycle state. + +1. An agent version starts as `draft`: visible in the registry while + the owner iterates. +2. The owner runs evaluations and reviews scores and traces (previous + journey). +3. Satisfied with quality, the owner promotes the version to + `active`, manually or from CI. Promotion is informed by evals but + not gated on them; the registry records the decision, it does not + make it. +4. When a version is superseded or found vulnerable, it transitions + to `deprecated`: still visible, marked as superseded, discouraged + from new use. +5. Every transition is recorded as an auditable event with a + timestamp and an actor, whether the actor is a human or a CI/CD + identity. + +This is the same core `draft` → `active` → `deprecated` lifecycle +the MCP and Skill registries use (their soft-delete `deleted` state +and transition rules carry over as well), applied to the agent +itself. The +auditable transition history is the accountability chain that problem +1 identifies as missing: for any agent, the registry can say who +promoted it, when, and what its evaluation evidence looked like at +the time. + +### Assess the blast radius of a compromised component + +A security engineer learns a component is compromised and must find +every affected agent without inspecting deployments one by one. + +1. The skill `k8s-troubleshooter` version `1` is flagged as + compromised. +2. The engineer transitions that skill version to `deprecated` in the + Skill Registry. +3. They query the Agent Registry for consumers of the compromised + version: + ```python + versions = mlflow.genai.search_agent_versions( + filter_string=( + "bom.skill.name = 'k8s-troubleshooter' " + "AND bom.skill.version = 1" + ) + ) + ``` + Dropping the version clause widens the query to consumers of any + version, useful when every version of the skill is suspect or when + assembling the full consumer list before deciding who is affected. +4. The registry returns the affected agent versions with their owners: + for example, three agents across two teams. +5. The engineer contacts the owning teams, who ship new agent + versions with the skill removed or upgraded, and deprecate the + affected versions (previous journey). + +The same query works for the other BOM axes: "which agents use MCP +server Y whose tool schema changed?" and "which agents call model Z +being retired?" Because BOM entries are structured soft references, +the query is a registry lookup with exact-match semantics rather than +a scan of running infrastructure. Because a BOM holds many refs per +axis, the name and version predicates must bind to the same BOM +entry; matching a name on one skill ref and a version on another +would produce false positives. That same-entry matching requirement +is a store-level obligation for the detailed design, in the way +exact matching on span attributes is for RFC-0009. The registry +answers with consumers and owners; automated notification of those +owners is deliberately not in the MVP (see Out of scope). + +Coverage has two limits a reader of the results should keep in mind: +the query sees only the component types the registry tracks, and +within those, only what registrants declared. Agents with +undeclared composition (black-box, card-anchored registrations) can +never match, so results should surface them alongside matches: "3 +agents declare the compromised skill; 12 more have undeclared +composition." The query replaces infrastructure inspection for the +governed axes; it does not claim completeness beyond them. + +## Out of scope + +The following are explicitly out of scope for this RFC. Several are +natural follow-ons; their exclusion here is sequencing, not +rejection. + +- **Runtime state.** Health, liveness, deployment status, scaling, + and placement are the serving platform's domain. The registry + stores no runtime state and performs no polling or health checks. A + platform's runtime view can join registry records against its own + inventory at query time; the full "which *running* agents are + affected?" question is that join, with this registry supplying the + consumer-and-owner half. +- **Deployment and orchestration.** The registry does not deploy + agents. Access bindings record where an approved endpoint is; they + do not create it. +- **Registry synchronization from deployments.** Auto-registering + agents when they deploy, and keeping BOMs fresh when composition + changes at deploy time, calls for platform-side glue (for example a + Kubernetes controller) pushing to the registry APIs this RFC + defines. Deferred. +- **Auto-discovery of composition.** BOMs are developer-asserted in + the MVP. Inferring actual composition from traces (for example, + from RFC-0009 `SKILL` spans) and notifying owners when assertion + and observation disagree is a compelling follow-on, and the span + data this RFC and RFC-0009 produce is designed to enable it. +- **Detection of unregistered agents.** Surfacing "shadow" agents + running without registry entries requires runtime scanning, + which is platform work built on top of this registry. +- **Automated notifications.** The blast-radius journey ends with the + registry naming owners; notifying them is left to the organization + in the MVP. +- **Agent-to-agent runtime discovery.** A programmatic "find me an + agent that can do X and call it" surface for running agents is a + gateway concern, as is any request routing. +- **Cost attribution.** Per-agent token cost is an observability + rollup over agent-linked traces, not registry metadata. +- **Cross-workspace federation.** Discovery across registries is + future work, potentially via A2A. +- **Discovery for reuse as a first-class journey.** Registry listings + are workspace-scoped and searchable, which gives teams a working + answer to "what do we have?", but curated cross-team browsing + experiences are not an MVP goal. + +# Detailed design + +TBD. + +# Drawbacks + +TBD. + +# Alternatives + +TBD. + +# Adoption strategy + +TBD. + +# Open questions + +- **How canonical is the A2A Agent Card?** This RFC treats the card + as canonical when present and optional overall. Arguments for + requiring it: industry convergence on the schema, and one canonical + payload instead of two metadata shapes. Arguments against: A2A + exists to let independent agents discover and interoperate with + agents they have never met, and a large share of production agents + today are single-agent applications or closed teams of agents that + already know each other, for which the card is a heavy dependency + whose primary purpose does not apply. Requiring it would also force + synthesized cards for manual registrations, and the card schema + carries no BOM, so MLflow-managed composition metadata exists + either way. Part of the same question is how thin a card-only + record may be: this RFC lets a card-anchored registration omit the + BOM (recorded as undeclared composition), since a black-box + agent's components are unknowable to its registrant. Is + canonical-when-present the right landing, and should registration + be able to synthesize a valid card from manual metadata for + export? + +- **Should agent-centric traces and evaluations be a separate RFC?** + The experiments bridge (`agent_id` resolving to a default + experiment) touches tracing APIs, evaluation APIs, and UI surface + area well beyond the registry itself, and there is precedent for + splitting: RFC-0008 defined the Skill Registry and RFC-0009 + followed with skill tracing. The counter-argument is that + agent-anchored traces and evals are the registry's core value; a + registry without them is a list of names. This RFC keeps the + journey in scope on the additive framing above, but if reviewers + prefer a narrower registry RFC, the journey splits cleanly along + the RFC-0008/0009 seam. + +- **Do endpoint records belong in MLflow at all?** The access binding + model resolves the mechanical objections to endpoints (mutability + against immutable versions, staleness on version records), and + RFC-0004 sets the precedent. The remaining objection is + architectural: an endpoint could be considered runtime metadata, and + the record/runtime boundary could place all of it on the platform + side. The position taken here is that "where an approved endpoint + for this agent is" belongs to the record, while "whether anything + answers there" belongs to the platform, and that without endpoints + the registry cannot trace or evaluate agents whose code the user + cannot run. This boundary needs explicit review. A related + sub-question: when an agent is exposed as an MCP server, should an + `mcp` binding cross-reference the MCP Registry entry for the same + endpoint instead of duplicating it? + +- **How should harness-based agents be described?** Agents that run + as configurations of a packaged harness (Claude Code, OpenCode, + Goose) have no user source repository; the agent is the harness + plus its configuration. This RFC proposes three mechanisms for + them, going beyond the source-centric design the journeys otherwise + follow: a harness reference axis in the BOM (an external + identifier, like external model references), an immutable + configuration snapshot stored as a version artifact to serve as the + definitional anchor, and a relaxation of the required fields from + "source" to "at least one definitional anchor" (source, + configuration snapshot, or A2A card). Sub-questions: + + - Do configuration snapshots invite secret leakage badly enough to + need enforced redaction rather than guidance? + - Should a harness axis wait for some notion of harness identity + governance? + - What are the identifier semantics of a harness reference? The + options are an opaque asserted name/version pair; an open + vocabulary in which free-text names are always accepted but + names matching MLflow's known harness integrations are + recognized and normalized; or a resolvable package identity. The + open vocabulary is the likely landing: fragmented spellings + would undermine cross-agent queries, while no single package + ecosystem could serve as an authority given how heterogeneously + harnesses are distributed. + - Should the set of files that constitutes a harness's + configuration surface be defined by per-harness integrations + (the harness integrations contemplated by RFC-0009 would be a + natural home) rather than hand-picked by each registrant, given + that the surface must also exclude content the BOM already + governs? + - Does configuration-as-artifact belong in this RFC or a + follow-on? + +- **What is the BOM reference format?** The journeys sketch URI-style + references (`skills:/billing-policy/1`, + `mcp-servers:/acme.internal/payments-db/2.0.0`, + `models:/acme-billing-llm/3`). The skill scheme follows RFC-0008's + member references. RFC-0004 defines no MCP URI scheme, so the MCP + scheme adopts the `mcp-servers:/` proposal from RFC-0010; the refs + here respect RFC-0004's reverse-DNS server names and semantic + versions. Bare identifiers cover external models (`gpt-4o`). BOM + references are soft (string-resolved, valid when + the target is unregistered), which URI syntax may misleadingly + suggest otherwise. Alternatives include structured + `{registry, name, version}` objects. Relatedly: should the BOM be + able to reference agent plugins (RFC-0008/RFC-0010) as composed + units, and should it be able to reference other agents to represent + multi-agent systems? From 56b7bf633e4d159fd168a92d721eeb504f7880c2 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Mon, 24 Aug 2026 16:32:53 -0400 Subject: [PATCH 02/18] Fill in rfc_pr link for RFC-0011 Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index a311f2f..5330771 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -3,7 +3,7 @@ | start_date | 2026-08-24 | | :----------- | :--------- | | mlflow_issue | | -| rfc_pr | | +| rfc_pr | https://github.com/mlflow/rfcs/pull/39 | | Author(s) | [Bill Murdock](https://github.com/jwm4) (Red Hat) | | :--------------------- | :-- | From 8541375f982fdb7c6e9f9491570d81bcb3b977f4 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Tue, 25 Aug 2026 13:51:29 -0400 Subject: [PATCH 03/18] Promote agent plugin references to a BOM axis Plugins are referenced as composed units and expand through their RFC-0008-recorded members for cross-registry queries, so blast-radius lookups find agents that consume a skill via a plugin. Previously held as an open question; resolved by team direction and confirmed by the registry UI prototype's "Link plugin" flow. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 25 ++++++++++++------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 5330771..b11b991 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -62,7 +62,9 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: `{workspace, organization, name}` coordinates is a detailed-design point. - **AgentVersion**: an immutable snapshot of the agent's composition, - its **bill of materials (BOM)**: skill references, MCP server + its **bill of materials (BOM)**: skill references, agent plugin + references (a plugin is referenced as a composed unit and expands + through its registered members for queries), MCP server references, model references (registry models or external model identifiers such as `gpt-4o`), and, for agents that run as configurations of a packaged harness, a harness reference (a @@ -73,7 +75,8 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: new version. BOM entries are soft references, structured values rather than -foreign keys. They resolve against the Skill Registry, MCP Server +foreign keys. They resolve against the Skill Registry (which +RFC-0008 defines for both skills and agent plugins), MCP Server Registry, and Model Registry when matching entries exist, and they remain valid when they do not. This is what makes cross-registry questions ("which agents use skill X?") answerable as registry @@ -164,6 +167,7 @@ mlflow.genai.register_agent( ref="8f4e2a1", ), skills=["skills:/billing-policy/1", "skills:/refund-rules/2"], + agent_plugins=["agent-plugins:/billing-workflow/1.2.0"], mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], models=["models:/acme-billing-llm/3", "gpt-4o"], ) @@ -316,8 +320,8 @@ the record. 2. MLflow creates an `AgentVersion` record with initial status `draft`. 3. The agent appears in the registry listing for its workspace, with - its BOM entries linked to the Skill, MCP, and Model registry pages - where matching entries exist. + its BOM entries linked to the skill, agent plugin, MCP server, + and model registry pages where matching entries exist. 4. **A2A path:** an agent that has an Agent Card registers by importing it. The UI registration form offers two modes, "import from A2A card" and "manual"; the import mode pre-fills @@ -598,7 +602,11 @@ every affected agent without inspecting deployments one by one. The same query works for the other BOM axes: "which agents use MCP server Y whose tool schema changed?" and "which agents call model Z -being retired?" Because BOM entries are structured soft references, +being retired?" Agent plugin references expand through their +members: RFC-0008 plugin versions immutably record which registered +skills they contain, so "which agents use skill X" also finds +agents that consume X through a plugin, rather than leaving a +blind spot behind composed units. Because BOM entries are structured soft references, the query is a registry lookup with exact-match semantics rather than a scan of running infrastructure. Because a BOM holds many refs per axis, the name and version predicates must bind to the same BOM @@ -771,7 +779,6 @@ TBD. references are soft (string-resolved, valid when the target is unregistered), which URI syntax may misleadingly suggest otherwise. Alternatives include structured - `{registry, name, version}` objects. Relatedly: should the BOM be - able to reference agent plugins (RFC-0008/RFC-0010) as composed - units, and should it be able to reference other agents to represent - multi-agent systems? + `{registry, name, version}` objects. Relatedly: should the BOM + also be able to reference other agents, to represent multi-agent + systems? From d7dfe85d4de463847ee6de6dd4a13dfa4e17d57a Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Tue, 25 Aug 2026 13:52:57 -0400 Subject: [PATCH 04/18] Generalize source to RFC-0008 typed source pointers Source provenance now covers Git repos, OCI images, and archives via the typed source pointers RFC-0008 defines, matching how agents are actually distributed, instead of Git-only. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index b11b991..2ccd492 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -70,7 +70,8 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: configurations of a packaged harness, a harness reference (a proposed axis; see [Open questions](#open-questions)). Each version also carries at least one **definitional anchor**: source - provenance (Git URI plus commit), an immutable configuration + provenance (a typed source pointer as in RFC-0008: a Git repo and + ref, an OCI image, or an archive), an immutable configuration snapshot, or an A2A Agent Card. Each change to composition is a new version. @@ -89,7 +90,7 @@ rather than by agent anatomy: an agent's prompt strategy or memory configuration has no axis because nothing governs one. Three layers share the job of describing an agent. Structured BOM references are selective but queryable across agents. Definitional anchors (a source -commit, a configuration snapshot) are complete but opaque: they +pointer, a configuration snapshot) are complete but opaque: they capture everything about one agent without supporting cross-agent queries. Free-form tags are the catch-all for facts that fit neither, and there will always be some, because agents acquire new kinds of @@ -174,7 +175,9 @@ mlflow.genai.register_agent( ``` This creates the `Agent` (if new) and an `AgentVersion` with status -`draft`. +`draft`. `GitSource` is one of the typed source pointers shared +with RFC-0008; an agent distributed as a container image or an +archive registers with the corresponding source type instead. ## Register an agent from an A2A Agent Card @@ -394,8 +397,8 @@ the immutable version. The harness path is the newest part of this design and the least settled (see [Open questions](#open-questions)). For framework-built -and custom agents, the source commit is the natural complete record -and remains the expected anchor. For harness-based agents, requiring +and custom agents, the registered source is the natural complete +record and remains the expected anchor. For harness-based agents, requiring source would force registrations that point at the harness vendor's repository, which identifies nothing about the specific agent, while the configuration that actually defines it went unrecorded. From ae905396bf1704bded2c313eec0f4eb2e5408dbe Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Tue, 25 Aug 2026 13:56:40 -0400 Subject: [PATCH 05/18] State the version identity policy for agent versions Agent versions are registry-minted serial numbers, per the policy from the earlier registries: adopt the artifact's inherent version when its format defines one, mint serial numbers when it does not. No standard agent artifact defines one; an A2A card's provider-defined version string is preserved as metadata, not adopted as identity. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 2ccd492..3370673 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -75,6 +75,18 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: snapshot, or an A2A Agent Card. Each change to composition is a new version. +Agent versions are registry-minted serial numbers (1, 2, 3), +following the policy the earlier registries establish: a registry +entry adopts the underlying artifact's own version when its format +defines one (RFC-0004's `server_json`, RFC-0008's plugin manifests) +and mints serial numbers when it does not (RFC-0008's skills, whose +format defines no version field). No standard agent artifact defines +an inherent agent version. An A2A Agent Card carries a +provider-defined `version` string, but the card is optional and the +A2A specification leaves that field's format to the provider, so it +is preserved in the canonical payload as descriptive metadata rather +than adopted as version identity. + BOM entries are soft references, structured values rather than foreign keys. They resolve against the Skill Registry (which RFC-0008 defines for both skills and agent plugins), MCP Server From 6bf4e1564ea1f5d328f3c3023d55f79d12d1b743 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Tue, 25 Aug 2026 15:40:35 -0400 Subject: [PATCH 06/18] Allow multiple typed sources per agent version A version's source provenance is now a list of typed pointers, so an agent built from a Git repo and shipped as an OCI image records both, and source entries join the queryable axes ("which agents ship image X?"). Deliberate divergence from RFC-0008's one-source-per-version rule, with the rationale (agents have no content digest to reconcile per-source duplicates) stated in Open questions. Prompted by Ann Marie's review question. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 48 ++++++++++++++----- 1 file changed, 35 insertions(+), 13 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 3370673..cd3bd40 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -70,9 +70,9 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: configurations of a packaged harness, a harness reference (a proposed axis; see [Open questions](#open-questions)). Each version also carries at least one **definitional anchor**: source - provenance (a typed source pointer as in RFC-0008: a Git repo and - ref, an OCI image, or an archive), an immutable configuration - snapshot, or an A2A Agent Card. Each change to composition is a + provenance (one or more typed source pointers as in RFC-0008: a + Git repo and ref, an OCI image, an archive), an immutable + configuration snapshot, or an A2A Agent Card. Each change to composition is a new version. Agent versions are registry-minted serial numbers (1, 2, 3), @@ -175,10 +175,13 @@ from mlflow.genai import GitSource mlflow.genai.register_agent( name="acme/billing-agent", description="Answers customer billing questions.", - source=GitSource( - url="https://github.com/acme/billing-agent.git", - ref="8f4e2a1", - ), + sources=[ + GitSource( + url="https://github.com/acme/billing-agent.git", + ref="8f4e2a1", + ), + OciSource("quay.io/acme/billing-agent@sha256:9f2c1e"), + ], skills=["skills:/billing-policy/1", "skills:/refund-rules/2"], agent_plugins=["agent-plugins:/billing-workflow/1.2.0"], mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], @@ -187,9 +190,12 @@ mlflow.genai.register_agent( ``` This creates the `Agent` (if new) and an `AgentVersion` with status -`draft`. `GitSource` is one of the typed source pointers shared -with RFC-0008; an agent distributed as a container image or an -archive registers with the corresponding source type instead. +`draft`. `GitSource` and `OciSource` are typed source pointers as +in RFC-0008. A version may record multiple sources; here, the Git +repo the agent is built from and the container image it ships as. +Unlike skill versions, which carry exactly one source, agent +versions allow several (see [Open questions](#open-questions) for +the rationale). ## Register an agent from an A2A Agent Card @@ -316,10 +322,10 @@ the record. mlflow.genai.register_agent( name="acme/billing-agent", description="Answers customer billing questions.", - source=GitSource( + sources=[GitSource( url="https://github.com/acme/billing-agent.git", ref="8f4e2a1", - ), + )], skills=["skills:/billing-policy/1"], mcp_servers=["mcp-servers:/acme.internal/payments-db/2.0.0"], models=["models:/acme-billing-llm/3"], @@ -617,7 +623,9 @@ every affected agent without inspecting deployments one by one. The same query works for the other BOM axes: "which agents use MCP server Y whose tool schema changed?" and "which agents call model Z -being retired?" Agent plugin references expand through their +being retired?" Source entries are queryable the same way: "which +agents ship OCI image X?" is the container-CVE variant of this +journey. Agent plugin references expand through their members: RFC-0008 plugin versions immutably record which registered skills they contain, so "which agents use skill X" also finds agents that consume X through a plugin, rather than leaving a @@ -783,6 +791,20 @@ TBD. - Does configuration-as-artifact belong in this RFC or a follow-on? +- **Should a version record multiple sources?** This RFC says yes: a + version's source provenance is a list of typed pointers, so an + agent built from a Git repo and shipped as an OCI image records + both, and source entries are queryable like BOM axes ("which + agents ship image X?"). This deliberately diverges from RFC-0008, + where each skill version has exactly one source and the content + digest reconciles identical content registered from different + sources. Agents have no defined content bundle to digest, so the + skill pattern applied to agents would produce irreconcilable + duplicate versions of what is really one agent. The list is + currently an unordered set of asserted pointers. Reviewers who + weigh cross-registry consistency + heavily should push back here if the divergence is not worth it. + - **What is the BOM reference format?** The journeys sketch URI-style references (`skills:/billing-policy/1`, `mcp-servers:/acme.internal/payments-db/2.0.0`, From 9ffdbb6904ce4a467ff1226d7dda0227b28d0631 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Tue, 25 Aug 2026 16:36:53 -0400 Subject: [PATCH 07/18] Add per-agent version schemes with monotonic default An Agent chooses its version scheme at creation: monotonic (registry-assigned serial numbers, the default when no version is supplied), semver, or freeform, with possible autodetection from the first registration. Keeps the simple path as simple as the Skill Registry while letting provider-versioned agents keep their own versioning. Proposed by Matthew Prahl in review discussion. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 37 ++++++++++++------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index cd3bd40..405f812 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -75,17 +75,26 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: configuration snapshot, or an A2A Agent Card. Each change to composition is a new version. -Agent versions are registry-minted serial numbers (1, 2, 3), -following the policy the earlier registries establish: a registry -entry adopts the underlying artifact's own version when its format -defines one (RFC-0004's `server_json`, RFC-0008's plugin manifests) -and mints serial numbers when it does not (RFC-0008's skills, whose -format defines no version field). No standard agent artifact defines -an inherent agent version. An A2A Agent Card carries a -provider-defined `version` string, but the card is optional and the -A2A specification leaves that field's format to the provider, so it -is preserved in the canonical payload as descriptive metadata rather -than adopted as version identity. +Version identity is a per-agent choice made when the agent is +created: `monotonic` (registry-assigned serial numbers: 1, 2, 3), +`semver` (registrant-supplied semantic versions), or `freeform` +(registrant-supplied opaque strings). `monotonic` is the default +when registration supplies no version, so the simple path stays as +simple as the Skill Registry's: registrants who never think about +versioning get serial numbers automatically, while an agent that +already carries its own versioning (a team's release train, a +provider-versioned A2A agent) can keep it. The scheme may also be +autodetected from the first registration's input. This refines the +policy the earlier registries establish, where an entry adopts the +underlying artifact's version when its format defines one +(RFC-0004's `server_json`, RFC-0008's plugin manifests) and mints +serial numbers when it does not (RFC-0008's skills): no standard +agent artifact defines an inherent agent version, so minting is the +default rather than the rule. An A2A card's provider-defined +`version` string is preserved in the canonical payload either way; +an agent registered under `semver` or `freeform` may mirror it. +Aliases and `latest` resolution are well defined for `monotonic` +and `semver`; `freeform` versions order by registration time. BOM entries are soft references, structured values rather than foreign keys. They resolve against the Skill Registry (which @@ -335,8 +344,10 @@ the record. anchor: source provenance, a configuration snapshot, or an A2A Agent Card. Composition (the BOM) is required for source- and config-anchored registrations and may be empty or partial for - card-anchored ones (see below). Optional: tags and an endpoint - (a URL plus protocol; the endpoint journey below covers + card-anchored ones (see below). Optional: tags, an explicit + version where the agent's version scheme takes one (the default + `monotonic` scheme assigns versions automatically), and an + endpoint (a URL plus protocol; the endpoint journey below covers bindings). 2. MLflow creates an `AgentVersion` record with initial status `draft`. From bfd3abb1801a67300711edcf7c10cfba27e74d4b Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 10:57:59 -0400 Subject: [PATCH 08/18] Fetch Agent Cards on demand instead of storing them The agent's endpoint is the card's system of record: registration imports descriptive metadata (card name seeds display_name) and creates the a2a binding, the UI renders the card read-only by fetching through the binding, and no card payload is persisted. This deliberately departs from the RFC-0004/0008 canonical-payload pattern. Endpoint-only registrations become interface-only records with no definitional anchor, marked as such. The canonicity open question is replaced by whether registration and lifecycle events should capture card copies as audit evidence. Direction from design-doc review comments, confirmed by Bill. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 175 ++++++++++-------- 1 file changed, 97 insertions(+), 78 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 405f812..24ec86a 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -71,9 +71,10 @@ SDK namespace, following the pattern of RFC-0004 and RFC-0008: proposed axis; see [Open questions](#open-questions)). Each version also carries at least one **definitional anchor**: source provenance (one or more typed source pointers as in RFC-0008: a - Git repo and ref, an OCI image, an archive), an immutable - configuration snapshot, or an A2A Agent Card. Each change to composition is a - new version. + Git repo and ref, an OCI image, an archive) and/or an immutable + configuration snapshot. An agent registered from an A2A endpoint + alone is an interface-only record with no anchor (see below). + Each change to composition is a new version. Version identity is a per-agent choice made when the agent is created: `monotonic` (registry-assigned serial numbers: 1, 2, 3), @@ -91,7 +92,7 @@ underlying artifact's version when its format defines one serial numbers when it does not (RFC-0008's skills): no standard agent artifact defines an inherent agent version, so minting is the default rather than the rule. An A2A card's provider-defined -`version` string is preserved in the canonical payload either way; +`version` string lives on the live card rather than in the registry; an agent registered under `semver` or `freeform` may mirror it. Aliases and `latest` resolution are well defined for `monotonic` and `semver`; `freeform` versions order by registration time. @@ -118,17 +119,27 @@ and there will always be some, because agents acquire new kinds of parts faster than any schema anticipates. New structured axes are expected as the governance surface grows. -**A2A Agent Cards are the canonical interchange payload when -present.** The [A2A protocol](https://a2a-protocol.org/) defines the -Agent Card schema that much of the industry has converged on for -describing an agent's identity, capabilities, and endpoint. Following -the hybrid storage pattern of RFC-0004 (`server_json`) and RFC-0008 -(`plugin.json`), an agent version registered from an Agent Card -preserves the complete card immutably, with MLflow-managed fields -(BOM, lifecycle, tags, aliases) kept outside the canonical payload. -The card is optional: agents that do not speak A2A register with -plain metadata, and gain a card later if they adopt one. Whether the -card should instead be required or synthesized is an open question. +**A2A Agent Cards are fetched, not stored.** The +[A2A protocol](https://a2a-protocol.org/) makes an agent's endpoint +the authoritative home of its Agent Card: the live card is always +one GET away at the endpoint's well-known path, and every A2A +client reads it from there. The registry follows suit. Registering +from an endpoint imports the card's descriptive metadata +(description, capabilities, and its free-form name, which seeds the +mutable MLflow-managed `display_name`) into ordinary registry +fields and creates an `a2a` access binding; the card content itself +is not persisted. The UI renders the card read-only by fetching it +through the binding at view time, so what MLflow displays can never +drift from what the agent serves. This deliberately departs from +the canonical-payload pattern of RFC-0004 (`server_json`) and +RFC-0008 (`plugin.json`): MLflow is the system of record for those +payloads, while an Agent Card's system of record is the agent +itself. An agent registered from an endpoint alone, with no source +and no configuration snapshot, is an **interface-only record**: the +registry captures the claim surface (identity, imported metadata, +endpoint) and marks that it holds no definitional anchor. Whether +registration and lifecycle events should optionally capture a card +copy as audit evidence is an open question. **Endpoints are access bindings, not version fields.** Some agents are reachable at a URL (A2A agents inherently; deployed agents @@ -156,8 +167,9 @@ change is additive: under the hood an agent owns a default experiment, the agent location resolves to it, and experiment-based workflows continue unchanged. -**Relationship to other RFCs.** RFC-0004 establishes the canonical -payload and access binding patterns this RFC reuses. RFC-0008 defines +**Relationship to other RFCs.** RFC-0004 establishes the access +binding pattern this RFC reuses (its canonical-payload pattern is +deliberately not applied to Agent Cards; see above). RFC-0008 defines the skills and agent plugins that agent BOMs reference; note that an *agent plugin* (a package of components installed into a harness) is not an *agent* (an application that acts); this registry governs the @@ -217,20 +229,22 @@ mlflow.genai.register_agent( ``` Given an endpoint, the client SDK fetches the Agent Card from the -endpoint's well-known path, stores it immutably as the canonical -payload and the version's definitional anchor (supplying -description and capability metadata), and creates an `a2a` access -binding for the endpoint. The registry `name` is always chosen by the -registrant: an Agent Card's `name` is a free-form display string, -not an identity, so it feeds presentation rather than naming. -Following RFC-0004's pattern for `server_json["title"]`, the UI -falls back to the card's name when the mutable MLflow-managed -`display_name` is unset, and to the registry `name` after that. The card is whatever the running agent serves; it -need not exist as a static file anywhere. The fetch always happens in -the client: the registry server never fetches user-supplied URLs, -consistent with RFC-0008. A caller that wants to inspect or adjust -card metadata before registering can fetch the card itself and pass -it via `a2a_card=` instead. +endpoint's well-known path, imports its descriptive metadata +(description, capabilities, and its free-form name, which seeds the +mutable MLflow-managed `display_name`), and creates an `a2a` access +binding for the endpoint. The card content is not persisted: the +endpoint is the card's system of record, and the UI renders the +card read-only by fetching it through the binding at view time. The +registry `name` is always chosen by the registrant, since a card's +`name` is a display string, not an identity. The card is whatever +the running agent serves; it need not exist as a static file +anywhere. Fetches always happen in the client or the browser: the +registry server never fetches user-supplied URLs, consistent with +RFC-0008. A caller that wants to inspect or adjust the imported +metadata before registering can fetch the card itself and pass it +via `a2a_card=` instead. A version registered this way, with no +source and no configuration snapshot, is an interface-only record +(see the register journey). ## Register a harness-based agent @@ -340,11 +354,14 @@ the record. models=["models:/acme-billing-llm/3"], ) ``` - Required: name, description, and at least one definitional - anchor: source provenance, a configuration snapshot, or an A2A - Agent Card. Composition (the BOM) is required for source- and - config-anchored registrations and may be empty or partial for - card-anchored ones (see below). Optional: tags, an explicit + Required: name, description, and at least one of: a definitional + anchor (source provenance or a configuration snapshot) or an A2A + endpoint. These combine freely; a first-party A2A agent registers + with both source and endpoint. Only a registration with an + endpoint and no anchor produces an interface-only record (see + below). Composition (the BOM) is required when an anchor is + present and may be empty or partial for interface-only records + (see below). Optional: tags, an explicit version where the agent's version scheme takes one (the default `monotonic` scheme assigns versions automatically), and an endpoint (a URL plus protocol; the endpoint journey below covers @@ -354,20 +371,21 @@ the record. 3. The agent appears in the registry listing for its workspace, with its BOM entries linked to the skill, agent plugin, MCP server, and model registry pages where matching entries exist. -4. **A2A path:** an agent that has an Agent Card registers by - importing it. The UI registration form offers two modes, "import +4. **A2A path:** an agent that serves an Agent Card registers from + its endpoint. The UI registration form offers two modes, "import from A2A card" and "manual"; the import mode pre-fills - descriptive and capability fields from the card, stores the card - as the immutable canonical payload, and creates an `a2a` access - binding for the card's endpoint. The registry `name` is supplied by the - registrant on this path too: the card's free-form name is - presentation, not identity, and reaches the UI through the - display-name fallback chain (RFC-0004's `title` pattern). In the SDK and CLI, the client fetches the - card from the agent's endpoint; in the UI, the card is fetched by - the browser when the endpoint permits it, or pasted, since the - registry server never fetches user-supplied URLs (consistent with - RFC-0008). The BOM is supplied alongside the card, since the card - schema does not carry component version pins. + descriptive and capability fields from the card (its free-form + name seeds the mutable `display_name`; the registry `name` is + supplied by the registrant) and creates an `a2a` access binding + for the endpoint. The card content is not persisted: the agent + detail page renders the card read-only by fetching it through + the binding, so the display never drifts from what the agent + serves. In the SDK and CLI, the client fetches the card at + import; in the UI, the browser fetches it when the endpoint + permits, or the user pastes it, since the registry server never + fetches user-supplied URLs (consistent with RFC-0008). The BOM + is supplied alongside, since the card schema does not carry + component version pins. 5. **CI path:** the same call runs from a pipeline, registering a new version on each release build with the source ref set to the build's commit. @@ -412,13 +430,16 @@ Composition is required wherever it is knowable, because the BOM is the value: a registry record without composition is just a name in a list. A registrant anchoring on source or a configuration snapshot has the composition in front of them, so the requirement -holds there. A card-anchored registration is different: the -registrant of a vendor or partner agent cannot know which skills, -servers, and models are inside a black box, and forcing a -declaration would invite invented BOMs that pollute cross-registry -queries. For card-anchored registrations the BOM may therefore be -empty or partial, and an absent BOM is recorded as *undeclared* -composition, not as an empty dependency list. Everything else is +holds there. An interface-only registration (an A2A endpoint with +no source or configuration snapshot) is different: the registrant +of a vendor or partner agent cannot know which skills, servers, and +models are inside a black box, and forcing a declaration would +invite invented BOMs that pollute cross-registry queries. For +interface-only records the BOM may therefore be empty or partial, +an absent BOM is recorded as *undeclared* composition, not as an +empty dependency list, and the record itself is marked as holding +no definitional anchor: the registry knows the agent's claim +surface, not its contents. Everything else is progressive enrichment. The endpoint URL is optional; the trace-and-eval journey below explains which agents need one. When provided, the URL becomes an access binding rather than a field on @@ -427,7 +448,8 @@ the immutable version. The harness path is the newest part of this design and the least settled (see [Open questions](#open-questions)). For framework-built and custom agents, the registered source is the natural complete -record and remains the expected anchor. For harness-based agents, requiring +record and remains the expected anchor. For harness-based agents, +requiring source would force registrations that point at the harness vendor's repository, which identifies nothing about the specific agent, while the configuration that actually defines it went unrecorded. @@ -640,7 +662,8 @@ journey. Agent plugin references expand through their members: RFC-0008 plugin versions immutably record which registered skills they contain, so "which agents use skill X" also finds agents that consume X through a plugin, rather than leaving a -blind spot behind composed units. Because BOM entries are structured soft references, +blind spot behind composed units. Because BOM entries are +structured soft references, the query is a registry lookup with exact-match semantics rather than a scan of running infrastructure. Because a BOM holds many refs per axis, the name and version predicates must bind to the same BOM @@ -654,8 +677,9 @@ owners is deliberately not in the MVP (see Out of scope). Coverage has two limits a reader of the results should keep in mind: the query sees only the component types the registry tracks, and within those, only what registrants declared. Agents with -undeclared composition (black-box, card-anchored registrations) can -never match, so results should surface them alongside matches: "3 +undeclared composition (black-box, interface-only registrations) +can never match, so results should surface them alongside matches: +"3 agents declare the compromised skill; 12 more have undeclared composition." The query replaces infrastructure inspection for the governed axes; it does not claim completeness beyond them. @@ -722,24 +746,19 @@ TBD. # Open questions -- **How canonical is the A2A Agent Card?** This RFC treats the card - as canonical when present and optional overall. Arguments for - requiring it: industry convergence on the schema, and one canonical - payload instead of two metadata shapes. Arguments against: A2A - exists to let independent agents discover and interoperate with - agents they have never met, and a large share of production agents - today are single-agent applications or closed teams of agents that - already know each other, for which the card is a heavy dependency - whose primary purpose does not apply. Requiring it would also force - synthesized cards for manual registrations, and the card schema - carries no BOM, so MLflow-managed composition metadata exists - either way. Part of the same question is how thin a card-only - record may be: this RFC lets a card-anchored registration omit the - BOM (recorded as undeclared composition), since a black-box - agent's components are unknowable to its registrant. Is - canonical-when-present the right landing, and should registration - be able to synthesize a valid card from manual metadata for - export? +- **Should events capture Agent Card evidence?** This RFC treats + the agent's endpoint as the card's system of record: cards are + fetched for display and imported for metadata, never persisted. + That leaves governance without any record of what an agent + claimed at the moment somebody acted on the claim. Should + registration and lifecycle transitions (for example, promotion to + `active`) optionally capture a copy of the card as audit evidence + attached to the event? That would give regulated environments + decision-time evidence without reintroducing a stored payload + that pretends to define the version. Relatedly, how thin may an + interface-only record be before it is not worth registering: is + identity, imported metadata, a binding, and undeclared + composition enough? - **Should agent-centric traces and evaluations be a separate RFC?** The experiments bridge (`agent_id` resolving to a default From 2ad2c82a379a73eed0b4817575e5ad48df46ebaa Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 12:41:20 -0400 Subject: [PATCH 09/18] Record the agent version on evaluation runs Both evaluate sketches now pass agent_version alongside agent_id; version-to-version eval comparison (J3) needs the version captured at evaluation time. From design-doc review feedback. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 24ec86a..6221b2b 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -282,6 +282,7 @@ mlflow.genai.evaluate( data=eval_dataset, scorers=[correctness_scorer], agent_id="acme/billing-agent", + agent_version=3, ) ``` @@ -554,7 +555,8 @@ organized by agent and version, not by experiment. ```python mlflow.genai.evaluate(data=eval_dataset, scorers=[correctness_scorer], - agent_id="acme/billing-agent") + agent_id="acme/billing-agent", + agent_version=3) ``` 3. Open the agent's detail page. A Traces tab shows the agent's traces, filterable by version; an Evaluations tab shows eval runs; From 5b19e673dc6dc35123ea1cce23d4c6c000b9ad0f Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 12:48:09 -0400 Subject: [PATCH 10/18] Drop the card-evidence open question The case for capturing card copies as decision evidence was weak: no customer signal, promotion decisions rest on eval evidence, and registration already imports the card's claim surface into registry fields. The open question is now solely about how thin an interface-only record may be. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 23 ++++++------------- 1 file changed, 7 insertions(+), 16 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 6221b2b..2f5f139 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -137,9 +137,7 @@ payloads, while an Agent Card's system of record is the agent itself. An agent registered from an endpoint alone, with no source and no configuration snapshot, is an **interface-only record**: the registry captures the claim surface (identity, imported metadata, -endpoint) and marks that it holds no definitional anchor. Whether -registration and lifecycle events should optionally capture a card -copy as audit evidence is an open question. +endpoint) and marks that it holds no definitional anchor. **Endpoints are access bindings, not version fields.** Some agents are reachable at a URL (A2A agents inherently; deployed agents @@ -748,19 +746,12 @@ TBD. # Open questions -- **Should events capture Agent Card evidence?** This RFC treats - the agent's endpoint as the card's system of record: cards are - fetched for display and imported for metadata, never persisted. - That leaves governance without any record of what an agent - claimed at the moment somebody acted on the claim. Should - registration and lifecycle transitions (for example, promotion to - `active`) optionally capture a copy of the card as audit evidence - attached to the event? That would give regulated environments - decision-time evidence without reintroducing a stored payload - that pretends to define the version. Relatedly, how thin may an - interface-only record be before it is not worth registering: is - identity, imported metadata, a binding, and undeclared - composition enough? +- **How thin may an interface-only record be?** A black-box A2A + agent registers with a name, metadata imported from its card, an + access binding, undeclared composition, and no definitional + anchor. Is that enough of a record to be worth governing, and + should the registry require anything more of it before such a + record can be promoted to `active`? - **Should agent-centric traces and evaluations be a separate RFC?** The experiments bridge (`agent_id` resolving to a default From c918c25a7cf2cce31516102c53730b547db169be Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 13:22:36 -0400 Subject: [PATCH 11/18] Note that version-vs-new-agent is the registrant's call Two sentences in the versioning journey: the choice is the same judgment developers make for any software, the registry enforces no rule, and the consequence is that version comparison exists only within one agent. From design-doc review discussion. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 2f5f139..5b1f414 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -529,6 +529,13 @@ into. Combined with the evaluation comparison in the next journey, this answers the two-part question every regression investigation starts with: what changed, and did it matter? +Whether a change is a new version or a new agent is the +registrant's call, the same judgment developers already make for +any software: is this a new release of the same application, or a +different application? The registry enforces no rule; the practical +consequence of the choice is that version comparison exists only +within one agent. + ### Develop against agent-centric traces and evaluations A developer evaluating agent quality wants traces and eval results From b7ae6df75c0117f10f5a3d15f557dd494218499a Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 13:26:44 -0400 Subject: [PATCH 12/18] Reframe the versioning journey to avoid overpromising The journey no longer claims the diff reveals what broke and then walks it back. The investigator is framed from the start as someone other than the change author, the diff is a shared record of declared changes ("what differs in the declarations for two versions"), and it gives an investigation its starting facts rather than answers. From design-doc review discussion. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 21 ++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 5b1f414..68bf9a0 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -504,8 +504,10 @@ versions. ### Version an agent and compare bills of materials -A developer iterating on an agent changes its composition and needs -to see exactly what changed between two builds. +A developer iterating on an agent registers each change as a new +version. Later, when behavior shifts, whoever is investigating, and +it is often not the person who made the change, compares two +versions to see exactly what differs. 1. The developer updates the agent: bumps a skill version, adds an MCP server, or swaps a model. @@ -523,11 +525,16 @@ to see exactly what changed between two builds. Registry entries, where their own version histories and changelogs live. -Immutable versions are what make the diff trustworthy: the comparison -reflects what was registered, not what a mutable record has drifted -into. Combined with the evaluation comparison in the next journey, -this answers the two-part question every regression investigation -starts with: what changed, and did it matter? +Immutable versions are what make the diff trustworthy: the +comparison reflects what was registered, not what a mutable record +has drifted into. The diff is a shared record of declared changes: +its value is that a teammate, reviewer, or security engineer can +see what differs in the declarations for two versions without +depending on the change author's memory or availability. Combined +with the evaluation comparison in the next journey, it gives an +investigation its starting facts: what changed, and did it matter? +Discovering changes that nobody declared belongs to the deferred +auto-discovery work (see Out of scope). Whether a change is a new version or a new agent is the registrant's call, the same judgment developers already make for From 5f34e0079d8a5b4e34341f7119d7f5070c5d0dc3 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 15:16:10 -0400 Subject: [PATCH 13/18] Scope trace destinations per deployment New J3 paragraph: the default experiment is a default, not a router. Deployments log wherever their own configuration points; scale-out replicas aggregate by sharing config; deployments needing separation override the destination (destinations, not tags, because permissions are experiment-scoped). Proposes, as not yet settled, that such deployments notify the registry so the experiment ID is recorded on their access binding. Sync bullet gains trace-location link upkeep. From the deployment/experiment-mapping review discussions. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 29 ++++++++++++++++--- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 68bf9a0..bc651ae 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -588,6 +588,26 @@ experiment-based workflows (including model training and fine-tuning) continue unchanged. The change is additive, not a data model rewrite. +The default experiment is a default, not a router. A running agent +logs traces to whatever destination its own deployment +configuration sets; the registry is not in the call path. When +nothing is set, traces land in the agent's default experiment, +which is the right behavior for the development loop and +registry-driven evaluations. Scale-out replicas of one deployment +share its configuration, so their traces aggregate without further +arrangement. A deployment that needs its traces kept separate from +other deployments of the same agent (a different owner or user +base) overrides the destination in its own configuration; because +permissions are experiment-scoped, separation is done with +destinations, not trace tags. To keep such traces findable from the +agent's registry page, this RFC proposes that a deployment using a +non-default experiment notify the registry, which records the +experiment ID on that deployment's access binding. A deployment +with no binding would need another place to record the link, which +is one reason this mechanism is a proposal rather than settled. +Automating the setup and upkeep of these links at deploy time +belongs to the deferred registry synchronization glue. + No endpoint is needed for any of this when the developer has the agent's code: the agent runs locally or in CI, autologging captures traces during execution, and evaluation scores outputs against a test @@ -715,10 +735,11 @@ rejection. agents. Access bindings record where an approved endpoint is; they do not create it. - **Registry synchronization from deployments.** Auto-registering - agents when they deploy, and keeping BOMs fresh when composition - changes at deploy time, calls for platform-side glue (for example a - Kubernetes controller) pushing to the registry APIs this RFC - defines. Deferred. + agents when they deploy, keeping BOMs fresh when composition + changes at deploy time, and maintaining deployment trace-location + links (see the trace journey) calls for platform-side glue (for + example a Kubernetes controller) pushing to the registry APIs + this RFC defines. Deferred. - **Auto-discovery of composition.** BOMs are developer-asserted in the MVP. Inferring actual composition from traces (for example, from RFC-0009 `SKILL` spans) and notifying owners when assertion From 708f9640561e0b7c8f92f9b8b51d0d26621d6004 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 15:23:11 -0400 Subject: [PATCH 14/18] Explain why versions share the agent's experiment One sentence in the J3 deployment paragraph: a version is an analysis dimension recorded on every trace, not an access boundary, and per-version experiments would break the longitudinal view across upgrades. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index bc651ae..9a67877 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -599,7 +599,11 @@ arrangement. A deployment that needs its traces kept separate from other deployments of the same agent (a different owner or user base) overrides the destination in its own configuration; because permissions are experiment-scoped, separation is done with -destinations, not trace tags. To keep such traces findable from the +destinations, not trace tags. Versions share the agent's experiment +for the same reason seen from the other side: a version is an +analysis dimension recorded on every trace, not an access boundary, +and per-version experiments would break the longitudinal view of an +agent's behavior across upgrades. To keep such traces findable from the agent's registry page, this RFC proposes that a deployment using a non-default experiment notify the registry, which records the experiment ID on that deployment's access binding. A deployment From ec342cd00b85521abd5c649a21652f0b9acecd8a Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Wed, 26 Aug 2026 17:45:38 -0400 Subject: [PATCH 15/18] Open the per-agent vs per-version default experiment question The RFC keeps its per-agent position but now acknowledges the fork as an open question with both sides stated, since a reviewer holds the per-version position and the author is not settled. Whichever default is chosen, the other camp overrides per deployment, so the question is which behavior makes the better default. Co-Authored-By: Claude Fable 5 --- rfcs/0011-agent-registry/0011-agent-registry.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 9a67877..3dc9376 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -785,6 +785,20 @@ TBD. # Open questions +- **Is the default experiment per agent or per agent-version?** This + RFC says per agent: a version is an analysis dimension recorded on + every trace, one experiment preserves the longitudinal view of an + agent's behavior across upgrades, and the agent-to-experiment + mapping stays one-to-one, with no version required to resolve a + destination. The opposing position, held by at least one reviewer, + is one default experiment per version, which partitions each + version's traces physically, at the cost of making cross-upgrade + monitoring a cross-experiment query and making the version a + required argument wherever a destination is resolved. Whichever + default is chosen, users who want the other behavior override it + per deployment, so the question is which behavior makes the better + default, not which is possible. + - **How thin may an interface-only record be?** A black-box A2A agent registers with a name, metadata imported from its card, an access binding, undeclared composition, and no definitional From 3efd3cf3d4d1e9702edf98da40b926d8938e8f3c Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Thu, 27 Aug 2026 10:09:39 -0400 Subject: [PATCH 16/18] Clarify that destinations are agent-only and versions are metadata The API sketches implied the version participates in choosing the default experiment. Now stated explicitly in the Summary, basic example, and trace journey: a destination identifies the agent only and resolves to its one default experiment; the version is recorded on every trace and evaluation run as metadata. set_active_agent is defined as convenience over two separable pieces, with get_default_experiment_id exposed as a public agent-only lookup (partially adopting Ann Marie's two-call review suggestion), and deployment overrides swap only the destination piece. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 46 +++++++++++++++---- 1 file changed, 36 insertions(+), 10 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 3dc9376..5c229c0 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -159,8 +159,13 @@ agent development loop. This RFC makes an agent a trace destination: a new `MlflowAgentLocation` joins the existing `MlflowExperimentLocation`, usable wherever MLflow accepts a trace destination today, and evaluation and trace-search APIs gain agent -identity alongside experiment identity. Traces and eval results -appear on the agent's registry page, filterable by version. The +identity alongside experiment identity. A destination identifies +the agent only and resolves to the agent's one default experiment; +the version is never part of the destination and is instead +recorded on every trace and evaluation run as metadata, which is +what per-version filtering and comparison use. Traces and eval +results appear on the agent's registry page, filterable by +version. The change is additive: under the hood an agent owns a default experiment, the agent location resolves to it, and experiment-based workflows continue unchanged. @@ -284,6 +289,12 @@ mlflow.genai.evaluate( ) ``` +`set_active_agent` does two separable things: it points the trace +destination at the agent's one default experiment (the version +plays no part in that), and it records the agent and version as +metadata on everything emitted while it is active. The trace +journey shows the underlying pieces. + # Motivation ## The problem @@ -555,14 +566,25 @@ organized by agent and version, not by experiment. with mlflow.start_span(name="answer-question"): result = agent.run(question) ``` - `set_active_agent` is convenience over MLflow's existing trace - destination mechanism: it wraps - `mlflow.tracing.set_destination(MlflowAgentLocation(...))`, the - way `set_experiment` establishes an experiment destination - today. The location can also be passed per span through - `start_span`'s existing `trace_destination` parameter. Framework - and harness autologgers respect the active destination, so - instrumented applications need only state which agent they are. + `set_active_agent` is convenience over two separable pieces. + First, it sets the trace destination to the agent's one default + experiment, equivalent to calling `mlflow.set_experiment` on the + result of + `mlflow.genai.get_default_experiment_id("acme/billing-agent")`, + where `get_default_experiment_id` is a public lookup that takes + only the agent: the version is never part of the destination. (An + `MlflowAgentLocation` naming the agent can likewise be passed + anywhere MLflow accepts a trace destination, including + `start_span`'s existing `trace_destination` parameter.) Second, + it records the agent and version as trace-level metadata, the way + session and user metadata are recorded today; this is what + per-version filtering and comparison use. A deployment that + overrides its destination (below) swaps only the first piece, + pointing `set_experiment` at its own experiment, and still + declares the agent and version so its traces stay labeled. + Framework and harness autologgers respect the active destination + and metadata, so instrumented applications need only state which + agent they are. 2. Run evaluations against the agent: ```python mlflow.genai.evaluate(data=eval_dataset, @@ -570,6 +592,10 @@ organized by agent and version, not by experiment. agent_id="acme/billing-agent", agent_version=3) ``` + As with tracing, `agent_id` determines where the results land + (the agent's default experiment, unless overridden) and + `agent_version` is recorded on the evaluation run for filtering + and comparison. 3. Open the agent's detail page. A Traces tab shows the agent's traces, filterable by version; an Evaluations tab shows eval runs; a summary card shows latest eval score and trace volume. From b2b9a388920e697853122c35c6d45b9291ccbce3 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Thu, 27 Aug 2026 13:13:21 -0400 Subject: [PATCH 17/18] Conciseness pass Cut redundancy that accreted across incremental revisions: rules stated in multiple places (server-never-fetches, card-not-persisted, default-experiment mechanics) now appear once where they matter most, the blast-radius rationale is consolidated, and digressions are trimmed. No design content changed. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 276 ++++++++---------- 1 file changed, 118 insertions(+), 158 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 5c229c0..38aa5ef 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -26,11 +26,10 @@ # Summary -Add an Agent Registry to MLflow: a governed, record-level registry for -AI agents. The registry catalogs the agents an organization has built. -It records each agent's identity, ownership, composition, source -provenance, and lifecycle status, and it anchors the agent's traces -and evaluation results. It answers "what agents exist, who owns them, +Add an Agent Registry to MLflow: a governed, record-level registry +that catalogs each agent's identity, ownership, composition, source +provenance, and lifecycle status, and anchors the agent's traces and +evaluation results. It answers "what agents exist, who owns them, what are they made of, and how well do they work?" The Agent Registry is the third registry in a series, following the @@ -43,15 +42,12 @@ It completes a progression: MLflow can govern the tools an agent calls invokes (Model Registry), but the agent itself, the thing that acts, has no registry entry. This RFC adds one. -**The registry is record-level, not runtime-aware.** It stores what an -agent is: who built it, what components it is made of, where its -source lives, and what lifecycle state it is in. It does not store -whether the agent is running, healthy, or scaled. Runtime state -belongs to the serving platform. A platform that deploys registered -agents can join its own runtime inventory against registry records at -query time; the registry supplies the governance half of that join. -This is the same division of responsibility RFC-0004 draws between -the MCP registry and a gateway. +**The registry is record-level, not runtime-aware.** It stores what +an agent is, not whether it is running, healthy, or scaled. Runtime +state belongs to the serving platform, which can join its own +runtime inventory against registry records at query time. This is +the same division of responsibility RFC-0004 draws between the MCP +registry and a gateway. The registry manages two primary entities under the `mlflow.genai` SDK namespace, following the pattern of RFC-0004 and RFC-0008: @@ -80,11 +76,10 @@ Version identity is a per-agent choice made when the agent is created: `monotonic` (registry-assigned serial numbers: 1, 2, 3), `semver` (registrant-supplied semantic versions), or `freeform` (registrant-supplied opaque strings). `monotonic` is the default -when registration supplies no version, so the simple path stays as -simple as the Skill Registry's: registrants who never think about -versioning get serial numbers automatically, while an agent that -already carries its own versioning (a team's release train, a -provider-versioned A2A agent) can keep it. The scheme may also be +when registration supplies no version: registrants who never think +about versioning get serial numbers automatically, while an agent +that already carries its own versioning (a provider-versioned A2A +agent, for example) can keep it. The scheme may also be autodetected from the first registration's input. This refines the policy the earlier registries establish, where an entry adopts the underlying artifact's version when its format defines one @@ -101,9 +96,9 @@ BOM entries are soft references, structured values rather than foreign keys. They resolve against the Skill Registry (which RFC-0008 defines for both skills and agent plugins), MCP Server Registry, and Model Registry when matching entries exist, and they -remain valid when they do not. This is what makes cross-registry -questions ("which agents use skill X?") answerable as registry -queries while keeping registration decoupled from registration order. +remain valid when they do not. This makes cross-registry questions +("which agents use skill X?") answerable as registry queries +without constraining registration order. **The BOM is a component inventory, not a complete recipe.** Its structured axes exist because corresponding registries or identifier @@ -114,16 +109,15 @@ share the job of describing an agent. Structured BOM references are selective but queryable across agents. Definitional anchors (a source pointer, a configuration snapshot) are complete but opaque: they capture everything about one agent without supporting cross-agent -queries. Free-form tags are the catch-all for facts that fit neither, -and there will always be some, because agents acquire new kinds of -parts faster than any schema anticipates. New structured axes are -expected as the governance surface grows. +queries. Free-form tags are the catch-all for facts that fit +neither. New structured axes are expected as the governance surface +grows. **A2A Agent Cards are fetched, not stored.** The [A2A protocol](https://a2a-protocol.org/) makes an agent's endpoint -the authoritative home of its Agent Card: the live card is always -one GET away at the endpoint's well-known path, and every A2A -client reads it from there. The registry follows suit. Registering +the authoritative home of its Agent Card: every A2A client reads +the live card from the endpoint's well-known path, and the registry +follows suit. Registering from an endpoint imports the card's descriptive metadata (description, capabilities, and its free-form name, which seeds the mutable MLflow-managed `display_name`) into ordinary registry @@ -164,11 +158,9 @@ the agent only and resolves to the agent's one default experiment; the version is never part of the destination and is instead recorded on every trace and evaluation run as metadata, which is what per-version filtering and comparison use. Traces and eval -results appear on the agent's registry page, filterable by -version. The -change is additive: under the hood an agent owns a default -experiment, the agent location resolves to it, and experiment-based -workflows continue unchanged. +results appear on the agent's registry page, filterable by version. +The change is additive: the default experiment exists under the +hood, and experiment-based workflows continue unchanged. **Relationship to other RFCs.** RFC-0004 establishes the access binding pattern this RFC reuses (its canonical-payload pattern is @@ -239,15 +231,12 @@ binding for the endpoint. The card content is not persisted: the endpoint is the card's system of record, and the UI renders the card read-only by fetching it through the binding at view time. The registry `name` is always chosen by the registrant, since a card's -`name` is a display string, not an identity. The card is whatever -the running agent serves; it need not exist as a static file -anywhere. Fetches always happen in the client or the browser: the -registry server never fetches user-supplied URLs, consistent with -RFC-0008. A caller that wants to inspect or adjust the imported -metadata before registering can fetch the card itself and pass it -via `a2a_card=` instead. A version registered this way, with no -source and no configuration snapshot, is an interface-only record -(see the register journey). +`name` is a display string, not an identity. Fetches happen in the +client, never in the registry server, consistent with RFC-0008; a +caller can also fetch the card itself and pass it via `a2a_card=` +to inspect the imported metadata first. A version registered this +way, with no source and no configuration snapshot, is an +interface-only record (see the register journey). ## Register a harness-based agent @@ -264,14 +253,13 @@ mlflow.genai.register_agent( ) ``` -There is no user source repository: the agent is the harness plus its -configuration, so the configuration snapshot serves as the +There is no user source repository: the agent is the harness plus +its configuration, so the configuration snapshot serves as the definitional anchor. `config_snapshot` names a file or directory on -the caller's local disk; the client reads it and uploads the content -to MLflow artifact storage as an immutable artifact on the version. -The registry server never fetches user-supplied locations itself, -consistent with RFC-0008, which also means any redaction of secrets -must happen client-side before upload. +the caller's local disk; the client reads it and uploads the +content to MLflow artifact storage as an immutable artifact on the +version, which means any redaction of secrets must happen +client-side before upload. ## Trace and evaluate against the agent @@ -387,15 +375,11 @@ the record. descriptive and capability fields from the card (its free-form name seeds the mutable `display_name`; the registry `name` is supplied by the registrant) and creates an `a2a` access binding - for the endpoint. The card content is not persisted: the agent - detail page renders the card read-only by fetching it through - the binding, so the display never drifts from what the agent - serves. In the SDK and CLI, the client fetches the card at - import; in the UI, the browser fetches it when the endpoint - permits, or the user pastes it, since the registry server never - fetches user-supplied URLs (consistent with RFC-0008). The BOM - is supplied alongside, since the card schema does not carry - component version pins. + for the endpoint. In the SDK and CLI, the client fetches the + card at import; in the UI, the browser fetches it when the + endpoint permits, or the user pastes it, since the server never + fetches user-supplied URLs. The BOM is supplied alongside, since + the card schema does not carry component version pins. 5. **CI path:** the same call runs from a pipeline, registering a new version on each release build with the source ref set to the build's commit. @@ -421,51 +405,42 @@ the record. The configuration snapshot is read from the caller's local disk and stored as an immutable artifact on the version. It is what distinguishes this agent from every other installation of the - same harness: enabled tools, overridden defaults (for example, a - non-standard shell for the harness's shell tool), and behavioral + same harness: enabled tools, overridden defaults, and behavioral settings live there and nowhere else. - A harness's configuration surface is not always a single file: - Claude Code, for example, spreads behavioral configuration across - a settings file, instruction files, and subagent definitions. - `config_snapshot` therefore accepts a file or a directory. What it - should capture is the configuration the registry does not - otherwise represent; content that BOM references already govern, - such as installed skill directories or MCP server definitions, - should stay out of the snapshot, because an embedded copy is - invisible to cross-registry queries and can silently drift from - the declared references. + A harness's configuration surface is not always a single file + (Claude Code, for example, spreads it across a settings file, + instruction files, and subagent definitions), so + `config_snapshot` accepts a file or a directory. It should + capture only configuration the registry does not otherwise + represent: content that BOM references already govern, such as + installed skill directories or MCP server definitions, stays + out, because an embedded copy is invisible to cross-registry + queries and can drift from the declared references. Composition is required wherever it is knowable, because the BOM is the value: a registry record without composition is just a name in a list. A registrant anchoring on source or a configuration -snapshot has the composition in front of them, so the requirement -holds there. An interface-only registration (an A2A endpoint with -no source or configuration snapshot) is different: the registrant -of a vendor or partner agent cannot know which skills, servers, and -models are inside a black box, and forcing a declaration would -invite invented BOMs that pollute cross-registry queries. For -interface-only records the BOM may therefore be empty or partial, -an absent BOM is recorded as *undeclared* composition, not as an -empty dependency list, and the record itself is marked as holding -no definitional anchor: the registry knows the agent's claim -surface, not its contents. Everything else is -progressive enrichment. The endpoint URL is optional; the -trace-and-eval journey below explains which agents need one. When -provided, the URL becomes an access binding rather than a field on -the immutable version. +snapshot has the composition in front of them. The registrant of a +black-box vendor or partner agent does not, and forcing a +declaration would invite invented BOMs that pollute cross-registry +queries; for interface-only records the BOM may therefore be empty +or partial, an absent BOM is recorded as *undeclared* composition +rather than an empty dependency list, and the record is marked as +holding no definitional anchor: the registry knows the agent's +claim surface, not its contents. Everything else is progressive +enrichment. The endpoint is optional; the trace-and-eval journey +below explains which agents need one. The harness path is the newest part of this design and the least settled (see [Open questions](#open-questions)). For framework-built and custom agents, the registered source is the natural complete -record and remains the expected anchor. For harness-based agents, -requiring -source would force registrations that point at the harness vendor's -repository, which identifies nothing about the specific agent, while -the configuration that actually defines it went unrecorded. -Configuration files frequently embed secrets and environment-specific -values, so the snapshot mechanism needs redaction guidance at -minimum. +record and remains the expected anchor; for harness-based agents, +requiring source would force registrations that point at the +harness vendor's repository, which identifies nothing about the +specific agent. Configuration files frequently embed secrets and +environment-specific values, so the snapshot mechanism needs +redaction guidance at minimum. ### Publish and maintain an agent's endpoint @@ -500,17 +475,15 @@ disturbing the immutable version history. agent, its versions, and its history remain untouched. The protocol field is where agent bindings depart from RFC-0004, -whose bindings are always MCP and vary only by transport. Agent -endpoints speak different protocols, and the field is limited to -values that tell a caller something actionable: `a2a` and `mcp` are -self-describing (an Agent Card at the well-known path; the MCP -handshake), so URL plus protocol is enough to connect. Labels like -REST or gRPC name a transport without telling anyone how to call -the agent, so they are deliberately collapsed into `other`, which -records where an agent lives without claiming MLflow can invoke it. -This keeps the registry open to custom protocols without pretending -to understand them. As in RFC-0004, a binding that targets an alias -such as `production` follows the alias as it moves between +whose bindings are always MCP and vary only by transport. The field +is limited to values that tell a caller something actionable: `a2a` +and `mcp` are self-describing (an Agent Card at the well-known +path; the MCP handshake), so URL plus protocol is enough to +connect. Labels like REST or gRPC name a transport without telling +anyone how to call the agent, so they are deliberately collapsed +into `other`, which records where an agent lives without claiming +MLflow can invoke it. As in RFC-0004, a binding that targets an +alias such as `production` follows the alias as it moves between versions. ### Version an agent and compare bills of materials @@ -573,9 +546,8 @@ organized by agent and version, not by experiment. `mlflow.genai.get_default_experiment_id("acme/billing-agent")`, where `get_default_experiment_id` is a public lookup that takes only the agent: the version is never part of the destination. (An - `MlflowAgentLocation` naming the agent can likewise be passed - anywhere MLflow accepts a trace destination, including - `start_span`'s existing `trace_destination` parameter.) Second, + `MlflowAgentLocation` naming the agent works anywhere MLflow + accepts a trace destination.) Second, it records the agent and version as trace-level metadata, the way session and user metadata are recorded today; this is what per-version filtering and comparison use. A deployment that @@ -604,11 +576,10 @@ organized by agent and version, not by experiment. the cause, cross-referencing the BOM diff from the previous journey. -Backward compatibility is preserved by construction: an agent owns a -default experiment under the hood, and the agent location resolves -to that experiment's ID. Where MLflow already accepts a typed -destination or location (trace destinations, `search_traces` -locations), agent identity becomes a new accepted value; where it +Backward compatibility is preserved by construction. Where MLflow +already accepts a typed destination or location (trace +destinations, `search_traces` locations), agent identity becomes a +new accepted value; where it does not (`evaluate`), `agent_id` is new, optional surface. Existing experiment-based workflows (including model training and fine-tuning) continue unchanged. The change is additive, not a data @@ -626,10 +597,10 @@ other deployments of the same agent (a different owner or user base) overrides the destination in its own configuration; because permissions are experiment-scoped, separation is done with destinations, not trace tags. Versions share the agent's experiment -for the same reason seen from the other side: a version is an -analysis dimension recorded on every trace, not an access boundary, -and per-version experiments would break the longitudinal view of an -agent's behavior across upgrades. To keep such traces findable from the +for the same reason: a version is an analysis dimension recorded on +every trace, not an access boundary, and per-version experiments +would break the longitudinal view of an agent's behavior across +upgrades. So that overridden deployments stay findable from the agent's registry page, this RFC proposes that a deployment using a non-default experiment notify the registry, which records the experiment ID on that deployment's access binding. A deployment @@ -640,9 +611,8 @@ belongs to the deferred registry synchronization glue. No endpoint is needed for any of this when the developer has the agent's code: the agent runs locally or in CI, autologging captures -traces during execution, and evaluation scores outputs against a test -dataset. This is why code-only registration supports the full -trace-and-eval workflow. The exception is agents whose code the user +traces during execution, and evaluation scores outputs against a +test dataset. The exception is agents whose code the user cannot run: another team's A2A agent, a vendor agent, a partner service. For those, the endpoint is the only execution surface, and tracing and evaluation work by invoking the agent's access binding @@ -684,8 +654,8 @@ auditable lifecycle state. This is the same core `draft` → `active` → `deprecated` lifecycle the MCP and Skill registries use (their soft-delete `deleted` state and transition rules carry over as well), applied to the agent -itself. The -auditable transition history is the accountability chain that problem +itself. The auditable transition history is the accountability +chain that problem 1 identifies as missing: for any agent, the registry can say who promoted it, when, and what its evaluation evidence looked like at the time. @@ -718,35 +688,25 @@ every affected agent without inspecting deployments one by one. versions with the skill removed or upgraded, and deprecate the affected versions (previous journey). -The same query works for the other BOM axes: "which agents use MCP -server Y whose tool schema changed?" and "which agents call model Z -being retired?" Source entries are queryable the same way: "which -agents ship OCI image X?" is the container-CVE variant of this -journey. Agent plugin references expand through their -members: RFC-0008 plugin versions immutably record which registered -skills they contain, so "which agents use skill X" also finds -agents that consume X through a plugin, rather than leaving a -blind spot behind composed units. Because BOM entries are -structured soft references, -the query is a registry lookup with exact-match semantics rather than -a scan of running infrastructure. Because a BOM holds many refs per -axis, the name and version predicates must bind to the same BOM -entry; matching a name on one skill ref and a version on another -would produce false positives. That same-entry matching requirement -is a store-level obligation for the detailed design, in the way -exact matching on span attributes is for RFC-0009. The registry -answers with consumers and owners; automated notification of those -owners is deliberately not in the MVP (see Out of scope). - -Coverage has two limits a reader of the results should keep in mind: -the query sees only the component types the registry tracks, and -within those, only what registrants declared. Agents with -undeclared composition (black-box, interface-only registrations) -can never match, so results should surface them alongside matches: -"3 -agents declare the compromised skill; 12 more have undeclared -composition." The query replaces infrastructure inspection for the -governed axes; it does not claim completeness beyond them. +The same query works for the other BOM axes ("which agents use MCP +server Y whose tool schema changed?", "which agents call model Z +being retired?") and for source entries: "which agents ship OCI +image X?" is the container-CVE variant. Agent plugin references +expand through their members: RFC-0008 plugin versions immutably +record which registered skills they contain, so the query also +finds agents that consume a skill through a plugin. The query has +exact-match semantics, and the name and version predicates must +bind to the same BOM entry (a store-level obligation for the +detailed design, like exact span-attribute matching in RFC-0009). +The registry answers with consumers and owners; automated +notification of those owners is deliberately not in the MVP (see +Out of scope). + +Coverage has two limits: the query sees only the component types +the registry tracks, and within those, only what registrants +declared. Agents with undeclared composition can never match, so +results should surface them alongside matches: "3 agents declare +the compromised skill; 12 more have undeclared composition." ## Out of scope @@ -773,8 +733,8 @@ rejection. - **Auto-discovery of composition.** BOMs are developer-asserted in the MVP. Inferring actual composition from traces (for example, from RFC-0009 `SKILL` spans) and notifying owners when assertion - and observation disagree is a compelling follow-on, and the span - data this RFC and RFC-0009 produce is designed to enable it. + and observation disagree is a follow-on that the span data from + this RFC and RFC-0009 is designed to enable. - **Detection of unregistered agents.** Surfacing "shadow" agents running without registry entries requires runtime scanning, which is platform work built on top of this registry. @@ -904,8 +864,8 @@ TBD. skill pattern applied to agents would produce irreconcilable duplicate versions of what is really one agent. The list is currently an unordered set of asserted pointers. Reviewers who - weigh cross-registry consistency - heavily should push back here if the divergence is not worth it. + weigh cross-registry consistency heavily should push back here if + the divergence is not worth it. - **What is the BOM reference format?** The journeys sketch URI-style references (`skills:/billing-policy/1`, @@ -915,9 +875,9 @@ TBD. scheme adopts the `mcp-servers:/` proposal from RFC-0010; the refs here respect RFC-0004's reverse-DNS server names and semantic versions. Bare identifiers cover external models (`gpt-4o`). BOM - references are soft (string-resolved, valid when - the target is unregistered), which URI syntax may misleadingly - suggest otherwise. Alternatives include structured + references are soft (string-resolved, valid when the target is + unregistered), which URI syntax may misleadingly suggest + otherwise. Alternatives include structured `{registry, name, version}` objects. Relatedly: should the BOM also be able to reference other agents, to represent multi-agent systems? From f464589ae41abc8f6e6cfb7710d7f693a5e47787 Mon Sep 17 00:00:00 2001 From: Bill Murdock Date: Thu, 27 Aug 2026 13:34:33 -0400 Subject: [PATCH 18/18] Move design positions out of the Summary The Summary is now a summary: opening, lineage, a one-paragraph tour of the positions, and RFC relationships. The full position statements move intact to a Design positions subsection under Detailed design, where they seed the eventual full design. No content changed beyond the pointer sentences. Co-Authored-By: Claude Fable 5 --- .../0011-agent-registry.md | 260 ++++++++++-------- 1 file changed, 139 insertions(+), 121 deletions(-) diff --git a/rfcs/0011-agent-registry/0011-agent-registry.md b/rfcs/0011-agent-registry/0011-agent-registry.md index 38aa5ef..6411b5a 100644 --- a/rfcs/0011-agent-registry/0011-agent-registry.md +++ b/rfcs/0011-agent-registry/0011-agent-registry.md @@ -19,6 +19,7 @@ - [User journeys](#user-journeys) - [Out of scope](#out-of-scope) - [Detailed design](#detailed-design) + - [Design positions](#design-positions) - [Drawbacks](#drawbacks) - [Alternatives](#alternatives) - [Adoption strategy](#adoption-strategy) @@ -42,129 +43,22 @@ It completes a progression: MLflow can govern the tools an agent calls invokes (Model Registry), but the agent itself, the thing that acts, has no registry entry. This RFC adds one. -**The registry is record-level, not runtime-aware.** It stores what -an agent is, not whether it is running, healthy, or scaled. Runtime -state belongs to the serving platform, which can join its own -runtime inventory against registry records at query time. This is -the same division of responsibility RFC-0004 draws between the MCP -registry and a gateway. - -The registry manages two primary entities under the `mlflow.genai` -SDK namespace, following the pattern of RFC-0004 and RFC-0008: - -- **Agent**: a named, owned entity with DNS-style naming - (`org/agent-name`), in the spirit of the namespaced names the MCP - and Skill registries use; exact alignment with RFC-0008's - `{workspace, organization, name}` coordinates is a detailed-design - point. -- **AgentVersion**: an immutable snapshot of the agent's composition, - its **bill of materials (BOM)**: skill references, agent plugin - references (a plugin is referenced as a composed unit and expands - through its registered members for queries), MCP server - references, model references (registry models or external model - identifiers such as `gpt-4o`), and, for agents that run as - configurations of a packaged harness, a harness reference (a - proposed axis; see [Open questions](#open-questions)). Each - version also carries at least one **definitional anchor**: source - provenance (one or more typed source pointers as in RFC-0008: a - Git repo and ref, an OCI image, an archive) and/or an immutable - configuration snapshot. An agent registered from an A2A endpoint - alone is an interface-only record with no anchor (see below). - Each change to composition is a new version. - -Version identity is a per-agent choice made when the agent is -created: `monotonic` (registry-assigned serial numbers: 1, 2, 3), -`semver` (registrant-supplied semantic versions), or `freeform` -(registrant-supplied opaque strings). `monotonic` is the default -when registration supplies no version: registrants who never think -about versioning get serial numbers automatically, while an agent -that already carries its own versioning (a provider-versioned A2A -agent, for example) can keep it. The scheme may also be -autodetected from the first registration's input. This refines the -policy the earlier registries establish, where an entry adopts the -underlying artifact's version when its format defines one -(RFC-0004's `server_json`, RFC-0008's plugin manifests) and mints -serial numbers when it does not (RFC-0008's skills): no standard -agent artifact defines an inherent agent version, so minting is the -default rather than the rule. An A2A card's provider-defined -`version` string lives on the live card rather than in the registry; -an agent registered under `semver` or `freeform` may mirror it. -Aliases and `latest` resolution are well defined for `monotonic` -and `semver`; `freeform` versions order by registration time. - -BOM entries are soft references, structured values rather than -foreign keys. They resolve against the Skill Registry (which -RFC-0008 defines for both skills and agent plugins), MCP Server -Registry, and Model Registry when matching entries exist, and they -remain valid when they do not. This makes cross-registry questions -("which agents use skill X?") answerable as registry queries -without constraining registration order. - -**The BOM is a component inventory, not a complete recipe.** Its -structured axes exist because corresponding registries or identifier -conventions exist, so it is bounded by MLflow's governance surface -rather than by agent anatomy: an agent's prompt strategy or memory -configuration has no axis because nothing governs one. Three layers -share the job of describing an agent. Structured BOM references are -selective but queryable across agents. Definitional anchors (a source -pointer, a configuration snapshot) are complete but opaque: they -capture everything about one agent without supporting cross-agent -queries. Free-form tags are the catch-all for facts that fit -neither. New structured axes are expected as the governance surface -grows. - -**A2A Agent Cards are fetched, not stored.** The -[A2A protocol](https://a2a-protocol.org/) makes an agent's endpoint -the authoritative home of its Agent Card: every A2A client reads -the live card from the endpoint's well-known path, and the registry -follows suit. Registering -from an endpoint imports the card's descriptive metadata -(description, capabilities, and its free-form name, which seeds the -mutable MLflow-managed `display_name`) into ordinary registry -fields and creates an `a2a` access binding; the card content itself -is not persisted. The UI renders the card read-only by fetching it -through the binding at view time, so what MLflow displays can never -drift from what the agent serves. This deliberately departs from -the canonical-payload pattern of RFC-0004 (`server_json`) and -RFC-0008 (`plugin.json`): MLflow is the system of record for those -payloads, while an Agent Card's system of record is the agent -itself. An agent registered from an endpoint alone, with no source -and no configuration snapshot, is an **interface-only record**: the -registry captures the claim surface (identity, imported metadata, -endpoint) and marks that it holds no definitional anchor. - -**Endpoints are access bindings, not version fields.** Some agents -are reachable at a URL (A2A agents inherently; deployed agents -generally), and recording that URL lets the registry drive tracing -and evaluation for agents whose code the user cannot run. But -endpoints change independently of composition, and agent versions are -immutable. Following RFC-0004's `MCPAccessBinding` model, approved -endpoints are separate mutable binding records that target a version -or alias, created and deleted as connectivity changes without -touching version history. Where an MCP binding's protocol is always -MCP, an agent binding declares its protocol: `a2a`, `mcp` (for -agents exposed as MCP servers), or `other`. Registration accepts an -optional endpoint as a convenience that creates a binding. - -**Agents become the primary anchor for GenAI traces and -evaluations.** Today traces and evaluation runs attach to -experiments, an abstraction that fits model training but not the -agent development loop. This RFC makes an agent a trace -destination: a new `MlflowAgentLocation` joins the existing -`MlflowExperimentLocation`, usable wherever MLflow accepts a trace -destination today, and evaluation and trace-search APIs gain agent -identity alongside experiment identity. A destination identifies -the agent only and resolves to the agent's one default experiment; -the version is never part of the destination and is instead -recorded on every trace and evaluation run as metadata, which is -what per-version filtering and comparison use. Traces and eval -results appear on the agent's registry page, filterable by version. -The change is additive: the default experiment exists under the -hood, and experiment-based workflows continue unchanged. +In brief, the design takes these positions, each stated in full in +[Design positions](#design-positions): the registry is +record-level, not runtime-aware; an agent's versions are immutable +snapshots of its composition (a bill of materials of skills, agent +plugins, MCP servers, and models) plus at least one definitional +anchor (source pointers and/or a configuration snapshot); A2A Agent +Cards are fetched from the agent's endpoint, never stored; +endpoints are mutable, protocol-typed access bindings rather than +version fields; and an agent is a trace destination with one +default experiment, with the version recorded on every trace and +evaluation run. **Relationship to other RFCs.** RFC-0004 establishes the access binding pattern this RFC reuses (its canonical-payload pattern is -deliberately not applied to Agent Cards; see above). RFC-0008 defines +deliberately not applied to Agent Cards; see +[Design positions](#design-positions)). RFC-0008 defines the skills and agent plugins that agent BOMs reference; note that an *agent plugin* (a package of components installed into a harness) is not an *agent* (an application that acts); this registry governs the @@ -755,7 +649,131 @@ rejection. # Detailed design -TBD. +## Design positions + +The following positions are settled enough to draft against; the +remainder of the detailed design is TBD. + +**The registry is record-level, not runtime-aware.** It stores what +an agent is, not whether it is running, healthy, or scaled. Runtime +state belongs to the serving platform, which can join its own +runtime inventory against registry records at query time. This is +the same division of responsibility RFC-0004 draws between the MCP +registry and a gateway. + +The registry manages two primary entities under the `mlflow.genai` +SDK namespace, following the pattern of RFC-0004 and RFC-0008: + +- **Agent**: a named, owned entity with DNS-style naming + (`org/agent-name`), in the spirit of the namespaced names the MCP + and Skill registries use; exact alignment with RFC-0008's + `{workspace, organization, name}` coordinates is a detailed-design + point. +- **AgentVersion**: an immutable snapshot of the agent's composition, + its **bill of materials (BOM)**: skill references, agent plugin + references (a plugin is referenced as a composed unit and expands + through its registered members for queries), MCP server + references, model references (registry models or external model + identifiers such as `gpt-4o`), and, for agents that run as + configurations of a packaged harness, a harness reference (a + proposed axis; see [Open questions](#open-questions)). Each + version also carries at least one **definitional anchor**: source + provenance (one or more typed source pointers as in RFC-0008: a + Git repo and ref, an OCI image, an archive) and/or an immutable + configuration snapshot. An agent registered from an A2A endpoint + alone is an interface-only record with no anchor (see below). + Each change to composition is a new version. + +Version identity is a per-agent choice made when the agent is +created: `monotonic` (registry-assigned serial numbers: 1, 2, 3), +`semver` (registrant-supplied semantic versions), or `freeform` +(registrant-supplied opaque strings). `monotonic` is the default +when registration supplies no version: registrants who never think +about versioning get serial numbers automatically, while an agent +that already carries its own versioning (a provider-versioned A2A +agent, for example) can keep it. The scheme may also be +autodetected from the first registration's input. This refines the +policy the earlier registries establish, where an entry adopts the +underlying artifact's version when its format defines one +(RFC-0004's `server_json`, RFC-0008's plugin manifests) and mints +serial numbers when it does not (RFC-0008's skills): no standard +agent artifact defines an inherent agent version, so minting is the +default rather than the rule. An A2A card's provider-defined +`version` string lives on the live card rather than in the registry; +an agent registered under `semver` or `freeform` may mirror it. +Aliases and `latest` resolution are well defined for `monotonic` +and `semver`; `freeform` versions order by registration time. + +BOM entries are soft references, structured values rather than +foreign keys. They resolve against the Skill Registry (which +RFC-0008 defines for both skills and agent plugins), MCP Server +Registry, and Model Registry when matching entries exist, and they +remain valid when they do not. This makes cross-registry questions +("which agents use skill X?") answerable as registry queries +without constraining registration order. + +**The BOM is a component inventory, not a complete recipe.** Its +structured axes exist because corresponding registries or identifier +conventions exist, so it is bounded by MLflow's governance surface +rather than by agent anatomy: an agent's prompt strategy or memory +configuration has no axis because nothing governs one. Three layers +share the job of describing an agent. Structured BOM references are +selective but queryable across agents. Definitional anchors (a source +pointer, a configuration snapshot) are complete but opaque: they +capture everything about one agent without supporting cross-agent +queries. Free-form tags are the catch-all for facts that fit +neither. New structured axes are expected as the governance surface +grows. + +**A2A Agent Cards are fetched, not stored.** The +[A2A protocol](https://a2a-protocol.org/) makes an agent's endpoint +the authoritative home of its Agent Card: every A2A client reads +the live card from the endpoint's well-known path, and the registry +follows suit. Registering from an endpoint imports the card's +descriptive metadata (description, capabilities, and its free-form +name, which seeds the mutable MLflow-managed `display_name`) into +ordinary registry fields and creates an `a2a` access binding; the +card content itself is not persisted. The UI renders the card +read-only by fetching it through the binding at view time, so what +MLflow displays can never drift from what the agent serves. This +deliberately departs from the canonical-payload pattern of RFC-0004 +(`server_json`) and RFC-0008 (`plugin.json`): MLflow is the system +of record for those payloads, while an Agent Card's system of +record is the agent itself. An agent registered from an endpoint +alone, with no source and no configuration snapshot, is an +**interface-only record**: the registry captures the claim surface +(identity, imported metadata, endpoint) and marks that it holds no +definitional anchor. + +**Endpoints are access bindings, not version fields.** Some agents +are reachable at a URL (A2A agents inherently; deployed agents +generally), and recording that URL lets the registry drive tracing +and evaluation for agents whose code the user cannot run. But +endpoints change independently of composition, and agent versions +are immutable. Following RFC-0004's `MCPAccessBinding` model, +approved endpoints are separate mutable binding records that target +a version or alias, created and deleted as connectivity changes +without touching version history. Where an MCP binding's protocol +is always MCP, an agent binding declares its protocol: `a2a`, `mcp` +(for agents exposed as MCP servers), or `other`. Registration +accepts an optional endpoint as a convenience that creates a +binding. + +**Agents become the primary anchor for GenAI traces and +evaluations.** Today traces and evaluation runs attach to +experiments, an abstraction that fits model training but not the +agent development loop. This RFC makes an agent a trace +destination: a new `MlflowAgentLocation` joins the existing +`MlflowExperimentLocation`, usable wherever MLflow accepts a trace +destination today, and evaluation and trace-search APIs gain agent +identity alongside experiment identity. A destination identifies +the agent only and resolves to the agent's one default experiment; +the version is never part of the destination and is instead +recorded on every trace and evaluation run as metadata, which is +what per-version filtering and comparison use. Traces and eval +results appear on the agent's registry page, filterable by version. +The change is additive: the default experiment exists under the +hood, and experiment-based workflows continue unchanged. # Drawbacks