Skip to content

Simplify live inspection and preparation for immutable Agents #20

Description

@Beckers26

Owner decision: direction accepted; design revision required before API freeze.

This is a breaking v1 umbrella. Compatibility with the current inspection and resource-mutation surface is secondary to a smaller, clearer long-term API. A breaking change is justified only when it removes concepts or makes ownership and lifecycle rules materially easier to understand.

Summary

Give embedding hosts truthful live discovery and optional startup preparation without adding a second execution path or turning the root package into a host UI framework.

The target design has three properties:

  1. One direct, read-only live inspection operation.
  2. One optional control-plane preparation operation for resource materialization and safe process pre-initialization.
  3. An Agent whose stable resource topology is immutable after construction.

Run and Stream remain the only execution verbs. Prepare never sends a prompt, creates a run, emits execution Events, or creates or mutates a Thread checkpoint.

Portable slash commands are no longer a root-package concern. Provider-native commands may be reported as live observations, while portable presentation and dispatch belong in an optional hosttool.

Why this change

An embedding host currently cannot reliably answer:

  • Which models and reasoning-effort choices are available to this configured account/provider?
  • Which context limits, skills, MCP servers, commands, quota, or configuration values were actually observed?
  • Is a value unavailable because authentication is missing, or unsupported by the installed provider?
  • Can startup work be completed before the first prompt?
  • Did a result come from the provider or from an SDK-maintained fallback?

The current API also exposes overlapping concepts:

  • Agent.Inspect() returns an Inspector with fragmented panel methods.
  • Models and configuration can fall back to static Descriptor declarations.
  • Optional probes accept cfg any even though configured Drivers already capture their configuration.
  • ProfileState, SyncProfile, and SelectSkills overlap with the proposed preparation lifecycle.
  • Stable resources can be mutated at invocation scope or through a live Agent, complicating fingerprints, persistent-process replacement, and Thread compatibility.

This workstream removes those overlaps rather than adding another compatibility layer.

Target consumer API

agent := adaptor.New(
    codex.Driver(codex.Config{Model: "gpt-5.4"}),
    adaptor.WithWorkspace("/repo"),
    adaptor.WithSkills(skillRefs...),
    adaptor.WithMCP(mcpServers...),
)

inspection, err := agent.Inspect(ctx) // read-only
prepared, err := agent.Prepare(ctx)  // optional control plane

result, err := agent.Run(ctx, prompt, callOptions...)
stream := agent.Stream(ctx, prompt, callOptions...)

Remove:

  • Inspector
  • Agent.ProfileState
  • Agent.SyncProfile
  • Agent.SelectSkills
  • WithTimeout

Agent.Inspect(ctx) replaces the read-only inspection panel and ProfileState.

Agent.Prepare(ctx) replaces the public resource-materialization role of SyncProfile. It is the only public mutating control-plane operation before execution.

New still returns an Agent that is ready to execute. Prepare is always optional: Run and Stream remain self-sufficient and perform the same required internal resolution/materialization stages when preparation has not already completed.

Do not add Start, Warm, ExecuteCommand, Reconfigure, SetSkills, or a parallel run handle.

Inspection

Conceptually:

type ObservationStatus string

const (
    ObservationObserved    ObservationStatus = "observed"
    ObservationUnavailable ObservationStatus = "unavailable"
    ObservationUnsupported ObservationStatus = "unsupported"
)

type ObservationReason struct {
    Code    string
    Message string
}

type ObservationEvidence struct {
    Transport       string
    Method          string
    ProviderVersion string
    Scope           ObservationScope
    ObservedAt      time.Time
    Duration        time.Duration
}

type Observation[T any] struct {
    Status   ObservationStatus
    Value    T
    Reason   *ObservationReason
    Evidence []ObservationEvidence
}

type Inspection struct {
    Driver        DriverInfo
    Environment   Observation[EnvironmentReport]
    Models        Observation[[]ModelInfo]
    Commands      Observation[[]NativeCommandInfo]
    Skills        Observation[SkillSnapshot]
    MCP           Observation[MCPReport]
    Quota         Observation[QuotaReport]
    Profile       Observation[ProfileSnapshot]
    Configuration Observation[ConfigurationReport]
}

This shape remains conceptual until the API-freeze workstream defines every referenced DTO. A field without a provider-neutral typed contract is omitted from v1 rather than represented as map[string]any or stringly metadata.

Observation invariants

  • Observed means the provider or configured Driver returned an authoritative value. An observed empty catalogue differs from unsupported.
  • Observed has no failure reason. Dynamic observations include at least one evidence record.
  • Unavailable means a supported probe cannot currently complete, for example because authentication or an executable is missing. Its value is zero and it has a stable reason code.
  • Unsupported means the installed provider transport/version exposes no official machine-readable probe. Its value is zero and it has a stable reason code.
  • Evidence names ProviderVersion, not an ambiguously named DriverVersion.
  • Multiple official calls contributing to one observation produce multiple evidence entries.
  • Returned snapshots are detached deep copies. Host mutation cannot affect Driver or Agent state.
  • Evidence and messages never expose secrets, raw environment values, auth tokens, or unredacted provider payloads.

For models:

  • Unknown numeric values use pointers.
  • DefaultEffort is *string, not string.
  • EffortInfo must be frozen as a provider-neutral DTO before ModelInfo expands.
  • Session usage remains separate from advertised model limits.
  • No marketing table, source enum, binary scan, TUI parsing, or static model fallback is allowed.

Static Driver configuration schema remains a declaration on DriverInfo or Descriptor. It is not wrapped in an Observed status or confused with provider runtime configuration. Effective provider configuration is included only after a separate typed, redacted ConfigurationReport contract is frozen.

Inspection side effects and errors

Agent.Inspect(ctx):

  • uses the exact configuration captured by the Driver passed to adaptor.New;
  • may start a transient provider process and perform authenticated read operations;
  • does not materialize skills, MCP, profiles, tools, workspaces, or runtime services;
  • does not invoke a side-effecting WorkspaceManager;
  • does not open an interactive login flow;
  • does not create a provider conversation or Thread record;
  • does not retain a newly started process;
  • asks for a current observation and introduces no core TTL or static fallback;
  • returns ErrAgentClosed after Close begins and must not start another process.

A top-level Go error is reserved for cancellation, closed Agent state, Driver contract violation, or failure to produce any coherent snapshot. Authentication-required, unsupported panels, and isolated panel failures are observations. Do not freeze an unspecified "may return a partial snapshot" contract.

Preparation

Conceptually:

type ProcessPreparationState string

const (
    ProcessPreparationUnsupported ProcessPreparationState = "unsupported"
    ProcessPreparationTransient   ProcessPreparationState = "transient"
    ProcessPreparationReady       ProcessPreparationState = "ready"
)

type PrepareReport struct {
    Inspection Inspection
    Process    ProcessPreparationState
}

Do not add redundant public fields such as both State=ready and Retained=true. Reused, pool keys, and multi-layer timing details remain implementation diagnostics unless a stable host use case is demonstrated before freeze.

Agent.Prepare(ctx) performs one ordered control-plane pipeline:

  1. Reject a closed Agent with ErrAgentClosed.
  2. Validate the configured Driver.
  3. Resolve construction-time resource declarations through the same internal resolver used by execution.
  4. Materialize stable resources using the same payload and fingerprint rules used by execution.
  5. Perform zero-prompt Driver initialization and live inspection.
  6. Retain at most one compatible unbound standby process when the Driver can prove this is safe.
  7. Return the post-preparation inspection and process state.

Preparation invariants

  • No prompt, RunID, public Event stream, synthetic Result, Thread lease, provider session checkpoint, or Thread record is created.
  • Prepare and execution share resolver/materializer implementations; they do not maintain two merge or fingerprint algorithms.
  • Required configuration or materialization failure is explicit and never degrades to a warning.
  • Optional inspection sections may remain unavailable without invalidating an otherwise executable Agent.
  • Concurrent identical Prepare calls coalesce; repeated successful calls are idempotent.
  • Each caller's cancellation is independent. Cancelling one waiter does not incorrectly abort successful waiters; when all waiters cancel, work and partial retained resources are cleaned up.
  • Any workspace, runtime endpoint, profile projection, or process retained by preparation becomes Agent-owned and is reclaimed by Agent.Close.
  • Run and Stream remain valid without an earlier Prepare call.
  • WithSpawn always bypasses prepared processes.

Prepared-process ownership and Thread safety

A ready prepared process is an unbound standby, not a Thread writer.

The following contract is a release blocker:

  • The standby is keyed by a deterministic compatibility signature covering every resolved input that affects process reuse: configured Driver identity, model/effort defaults, actual workspace when available, profile/materialized resources, tools, MCP, instructions, relevant policy dimensions, runtime endpoints/fingerprints, provider executable/version inputs, and Driver-specific requirements.
  • One compatible invocation atomically claims the standby.
  • A Thread invocation completes store lookup, lease acquisition, checkpoint validation, resume/fork compatibility, and old-writer coordination before the standby can be bound or registered as its writer.
  • A stateless Agent invocation may consume a standby, but the process closes after that run; stateless calls do not create persistent writers.
  • A Thread invocation may promote the claimed process to its persistent writer only under the existing successful checkpoint and single-writer rules.
  • A prepared process is never registered under a Thread key, engine session ID, or resume ID before that Thread owns the relevant lease.
  • Configuration drift or an incompatible call option causes a cold launch; it never mutates or misbinds the standby.
  • Old persistent writers complete bounded shutdown before a replacement is registered.
  • Prompt delivery failure rules are unchanged: one safe fallback is allowed only before the prompt may have been delivered, and no automatic replay occurs afterward.
  • Close cancels preparation, rejects new Inspect/Prepare/Run/Stream operations, and reclaims every prepared or persistent process.
  • A Driver that cannot meet these rules reports transient or unsupported; it never claims ready.

Existing Thread-bound prewarm helpers keyed by resume/session identity do not by themselves satisfy this Agent-level standby contract.

Immutable Agent resources and option scopes

The scope rule is based on semantic ownership, not merely on whether an option might reduce warm-hit rates.

Construction-only topology

Legal only in New:

  • workspace, workspace spec, and workspace manager;
  • identity;
  • instructions and instruction bundles;
  • tools;
  • skills, skill provider, and materializer;
  • MCP;
  • profile and profile resources;
  • sub-agent declarations, hooks, and config patches;
  • stable runtime service declarations and service manager;
  • Thread store;
  • event buffer and blocking/drop configuration.

Changing one constructs and prepares a replacement Agent. The SDK does not add mutable setters or Reconfigure. Publication, draining, and atomic replacement remain host responsibilities.

Invocation controls

Remain valid as Agent defaults and per-call overrides where applicable:

  • model;
  • reasoning effort;
  • policy;
  • approval handler;
  • metadata;
  • output schema / decode request;
  • WithSpawn;
  • genuinely run-scoped hosttool providers such as WithRunServices.

WithPolicy remains per-call because sandbox and approval policy can legitimately vary by task risk. Process reuse requires compatibility with the effective policy.

WithRunServices remains per-call unless a separate Runner-decorator design replaces it; stable resource immutability must not accidentally remove the existing run-scoped hosttool extension point.

WithTimeout is removed; callers use context.WithTimeout.

Skills append when multiple construction options contribute them, but invocation-scoped skill mutation is removed.

Driver SPI

One coordinated breaking revision:

type Driver interface {
    Descriptor() Descriptor
    ValidateConfig() error
    Inspect(ctx context.Context, req InspectRequest) (InspectResponse, error)
    Prepare(ctx context.Context, req PrepareRequest) (PrepareResponse, error)
    Run(ctx context.Context, req Request, sink EventSink) (Response, error)
}

Rules:

  • A configured Driver validates and observes its captured Config; no probe receives a second cfg any.
  • InspectRequest and PrepareRequest carry only core-resolved, root-independent inputs required at the Driver boundary.
  • InspectResponse contains provider observations only. Core maps SPI values into application-facing DTOs.
  • PrepareResponse reports preparation state while retained processes remain Driver-owned and are reclaimed through the lifecycle contract.
  • Every Driver returns explicit observed/unavailable/unsupported data; missing optional Go interfaces are not the availability model.
  • Provider protocol parsing remains inside the concrete Driver.
  • Descriptor retains static invariant capabilities and declared Go Config schema, but loses dynamic model catalogues and runtime fallbacks.
  • Remove or consolidate ModelLister, ModelDetector where used only by old inspection, EnvironmentProbe, ConfigSchemaProvider, QuotaProbe, and other fragmented panel interfaces.
  • Root public types do not alias or expose internal/*.

Command boundary

Core may report commands observed through an official provider protocol:

Inspection.Commands Observation[[]NativeCommandInfo]

That DTO contains provider-native facts needed for display and documented dispatch availability. It does not contain portable host actions such as model selection, new Thread allocation, resume, fork, status rendering, or help.

Portable slash commands belong in an optional package such as hosttools/slashcommands:

  • it consumes public Agent/Thread/Inspection contracts;
  • it does not call Driver or internal engine APIs;
  • it owns presentation order, localization, aliases, collision policy, and host callbacks;
  • it may help a host map /new, /resume, /fork, /model, /effort, /status, and /help;
  • it does not add ExecuteCommand to Agent and cannot bypass Run/Stream for provider prompt execution;
  • provider-native and portable collisions are resolved by explicit host policy, not silently by core.

This keeps slash syntax and UI state out of the six core nouns while allowing a reusable native-like host component.

Built-in Driver direction

Codex

  • Use official app-server initialization and read-only methods such as model/list and config/read.
  • Do not parse TUI source enums, binary strings, terminal popups, or undocumented help output.
  • Native commands remain unsupported until an official machine-readable catalogue exists.
  • A retained Agent-level app-server is ready only after the unbound-standby contract is implemented; existing Thread/resume prewarm is insufficient.

Claude

  • Use the official SDK/control initialization result for observed models and commands.
  • Unknown initial context limits remain unknown.
  • Retention is enabled only when an initialized process can remain unbound without creating a provider conversation or violating Thread writer rules.

CodeBuddy

  • Use official stream-json control initialization with no prompt.
  • Prefer bundled first-party prewarm IPC rather than an SDK-specific daemon.
  • account=null observations are product/process scoped, not account-entitlement claims.

Cursor

  • Use official ACP initialization and authentication state.
  • Authentication-required is unavailable, never a static model fallback.
  • Inspect and Prepare never start interactive login.
  • Session-scoped command notifications are not obtained by creating a conversation solely for inspection.
  • If the lifecycle cannot safely retain an unbound process, preparation reports transient.

Feasibility evidence, not API guarantees

Measurements from 2026-08-08 show live probing is practical. They are environment-specific feasibility data and must not become flaky CI timing assertions.

Provider/version Zero-prompt path Fresh p50 Fresh p95
Codex 0.146.0 initialize + model/list / config/read 166.9 ms 288.7 ms
Claude 2.1.220 official SDK initialization 584.7 ms 660.0 ms
CodeBuddy 2.133.0 stream-json control initialize 933.2 ms 952.5 ms
Cursor 2026.08.04-aaa8809 ACP initialize/auth handshake 387.2 ms 408.2 ms

Provider-side caching is acceptable when it remains the provider's current answer and evidence identifies method and provider version. Core adds no TTL or static replacement.

Workstreams

This issue is the umbrella for one breaking release boundary, implemented as separately reviewable workstreams:

  • Amend AGENTS.md and freeze the simplified root API and DTO invariants.
  • Implement truthful direct inspection and the mandatory Driver inspection SPI.
  • Make stable Agent resource topology construction-only; remove mutable and call-scoped resource paths.
  • Implement Prepare as the sole public materialization/preparation operation.
  • Prove unbound standby claim, Thread binding, cancellation, and Close lifecycle semantics.
  • Add the optional slash-command hosttool, without root command actions.
  • Update adapter conformance, root API golden, docs, examples, README, and CHANGELOG.
  • Remove all obsolete interfaces and compatibility paths before merge/release.

Intermediate PRs may be staged, but the final v1 tree contains no compatibility shims, duplicate inspection paths, or parallel materialization APIs.

Acceptance criteria

Core/API

  • Agent.Inspect(ctx) is direct, read-only, uses the configured Driver, and has no static fallback.
  • Agent.Prepare(ctx) is optional and the only public preparation/materialization control operation.
  • Run remains exactly Stream + drain + Result.
  • Inspect/Prepare/Run observe the same captured Driver configuration.
  • Prepare and execution share resource-resolution, materialization, and fingerprint implementation.
  • Unsupported, unavailable, authentication-required, observed-empty, malformed response, cancellation, and closed Agent are distinct.
  • Inspect after Close cannot spawn a process.
  • Snapshots are deep-copied and evidence is redacted.
  • No prompt, RunID, Event, Thread lease, record, or checkpoint is produced by Inspect or Prepare.
  • Removed symbols are absent from the complete root API golden.

Concurrency and lifecycle

  • Concurrent Prepare calls coalesce without cross-cancelling independent callers.
  • Prepare racing with Run, Stream, Thread operations, and Close is race-safe.
  • A standby is claimed by at most one invocation.
  • Thread lease and compatibility checks happen before standby binding.
  • Stateless standby consumption never leaves a persistent writer.
  • Incompatible calls do not consume or misbind a standby.
  • Close reclaims prepared resources and every Driver-managed process.
  • Race tests cover standby claim, cancellation, replacement, old-writer shutdown, and post-Close rejection.

Driver conformance

Each built-in Driver has fixtures for:

  • successful zero-prompt inspection;
  • observed empty catalogues;
  • missing executable;
  • missing authentication;
  • unsupported provider version/method;
  • malformed protocol response;
  • exact configured-Driver parity across Inspect, Prepare, and Run;
  • no prompt/checkpoint/session side effects;
  • preparation retention and cleanup when supported;
  • truthful transient/unsupported preparation otherwise;
  • evidence provenance and secret redaction.

Release gates

  • go test -count=1 ./...
  • go vet ./...
  • Linux go test -race ./...
  • all four built-in Drivers pass final adaptertest
  • live conformance remains behind explicit environment-variable gates
  • ordinary CI performs no paid calls or interactive login
  • examples compile and demonstrate Inspect, optional Prepare, per-call model/effort/policy, and replacement on resource changes
  • root API golden, godoc, README, API reference, run policy, streaming docs, profile docs, and CHANGELOG agree
  • no TODOs, temporary aliases, static provider model fallback, compatibility-only entry points, or obsolete optional probe interfaces

Outcome

After this change, a host can:

  • construct one immutable Agent configuration;
  • inspect it through one truthful read-only snapshot;
  • optionally prepare stable resources and a safe standby before accepting traffic;
  • render observed provider capabilities with provenance;
  • remain honest when authentication or provider support prevents discovery;
  • replace and prepare Agents when stable resources change;
  • add reusable slash-command UX through an optional hosttool;
  • and keep Run / Stream as the only execution path.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions