Skip to content

feat: v2.7 — service accounts, MCP interface v2, agnoctl, and eval suites#8747

Open
ashpreetbedi wants to merge 78 commits into
mainfrom
feat/v2.7
Open

feat: v2.7 — service accounts, MCP interface v2, agnoctl, and eval suites#8747
ashpreetbedi wants to merge 78 commits into
mainfrom
feat/v2.7

Conversation

@ashpreetbedi

@ashpreetbedi ashpreetbedi commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Summary

The v2.7 release branch: one backend for every frontend. A running AgentOS becomes operable by any MCP client with one command:

uvx agno connect        # at 2.7 stable; during the alpha: uvx agnoctl connect

The branch is released continuously as alphas — agno 2.7.0a12.7.0a4 and agnoctl 0.1.0a10.1.0a4 (versions in lockstep) are on PyPI via OIDC trusted publishing.

This PR consolidates and supersedes #8742 (service accounts + MCP interface v2) and #8743 (agnoctl), and cherry-picks #8745 (@kausmeows). Larger increments landed on the branch through individually reviewed PRs, each with its full write-up: #8752 and #8760 (security hardening), #8755 (MCP trace attribution), #8759 (pyproject extras), #8761 (eval suite runner), #8740 (community cookbook).

What's in v2.7

1. Service accounts (PAT auth)

Machine identities authenticated by opaque agno_pat_... tokens (GitHub PAT model): SHA-256-only storage, sa:<name> principals, conservative default scopes (run + read + config:read), 90-day default expiry, POST/GET/DELETE /service-accounts, TTL'd verification cache with instant local revocation, uniform-401/fail-closed/throttled verification.

2. MCP interface v2

The 19-tool REST-mirror surface becomes an 8-tool operator surface: get_agentos_config, run_agent, run_team, run_workflow, continue_run, cancel_run, get_sessions, get_session_runs.

  • Run results are trimmed by default; result_mode="full" and get_session_runs(run_id=...) restore full detail (transcript, events, metrics). Progress notifications during runs; HITL pause → continue_run/cancel_run lifecycle, with the admin-approval continuation gate shared with REST (run_continuation_blocked_reason).
  • One implementation for REST and MCP via the shared agno/os/services/ layer; every tool call is scope-gated by mapping it onto its REST-equivalent route, so per-resource scopes and admin bypass behave identically on both surfaces.
  • Request isolation matching REST: components resolve through the shared resolve_* helpers with a create_fresh deep copy (no shared-singleton state across concurrent runs), DB-registry components resolve and enumerate, the config roster is filtered to the caller's accessible resources, and a fresh session is minted per call when session_id is omitted (no cross-request history leaks).
  • MCP-initiated runs are correctly attributed in traces: the run span starts its own root trace instead of nesting under FastMCP's identity-less protocol span (fix: attribute MCP-initiated runs by detaching their trace from the tool-call span #8755).

3. agnoctl — the Agno CLI

New distribution at libs/agnoctl (PyPI agnoctl, command agno + alias agnoctl), zero dependency on the agno framework. Core agno declares [project.scripts] agno = "agnoctl.main:app" plus a dependency on agnoctl, which is what makes uvx agno connect work at stable.

  • agno connect: discover the AgentOS (ports 7777/7778/7779/8000, or --url/AGENTOS_URL) → mint per-client PATs → write client configs → verify with a real MCP handshake. Clients: Claude Code, Codex, Cursor, Claude Desktop (stdio→remote bridge via mcp-remote; token rides an env var, not argv), and ChatGPT (no local config exists to write — prints honest manual setup steps and reports a distinct non-failing manual status).
  • Credential guardrails: refuses to send a bearer credential over plaintext HTTP to a non-loopback host (--allow-http to opt in), atomic 0600-from-birth config writes, the PAT never appears on any process argv, rotated tokens print a restart reminder for the affected client.
  • agno tokens create/list/revoke (revoke confirms interactively), agno create (scaffolds from the agentos-<provider> template repos: docker/aws/railway), agno up/down/restart (compose), agno status. --json everywhere, deterministic exit codes, truthful outcomes, branded home screen.

4. One auth layer

AgentOS installs a single AuthMiddleware on the parent app whenever any credential source is configured (JWT, OS_SECURITY_KEY, or a service-account verifier). Starlette runs parent-app middleware before dispatching to mounts, so that one instance covers REST, the mounted /mcp app, WebSockets, and every interface that does not authenticate its own requests — the mounted MCP app carries no auth code of its own.

  • JWTMiddleware is now an alias of AuthMiddleware (JWT is one credential source among three). It refuses to construct with no credential source, and authorization=True with no JWT verification source fails closed at construction instead of silently serving an open instance.
  • Interfaces that verify their own webhook signatures (Slack, Telegram, WhatsApp) declare authenticates_own_requests = True and are excluded; everything else — including A2A and AGUI — sits behind the auth layer (fix: harden AgentOS auth surface (A2A auth, trace IDOR, PAT mint, validate=False) #8752 S1).
  • The WebSocket path has full parity: PATs prefix-dispatch before JWT handling, verified identity + scopes attach to the connection context, and the RBAC/attribution gates fire exactly as on REST.

5. Security hardening

The auth surface went through five review rounds (in-branch adversarial review, an independent external review by Codex + Opus, #8752, a PAT/JWT hardening pass, #8760). The final round returned no findings. Highlights beyond the items above:

  • A2A/AGUI authorization + identity (fix: gate A2A authorization + self-scope service-account principals (v2.7 B1/B2/D1) #8760 B1/B2): interfaces now declare their own RBAC scope mappings (Interface.get_scope_mappings(), merged against the actual mount prefix), closing a read→execute escalation where a config:read token could run agents via message:send. All A2A/AGUI run sites resolve identity server-side through a shared resolve_run_user_id() — client-supplied X-User-ID/metadata.userId/forwardedProps.user_id is honored only for anonymous callers and never for reserved principals, closing run/session/memory attribution spoofing and a session IDOR. A regression test asserts every mounted /a2a route resolves to a non-empty required-scope set.
  • Trace IDOR fixed (fix: harden AgentOS auth surface (A2A auth, trace IDOR, PAT mint, validate=False) #8752 S2, pre-dates this branch): GET /traces/{trace_id} now enforces user_isolation ownership, including the run_id/span_id paths and the RemoteDb branch.
  • JWTs cannot impersonate reserved principals: tokens whose subject claims sa:* or __scheduler__ are rejected on HTTP and WebSocket.
  • validate=False + authorization=True no longer fails open on undecodable tokens (fix: harden AgentOS auth surface (A2A auth, trace IDOR, PAT mint, validate=False) #8752 S3, pre-dates this branch); resource-type classification is anchored to the first path segment; the internal scheduler token keeps its strict scope check.

6. Eval suite runner (#8761)

agno.eval gains a suite layer: a frozen Case (one input to one agent + optional judge/reliability checks), a sequential single-event-loop runner (run_cases/arun_cases) that does zero console I/O and returns a SuiteResult whose to_dict() is a stable CI/JSON contract, and an argparse cli()/acli() built purely on the runner's hooks. Replaces the ~430-line runner previously copy-pasted per template. Also: show_spinner flag on the four existing eval classes, and sync-DB eval logging moved off the event loop.

7. Packaging, toolchain, and release engineering

  • Core deps 14 → 10: dropped gitpython, python-dotenv, typer (zero imports in libs/agno); python-multipart moved into the extras that serve FastAPI routes; fastapi[standard] unbundled into fastapi + python-multipart + uvicorn + websocketsagno[os] no longer drags sentry-sdk/fastapi-cloud-cli. fastmcp folded into agno[mcp] so AgentOS can serve /mcp without a separate install (a3).
  • Extras tidy (chore: tidy pyproject extras — alphabetize, close aggregate gaps, fix agno[embedders] #8759): alphabetized groups, aggregate gaps closed, underscore→hyphen key normalization (free PEP 503 aliases), and a real fix — agno[embedders] was unresolvable (dangling agno[vllm] reference). Verified by diffing the resolved closure of all 129 extras: zero removals.
  • Python floor raised to 3.9 for both packages (agno's old >=3.7 was already unsatisfiable via its own pydantic pin — honest metadata, not a support cut).
  • release.yml: OIDC trusted publishing (no long-lived API tokens), per-package "already on PyPI?" gates, explicit job permissions, concurrency guard, agno needs agnoctl.
  • format.sh / validate.sh are real gates: honest non-zero exit codes, per-target sections; mypy on libs/agno is clean for the first time (0 errors in 903 files) with the toolchain pinned at latest (mypy 2.1.0, ruff 0.15.20, pytest 9.1.1). dev_setup.sh/demo_setup.sh rebuilt on one pattern (fresh 3.12 venv, workspace agnoctl editable next to agno[dev]/agno[demo]); agno[dev] now collects and passes the entire first-party unit tree (4,767 tests) in a fresh venv; agno_infra is out of the core dev flow.

8. Cookbooks

Service accounts (cookbook/05_agent_os/middleware/), MCP demo updates (cookbook/05_agent_os/mcp_demo/), eval suites (cookbook/09_evals/suite/), and a community BGPT MCP example (#8740, cookbook/91_tools/mcp/bgpt.py).

Breaking changes (vs 2.6)

Intentional and documented in the design docs:

  1. MCP surface 19 → 8 tools. Memory/session-write tools removed; MCPServerConfig(include_tags={"memory"}) fails validation loudly. Run results trimmed by default (result_mode="full" restores the old payload).
  2. authorization=True now protects /a2a/* and /agui — authentication, per-route RBAC scopes, and server-side identity. Deployments that ran A2A/AGUI with auth enabled and relied on those routes being open now need credentials. Custom BaseInterface subclasses that verify their own inbound webhooks must set authenticates_own_requests = True (default is fail-closed).
  3. A2A tasks:get and tasks:cancel require contextId (400 instead of a deep 500; the a2a-sdk returns it in every message:send response). Client-supplied user ids on A2A/AGUI are honored only for anonymous callers.
  4. Packaging: code relying on agno transitively installing gitpython/typer/python-dotenv, or on fastapi[standard]'s bundled extras, must declare those deps itself. requires-python >= 3.9.

Alpha-tester notes (a1–a3 → a4): rotate PATs minted before config:read joined the default scopes (agno connect --rotate); AGNO_OS_URL is now AGENTOS_URL (renamed after 0.1.0a1).

Review map

Area Where
Service accounts agno/os/routers/service_accounts/, agno/os/service_accounts.py, service_accounts table in agno/db/*
Auth layer, scopes, identity agno/os/middleware/, agno/os/auth.py, agno/os/scopes.py
MCP interface agno/os/mcp.py, agno/os/mcp_results.py, agno/os/services/
Interface RBAC (A2A/AGUI) agno/os/interfaces/base.py, agno/os/interfaces/a2a/, agno/os/interfaces/agui/
CLI libs/agnoctl/
Eval suites agno/eval/, cookbook/09_evals/suite/
Packaging & release libs/agno/pyproject.toml, libs/agnoctl/pyproject.toml, .github/workflows/release.yml, scripts/

The merged sub-PRs (#8752, #8755, #8759, #8760, #8761) each carry a full standalone write-up, including per-finding provenance (what pre-dates this branch vs what v2.7 introduced) and test evidence.

Type of change

  • New feature
  • Bug fix
  • Breaking change

Checklist

  • Code complies with style guidelines
  • Ran format/validation scripts — ./scripts/format.sh and ./scripts/validate.sh are now hard gates and pass clean (mypy: 0 errors, 903 files)
  • Self-review completed — five review rounds over the auth surface (in-branch adversarial, external Codex + Opus, fix: harden AgentOS auth surface (A2A auth, trace IDOR, PAT mint, validate=False) #8752, PAT/JWT pass, fix: gate A2A authorization + self-scope service-account principals (v2.7 B1/B2/D1) #8760); the final round returned no findings
  • Documentation updated (comments, docstrings, libs/agnoctl/README.md, design docs in the private specs repo)
  • Examples and guides: service-accounts, MCP demo, and eval-suite cookbooks
  • Tested in clean environment — fresh-venv dev setup passes the full first-party unit tree; agno[os] closure and multipart upload verified in a clean venv; auth matrix validated over real uvicorn; MCP fixes reproduced against a live FastMCP server
  • Tests added/updated — os suite (unit + integration) + eval unit: 2,298 passed (the two former known failures — a stale JWKS-precedence test and a numpy-dependent serialization test — are repaired on the branch); agnoctl: 153 passed, now also run in CI

Duplicate and AI-Generated PR Check


Additional Notes

Release status

  • agno 2.7.0a1a4 + agnoctl 0.1.0a1a4 are live on PyPI (OIDC trusted publishing; the old agnoctl 0.1.0rc* pre-releases were removed, so resolvers prefer the alphas).
  • During the alpha the working commands are uvx agnoctl connect and uvx "agno==2.7.0a4" connect; bare uvx agno connect unlocks at 2.7 stable.

CI status

  • test.yml (the PR gate) is green. agnoctl now has its own CI job (ruff + mypy + its 153-test suite — the package previously published four alphas with zero CI); the style checks and test_setup.sh install the workspace agnoctl editable, so a version bump can no longer race the PyPI publish and redden checks.
  • Root causes fixed on the branch: agno[tests] had become unresolvable via chore: tidy pyproject extras — alphabetize, close aggregate gaps, fix agno[embedders] #8759's aggregate additions (infinity_client pins httpx<0.28.0 against a2a-sdk's >=0.28.1 floor; brave-search 0.2.0 ships metadata uv cannot parse) — both are excluded from the aggregates again, in the litellm/mistral style. test_setup.sh was rebuilt to fail fast (a failed install previously surfaced as 194 collection errors), and requirements.txt is regenerated from the trimmed core (closes the tracked follow-up).
  • The "Main Validation" model matrix runs on demand (workflow_dispatch) for feature branches instead of per-push: per-push it burned the full model-API matrix on every commit and inherited main's chronic model-side failures (Gemini SSL/structured-output, the dead DeepInfra tool-use path, live-external-API tests), drowning real signal. Release branches and the Saturday cron are unchanged. The branch-specific matrix failures it did surface are all fixed: a stale JWKS test, a mul-tool prompt flake, the system-test containers crashing on the trimmed core (they now install agno[os], the closure AgentOS actually declares), the dev-setup job asserting the deliberately-dropped agno-infra install, and a health gate that let a downed A2A server through as 27 ConnectErrors.
  • The chronic main-side matrix failures pre-date this branch and are tracked as a separate CI workstream.

Remaining before stable (tracked, deliberately not in this PR)

  • Port the Windows setup scripts (dev_setup.bat/.ps1, demo_setup.bat/.ps1) to the rebuilt flow.
  • Fix the chronic model-matrix failures on main (Gemini SSL/structured-output, DeepInfra tool-use, live-external-API tests) so Main Validation can gate again.
  • Drop agno-infra's agno console script (deferred to stable; co-installed pip envs have last-install-wins on bin/agno during the alpha — uvx is unaffected).
  • create_fresh=True as the default resolution mode is a v3.0 change (SDK-534).

Known, documented limitations: WS workflow reconnect under user_isolation=False replays buffered events to any connection (the documented single-tenant contract of that mode); authorization=True also gates the A2A agent card, with no anonymous-discovery escape hatch yet; per-resource scopes on a custom-prefix A2A mount degrade to the global run scope (fail-closed).

🤖 Generated with Claude Code

ashpreetbedi and others added 21 commits July 4, 2026 10:19
New table type service_accounts (default name agno_service_accounts) with a
ServiceAccount dataclass, schema definitions for postgres and sqlite, and an
optional CRUD method group on BaseDb/AsyncBaseDb implemented by the postgres,
async postgres, sqlite and async sqlite adapters. Name uniqueness applies to
active accounts only, via a new _partial_unique_indexes schema key
(UNIQUE ... WHERE revoked_at IS NULL), so a revoked name can be reused to
rotate a credential without changing the machine identity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
agno_pat_-prefixed opaque tokens: 32 random bytes base62-encoded, SHA-256
hash stored, 16-char display prefix, plaintext returned once. The
ServiceAccountVerifier resolves tokens against the AgentOS database (async
dbs awaited, sync dbs via threadpool so middleware never blocks the event
loop), fails closed on db errors, rate-limits failed lookups per client with
a bounded in-process window, and throttles last_used_at writes to one per
minute per token.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bearer tokens with the agno_pat_ prefix now authenticate as service accounts
in both auth paths. The JWT middleware verifies them against the AgentOS
database (verifier from the new service_account_verifier param or
app.state), attaches identity the same way as the JWT path (user_id =
sa:<name> principal, scopes = stored scopes), and returns a uniform 401 for
unknown/revoked/expired tokens, 429 when failed lookups are throttled, and
503 when the database is unreachable. The security-key dependency mirrors
the check so PATs work in os_security_key and open deployments too.

Service account scopes are first-party ACL data, so they are enforced
against the scope mappings in every mode - including deployments where JWT
authorization is disabled. The RBAC decision (route matching, per-resource
context, listing-endpoint filtering) is extracted into
scopes.check_route_scopes and shared by the JWT, internal-token, and
service-account branches. New service_accounts:read/write/delete scopes
gate the upcoming /service-accounts endpoints.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
POST /service-accounts mints a token (201, plaintext returned once): scopes
default to run+read, unknown scopes 400, privileged scopes (write/delete/
admin/service_accounts) require allow_privileged_scopes=true, and minted
scopes must be a subset of the creator's own scopes so a minter can never
escalate past itself. Expiry defaults to 90 days; non-expiring tokens
require never_expires=true. GET lists metadata and display prefixes only.
DELETE revokes (404 unknown, idempotent 204 when already revoked).

AgentOS builds one ServiceAccountVerifier per instance when a db is
configured, exposes it on app.state for the auth dependency, passes it to
the JWT middleware, and - because a mounted Starlette app has its own
app.state - also passes it to the MCP app's middleware so PATs work over
/mcp. The router registers in both get_app() and _reprovision_routers(),
with a disabled-503 stub when no db is configured.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
End-to-end over sqlite: an admin JWT mints a PAT, the PAT runs an agent and
the run attributes to sa:<name>; default scopes allow session reads but not
deletes or minting; the subset rule blocks delegated minters from
escalating; revoked/expired/unknown tokens all return the same 401 detail;
repeated failed lookups reach 429; revoke-then-recreate rotates a name onto
the same principal; list responses never contain hashes or plaintext.
Security-key mode and open instances verify PATs and enforce their scopes
too, and the internal scheduler token behaves identically after the RBAC
refactor. DB-layer tests cover the partial unique index and CRUD on sqlite
and postgres.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mint an agno_pat_ token for a machine identity with an admin JWT, run the
agent with it (session attributed to sa:claude-code), then list and revoke.
Verified end to end; TEST_LOG updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An adversarial review of the branch surfaced five real issues; all are fixed
with regression tests:

- Subset rule: a caller granting a per-resource scope it exactly holds (e.g.
  agents:my-agent:run) was rejected because has_required_scopes was called
  without resource context. Now parses each requested scope and passes the
  resource id, so least-privilege delegation works.
- DB outage misclassified as an invalid token: _get_table swallows
  connectivity errors and returns None, so get_service_account_by_token_hash
  read a real outage as 'unknown token' (401) and counted it toward the rate
  limiter. It now probes the connection and re-raises, so the verifier returns
  UNAVAILABLE (503) and the limiter is untouched. Fixed in all four adapters.
- Throttle could block valid tokens: the failed-lookup limiter gated before
  the DB lookup, so once a bucket filled (shared across clients behind a proxy)
  even valid tokens got 429. The token is now always looked up first; the
  limiter only shapes the failure response, never blocks a valid token.
- Listing allow-through was not GET-only, so a non-GET id-less route like
  POST /agents could be allowed through for internal-token and PAT callers.
  Restricted to GET.
- Name validation used $ instead of \Z, so a trailing newline slipped past
  the slug check; and PostgresDb/SqliteDb.from_dict dropped the new
  service_accounts_table override on rebuild. Both fixed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PAT auth previously hit the database on every request. The verifier now caches
successful verifications in a bounded, TTL'd LRU keyed by token hash (default
30s, AgnoAPISettings.service_account_cache_ttl_seconds; 0 disables it for
strict instant revocation). Only valid accounts are cached - failures are left
to the rate limiter. Token expiry is re-checked on every cache hit, so an
expired account is never served from cache.

Revocation stays native: DELETE /service-accounts/{id} evicts the local cache
entry, so the worker that processes the revoke rejects the token immediately;
other workers converge as their entry ages out within the TTL. Docs, the
cookbook, and the plan are updated to describe this timing (was 'instant').

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The /mcp run tools (run_agent/run_team/run_workflow) returned raw RunOutput
dataclasses: full transcript incl. system prompt (~8k tokens into the frontend
model's context for a ~370-token answer) and a serialization crash on binary
media. MCP tool results land directly in the consuming model's context window,
so the default result is now sized for it.

- New agno/os/mcp_results.py: trimmed ToolResult (answer text + media as MCP
  image/audio content blocks + structuredContent limited to run_id/session_id/
  status and unresolved requirements when paused). MCPServerConfig.result_mode
  ("trimmed" default, "full" escape hatch = complete media-safe to_dict()).
- Run tools drive agno's event stream and forward tool-call/step events as MCP
  progress notifications (monotonic values per the MCP spec; no-op when the
  client sends no progressToken). Long runs stay alive past client timeouts.
- Terminal states hardened: failed runs surface the real error message from the
  run-error event; paused/cancelled workflows (whose foreground streams end with
  no terminal event) are recovered via workflow.aget_run_output; nested-workflow
  events are depth-filtered so a recovered nested failure cannot abort the outer
  run; remote agents/teams/workflows use the non-streaming path.
- 13 new unit tests (trimmed/full/media/paused/error/nested/progress
  monotonicity); system-test assertions updated to the trimmed contract.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Redesigns the AgentOS MCP surface around what an LLM frontend does (discover,
run, continue, cancel, review history) instead of mirroring the REST database
console, and extracts a service layer both surfaces share.

Surface trim (19 -> 8): removes all 6 memory tools and all 5 session-write
tools plus get_session. MCP_BUILTIN_TAGS collapses to {core, session}; the
removed "memory" tag now fails Literal validation (breaking, intentional).
Deletes the two confirmed history-wiping footguns (create_session /
update_session on an existing session).

New run-lifecycle tools:
- continue_run resumes a PAUSED run with the client's resolved requirements
  (RunRequirement for agents/teams, StepRequirement for workflows). Remote
  components are rejected (mirrors the REST 400) rather than crashing on
  requirement forwarding; stream=False is pinned so a stream=True component
  can't return an iterator.
- cancel_run requests cancellation on a named component. Both tools require
  exactly one component id and enforce run ownership for scoped (non-admin)
  callers via the same session check the REST cancel/continue endpoints use,
  and call one component (the agent/team/workflow cancellation registries are
  process-global, so the old OR-fanout both leaked entries and always
  reported success).

Shared service layer (agno/os/services/): sessions.py (listing, session-type
auto-detection, per-run classification matching REST exactly, threadpool
offload for sync DBs, ownership verify) and runs.py (continue/cancel). The
REST get_sessions and get_session_runs handlers now delegate to it, so the
two surfaces cannot drift.

Ergonomics: MCP annotations on every tool (readOnlyHint on config/reads,
destructiveHint on cancel_run); descriptions state the operating workflow;
session_type is a Literal; db_id is optional (defaults on single-db
deployments); get_agentos_config is a compact discovery payload (~400 chars,
was a 15KB inline schema + full config); get_session_runs returns per-run
content only, never the message transcript / system prompt; RemoteDb calls
forward the caller's Authorization header.

Tests: test_mcp_surface_v2.py (annotations, continue/cancel identity +
requirement parsing + ownership gate + remote guard, history trimming,
service auto-detect/classify/threadpool/not-found, wiring). System suite
rewritten to the 8-tool surface. Full OS unit suite green (1419). Cookbook
README/test_client/TEST_LOG refreshed and verified with a live end-to-end run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… modes

Router dependencies never run for mounted sub-apps, so the MCP app served
at /mcp was completely unauthenticated in security-key mode, and service
account tokens were never checked there either. Add a dedicated middleware
to the MCP sub-app when JWT mode is not active, mirroring the REST rules
from agno.os.auth.get_authentication_dependency: accept the OS_SECURITY_KEY
(constant-time compare) or a valid agno_pat service account token (with
principal and scopes attached to request.state for attribution), stay open
when auth is fully disabled, and map verifier statuses to the same 401/429/503
responses as REST. JWT mode is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extends the unauthenticated /info endpoint with:
- mcp: {enabled, path} reporting MCP server availability (path /mcp when enabled)
- auth_mode: the effectively enforced authentication mode (none / security_key / jwt)

auth_mode mirrors the precedence in get_authentication_dependency: JWT
(authorization=True or JWT env config) takes precedence over the OS
security key. This lets external tools (agno connect CLI) discover the
OS configuration without probing or 401-detail sniffing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New distribution libs/agno_cli (PyPI: agnoctl, module: agno_cli, command: agno).
Talks plain HTTP to a running AgentOS; never imports the agno framework.

- agno connect: discover the AgentOS (structured /info fields with probe
  fallback), mint one service account per coding agent, write MCP configs for
  Claude Code (claude mcp add, .mcp.json fallback), Codex (config.toml
  section-scoped edit), and Cursor (mcp.json merge), then verify each
  connection with a real MCP initialize + tools/list handshake. Idempotent:
  working entries are skipped, broken ones rotated; --rotate/--skip-existing
  for non-interactive control.
- agno tokens create/list/revoke: thin wrappers over /service-accounts.
- agno status: discovered OS, auth mode, MCP URL, per-client config state.
- Agent-operator contract: --json on every command, deterministic exit codes
  (0 ok, 1 failure, 2 partial), no prompts in JSON mode, tokens never logged.
- Plugin seam: agno_cli.plugins entry-point group mounts external Typer apps.
- 54 tests against an in-memory fake AgentOS (httpx.MockTransport) and tmp
  client config dirs; ruff and mypy clean; wired into scripts/format.sh and
  scripts/validate.sh.

Design doc: specs/agno/features/agno-cli/ (uvx agno connect lands via a
follow-up two-line change in libs/agno once agnoctl is on PyPI).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…al review

One CLI: port the used surface of the legacy ag infra CLI so agno-infra can
drop its console scripts without a capability gap.

- agno create NAME [-t agentos-docker|agentos-aws|agentos-railway] [-u URL]:
  shallow-clone the template, strip git history, copy example secrets. No
  registry file: commands operate on the working directory.
- agno infra up/down/restart: the docker compose path (same detection names
  and command lines as the legacy CLI, plus --file/--volumes/--dry-run/--json).
  The legacy per-resource Docker/AWS engine is deliberately not ported; it
  plugs in later via the agno_cli.plugins entry-point group.

Review fixes (23 findings from a 29-agent adversarially verified review):
- Codex TOML rewrite no longer swallows [[array-of-tables]] sections or
  trailing comments; refuses to modify unparseable configs.
- JSON config writers refuse to clobber corrupt files and malformed
  mcpServers shapes; chmod 600 whenever a token is written.
- Claude Code: read follows real scope precedence (local > project > user);
  user-scope file fallback targets ~/.claude.json instead of a VCS-shared
  .mcp.json; subprocess calls get timeouts and no inherited stdin.
- connect: per-client isolation catches all exceptions (keeps the --json
  single-document contract); --skip-existing never touches entries pointing
  at a different AgentOS; post-write read-back verifies the entry the client
  will actually use (catches shadowing and dropped headers); --privileged
  passthrough; partial success exits 3 (2 stays click's usage-error code).
- Context-correct 409 hints; hardened response parsing; verify_mcp honors
  its never-raises contract.

78 tests; ruff and mypy clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The compose lifecycle is the hero path, and the compose file runs the whole
project (AgentOS included), so the infra noun under-described it. Root verbs
match the two-tier dev story (agno dev for in-process reload later, agno up
for container parity) and keep the quickstart to four commands:
agno create my-os && cd my-os && agno up && agno connect.

The infra command group is removed; a future infra 2.0 plugin can mount
agno deploy (or reclaim an infra group) via the agno_cli.plugins seam.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Service accounts (PAT auth), MCP surface v2, trimmed run results, shared service layer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
agnoctl CLI (agno connect, tokens, create, up/down/restart), /info discovery, /mcp auth enforcement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One name everywhere: PyPI dist, directory, module, and command alias are
all agnoctl (command stays agno). Plugin entry-point group becomes
agnoctl.plugins. Version set to 0.1.0rc2 (0.1.0rc1 is on PyPI).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- agno depends on agnoctl>=0.1.0rc1 (explicit rc pin so pip/uv resolve
  pre-releases) and exposes the agno console script pointing at
  agnoctl.main:app, per specs/agno/features/agno-cli decisions §2
- bump agno 2.6.22 -> 2.7.0a1
- release.yml: build-release-agnoctl job (continue-on-error publish like
  agno-infra); agno job waits on it so the dep always lands first

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… CLI fixes

Correctness (from the multi-angle review of the combined diff):
- Enforce service-account/JWT scopes on every MCP tool by mapping each call
  onto its REST-equivalent route and running check_route_scopes with the same
  mappings; a sessions:read PAT could previously run_agent via /mcp. Default
  mint scopes gain config:read so connect-minted tokens can still discover.
- Accept service-account tokens on the WebSocket path (REST accepted them in
  every mode; WS compared them byte-for-byte against the security key).
- Stream only native Agent/Team in MCP run tools: AgentProtocol
  implementations never yield a final RunOutput and always failed.
- get_session_runs(run_id=...) returns one run in full detail — restores the
  capability lost with get_session_run, without growing the 8-tool surface.
- include_tags=set() now registers no built-in tools instead of all of them.
- Restore the strict internal-token scope check (no GET-listing fallback) so
  scope misconfiguration 403s loudly instead of returning empty lists.
- Skip re-verification in JWTMiddleware when an outer instance already
  authenticated the request (double SA verification on /mcp).
- agnoctl connect --name: an already-connected client now hands the shared
  token to later clients instead of forcing a re-mint that revoked it.
- Marker on the agnoctl dependency (python_version >= '3.9') so agno stays
  installable on 3.7/3.8; release.yml uses skip-existing instead of
  continue-on-error so a failed agnoctl publish blocks the dependent agno job.
- Run sync service-account DB calls in a threadpool in the router.
- agnoctl license classifier corrected to Apache (matches the LICENSE file).

Cleanups: shared JWTMiddleware kwargs builder for REST + /mcp; shared JSON
config read/write helpers for the CLI adapters; session-run trimming moved
next to the run-result trimming in mcp_results.py; hash_token/get_principal
delegation; cached default scope mappings; paged find_service_account with
include_revoked=false; update_service_account can skip its re-fetch; dead
top_level/display_name removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ashpreetbedi and others added 5 commits July 4, 2026 16:02
User decision: the whole boat is a1. 0.1.0rc1 on PyPI must be YANKED when a1
publishes, or resolvers prefer rc1 (a1 < rc1 per PEP 440).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rsion gates

Replaces the publish-everything-with-soft-fail pattern: each package publishes
only when its pyproject version is absent from PyPI, real failures fail loudly
(no continue-on-error/skip-existing), agno still waits on agnoctl, and
publishing uses OIDC trusted publishing (environment: pypi) instead of
long-lived API tokens. TestPyPI steps dropped.

Requires one-time setup: add this repo/workflow/environment as a trusted
publisher on PyPI for agno, agnoctl, and agno-infra.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the _account result side-channel and the synthesized half-empty
ServiceAccount from the review fix: _resolve_shared_token reuses a verified
token from any already-connected client's config (the CLI's token store) or
mints once; the per-client loop only writes configs and can never hit a
mid-run name conflict. Per-client account metadata in results is now a
per-client-mode feature; in shared mode the account is run-scoped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…kets

Collapses the auth code that had spread across five places into one
middleware. Previously: a REST router dependency (security-key/PAT), a
separate _MCPKeyAuthMiddleware on the mounted MCP app, JWTMiddleware on the
parent app, a second JWTMiddleware mirrored onto the mounted app, and a
WebSocket path — plus /info sniffing the middleware stack.

Now AgentOS installs a single AuthMiddleware on the parent app whenever any
credential is configured (JWT, OS_SECURITY_KEY, or a service-account
verifier). Starlette runs parent-app middleware before dispatching to a
mount, so that one instance covers the REST routes and the mounted /mcp app
together — the mounted app carries no auth code at all.

- JWTMiddleware generalized to AuthMiddleware: JWT is now optional (validator
  built only when a JWT source is configured), and it accepts a security_key
  for the non-JWT path. JWTMiddleware stays as an alias. It refuses to
  construct with no credential source at all (was: refuse with no JWT key) —
  an auth layer that authenticates nothing silently authorizes everyone.
- Service-account verify-and-attach unified into
  service_accounts.authenticate_service_account_request, called by both the
  middleware and the REST dependency (was three drifting copies).
- get_mcp_server no longer adds any auth middleware; the per-tool scope gate
  is unchanged and still runs against the identity the parent layer attached.
- _has_jwt_middleware only counts instances that actually validate JWTs, so a
  security-key-only AuthMiddleware doesn't make /info report auth_mode=jwt.

Behavioral MCP-auth tests now drive the full app (auth is on the parent, not
the mount); structural tests assert on the parent AuthMiddleware. End-to-end
HTTP smoke (real uvicorn + mount) confirms no-token/key/PAT/bogus-PAT and
/info all behave. 1457 os unit tests + 32 service-account integration tests
green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
agno's >=3.7 claim was already fiction: its own pydantic dependency (2.11)
is requires-python >=3.9, so pip install agno has been failing on 3.7/3.8 at
resolution regardless; rich (>=3.8) and httpx (>=3.8) rule those out too, and
CI only tests 3.10/3.12. Both interpreters are past EOL.

Bump requires-python to >=3.9 (honest, matches the binding pydantic floor),
align classifiers to 3.9-3.13, and drop the python_version marker on the
agnoctl dependency so it is a plain agnoctl>=0.1.0a1. agno and agnoctl now
share one Python floor; agno connect works on every Python agno supports.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
kausmeows and others added 4 commits July 5, 2026 23:29
… agno[embedders] (#8759)

## Summary

A readability/organization pass over `libs/agno/pyproject.toml`'s
`[project.optional-dependencies]` (128 → 129 extras), plus one real
bugfix. 2.7 felt like the right window to tidy this up.

**Every change is zero-breakage for existing `pip install "agno[...]"`
users.** Verified by diffing the fully-resolved dependency closure of
all 129 extras before vs after: **zero removals**, only the intended
additions. Reorders never affect resolution; aggregate additions only
grow a set; the underscore→hyphen key renames are free PEP 503
name-normalization aliases (`agno[async_postgres]` still resolves to
`agno[async-postgres]`).

What changed:

- **fix:** define `vllm = ["vllm"]`. The `embedders` aggregate
referenced `agno[vllm]`, but no `vllm` extra existed — making
`agno[embedders]` (and transitively `agno[tests]`) unresolvable under
uv. There's a real vLLM embedder (`knowledge/embedder/vllm.py`) whose
local path does `from vllm import LLM`, so the extra is legitimate.
- **os cluster:** added an `# AgentOS server bundle` header (it was the
one unlabeled group) and pointed `os` at `agno[scheduler]` instead of
duplicating `croniter`/`pytz` (identical closure). `mcp` stays an
independent toggle.
- **ordering:** one law — alphabetical within Models, Tools, Storage,
Vector databases, Knowledge, and the aggregate lists. This is the main
readability win and gives new entries an obvious home.
- **aggregate coverage:** closed silent gaps so "all X" means all X —
`models` += `infinity`, `lmstudio`; `storage` += `mysql`; `tools` +=
`brave`, `daytona`, `telegram`, `whatsapp-crypto`, `you-com`.
Intentional exclusions (`litellm`/`mistral`/`crawl4ai`) remain
commented; `agno[mcp]` kept in `tools`.
- **naming:** normalized the four underscore extra keys to hyphen
(`async_postgres`, `async_mongo`, `google_bigquery`, `google_slides`).
- **docs:** inline comments on the intentional `redis`
(storage↔vectordbs) and `docling` (tool→knowledge) cross-listings so
they don't read as bugs; aligned the cookbook header.

Deliberately **not** in scope: no PEP 735 `[dependency-groups]` move for
dev/test (that one *is* a soft break), no category-prefix renames, no
codegen/pre-commit machinery — kept it to the low-maintenance, no-churn
wins.

## Type of change

- [x] Bug fix
- [ ] New feature
- [ ] Breaking change
- [x] Improvement
- [ ] Model update
- [ ] Other:

---

## Checklist

- [x] Code complies with style guidelines
- [ ] Ran format/validation scripts (`./scripts/format.sh` and
`./scripts/validate.sh`) — n/a: `pyproject.toml` metadata is not covered
by ruff/mypy. Instead validated the TOML parses, that no extra has a
dangling self-reference, and that no extra's resolved closure lost a
dependency.
- [x] Self-review completed
- [x] Documentation updated (comments, docstrings)
- [ ] Examples and guides: Relevant cookbook examples have been included
or updated (if applicable) — n/a
- [x] Tested in clean environment — before/after closure diff over all
129 extras
- [ ] Tests added/updated (if applicable) — optional guardrail test
(assert groups sorted + aggregates complete) available on request; not
included to keep the change minimal

### Duplicate and AI-Generated PR Check

- [x] I have searched existing open pull requests and confirmed that no
other PR already addresses this issue
- [ ] If a similar PR exists, I have explained below why this PR is a
better approach
- [x] Check if this PR was entirely AI-generated (by Copilot, Claude
Code, Cursor, etc.)

---

## Additional Notes

The `vllm` fix is the only functional change — everything else is
organizational. Reviewers can trust the "zero-breakage" claim
mechanically: the resolved-closure diff over all 129 extras shows only
these additions and no removals — `embedders`:+vllm,
`models`:+infinity/lmstudio, `storage`:+mysql,
`tools`:+brave/daytona/telegram/whatsapp-crypto, `tests`:+(union, incl.
vllm so it now resolves).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…v2.7 B1/B2/D1) (#8760)

## Summary

Closes the A2A/AGUI items from the v2.7 security review: the two release
blockers in the A2A router (**B1**, **B2**) and the service-account
session-visibility decision (**D1**). Both blockers were independently
reproduced live; this is the un-shipped follow-up to #8752, which
deferred A2A authorization/identity. The branch went through four
independent review rounds; the last returned no findings.

### B1 — A2A/AGUI routes enforced authentication but no authorization

`get_default_scope_mappings()` had zero entries for interface routes,
and unmapped routes default to *allow* — so under `authorization=True` a
`config:read`-only token was 403 on `POST /agents/{id}/runs` but
executed the same agent via `POST /a2a/agents/{id}/v1/message:send`
(read → execute escalation). The same hole applied to `POST /agui`.

- **Interfaces now declare their own RBAC entries** via
`Interface.get_scope_mappings()`, and `AgentOS._add_auth_middleware`
merges each non-self-authenticating interface's entries against its
**actual mount prefix** — covering custom prefixes (`A2A(prefix=...)`,
including `""`), subclasses, AGUI, and future interfaces in one
mechanism. Self-authenticating interfaces (Slack/Telegram/WhatsApp
webhook verification) are excluded from the central auth layer as
before.
- A2A entries: `message:send`/`message:stream`/`tasks:cancel` →
`<family>:run`; agent-card and `tasks:get` → `<family>:read`. AGUI:
`POST {prefix}/agui` → the bound family's `:run` (agent-wins precedence,
matching the router's dispatch).
- The deprecated `/a2a/message/{send,stream}` dispatchers resolve
agent/team/workflow at runtime, so the route gate carries a coarse
`agents:run` and the handler re-checks the resolved family's `:run` via
the canonical `check_resource_access`.
- `get_resource_context_from_path` understands the `/a2a` prefix, so
per-resource scopes (`agents:<id>:run`) authorize A2A exactly as they do
REST.
- **Root-cause guard:** a regression test asserts every mounted `/a2a`
route resolves to a non-empty required-scope set, so a future A2A route
can't silently ship ungated.

### B2 — A2A/AGUI took run identity from the client

Every A2A run/poll/cancel site read `X-User-ID` / `metadata.userId`, and
AGUI read `forwardedProps.user_id`, never `request.state.user_id` — so
any authenticated caller could attribute runs, sessions, and memory
writes to an arbitrary user (`X-User-ID: victim`), including reserved
principals (`sa:*` / `__scheduler__`), and under `user_isolation=True`
touch the victim's session (IDOR).

- All 8 A2A run/stream sites and the AGUI handler resolve identity
through a shared `resolve_run_user_id()`
(`os/middleware/user_scope.py`), mirroring the REST run route: a scoped
caller is pinned to its own principal; any other authenticated caller to
`request.state.user_id`; only an anonymous caller may supply an identity
for attribution — and never a reserved one (403).
- `tasks:get` verifies session ownership and the path component before
the read (`verify_run_in_session` with `component_type`/`component_id`,
fail-closed on cross-component runs); `tasks:cancel` does the same
before applying the global cancel intent. Admins and unscoped callers
read/cancel unfiltered, matching REST.
- `tasks:get`/`tasks:cancel` require `contextId` (the run is not
locatable without its session; a missing one previously 500'd deep in
storage, now 400). The a2a-sdk returns `contextId` in every
`message:send` response.

### D1 — service-account (PAT) session visibility

`get_scoped_user_id` **self-scopes service-account (`sa:`) principals to
the data they created**, regardless of the `user_isolation` flag, unless
the token carries admin. A default PAT (e.g. over MCP) reads only what
it created — the intended semantic; an operator who needs a cross-user
debugging token mints one with `agent_os:admin`. No change to
`DEFAULT_SERVICE_ACCOUNT_SCOPES`, so the common MCP read path is
unaffected. Concretely:

- `GET /sessions?user_id=<other>` / MCP `get_session_runs` return only
the PAT's own rows.
- On-behalf-of session/memory writes are coerced to the `sa:<name>`
principal.
- PAT `cancel`/`continue` on run-id routes require `session_id` for
scoped callers.

## Type of change

- [x] Bug fix

---

## Checklist

- [x] Code complies with style guidelines
- [x] Ran format/validation scripts (`ruff format`, `ruff check`, `mypy`
— all clean on changed files)
- [x] Self-review completed
- [x] Documentation updated (comments, docstrings)
- [ ] Examples and guides (n/a — security hardening, no cookbook
surface)
- [x] Tested in clean environment
- [x] Tests added/updated

### Duplicate and AI-Generated PR Check

- [x] I have searched existing open pull requests and confirmed none
already addresses this
- [x] Check if this PR was entirely AI-generated

---

## Additional Notes

**Security considerations:** the blockers only bite with
`authorization=True` **and** the A2A/AGUI interfaces enabled, but in
that config they fully defeated RBAC and user isolation on those
surfaces.

**Known limitations (documented, intended):**
- Per-resource scopes on a *custom-prefix* A2A mount degrade to the
global run scope (fail-**closed** 403, not a bypass); the default `/a2a`
prefix supports per-resource scopes.
- Scoped and admin A2A `tasks:get`/`tasks:cancel` require `contextId` —
REST parity (ownership can't be verified, and the run can't be located,
without identifying the session).

**Tests:** 34 integration tests across `test_a2a_authorization.py`
(scope enforcement per route incl. custom/root prefixes, identity
pinning, reserved-principal rejection, anonymous-attribution
back-compat, tasks:get/cancel ownership + component pins, the root-cause
mapping guard, dynamic-dispatch family gates) and
`test_agui_authorization.py` (scope gating per prefix, family
precedence, identity pinning); plus unit coverage for `sa:`
self-scoping. Full os suite: 2179 passed (2 pre-existing environmental
failures also present on base).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@agno-agi agno-agi deleted a comment from claude Bot Jul 5, 2026
ashpreetbedi and others added 2 commits July 5, 2026 15:42
Adds an eval *suite* layer to agno.eval: a frozen Case (one input to one agent + optional judge/reliability checks), a sequential single-event-loop runner (arun_cases/run_cases) that does zero console I/O and returns a SuiteResult whose to_dict() is a stable CI/JSON contract, and an argparse cli()/acli() built purely on the runner's hooks. Adds a show_spinner flag to the four existing eval classes and moves sync-db eval logging off the event loop. Replaces the ~430-line per-template runner with a small public surface. Targets 2.7.0a4.
Bump agno to 2.7.0a4 and agnoctl to 0.1.0a4 (alpha versions kept in lockstep). Ships the eval suite runner (Case, run_cases, cli) in agno.eval (#8761).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ashpreetbedi ashpreetbedi changed the title feat: v2.7 — service accounts, MCP interface v2, and agnoctl feat: v2.7 — service accounts, MCP interface v2, agnoctl, and eval suites Jul 5, 2026
ashpreetbedi and others added 15 commits July 6, 2026 00:27
…gregates

The extras tidy (#8759) closed aggregate gaps by adding every model/tool
extra, which broke the tests union in the published 2.7.0a4:

- models gained agno[infinity]: infinity_client pins httpx<0.28.0, which is
  unresolvable next to a2a-sdk's httpx>=0.28.1 floor once tests unions both.
- tools gained agno[brave]: brave-search 0.2.0 ships invalid metadata (a bad
  httpx specifier) that uv cannot parse at all — the reason test_setup.sh has
  always installed it with --no-deps.

Both extras remain available for direct installs; only the aggregate
references are commented out, in the same style as the litellm/mistral
exclusions. #8759's zero-breakage closure diff checked each extra
individually, which is exactly how a union-only conflict slipped through.

Verified: uv dry-run resolve of agno[tests] together with agnoctl succeeds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The lock predated the dependency-surface trim and pinned the old closure.
Regenerated with ./libs/agno/scripts/generate_requirements.sh upgrade: 27
pins including agnoctl==0.1.0a4 and httpx==0.28.1. Remaining consumers are
cookbook_setup.sh and the test-clean CI jobs; test_setup.sh no longer
preseeds from it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rebuilt in the dev_setup.sh pattern: set -e (a failed install previously
surfaced as 194 pytest collection errors instead of a setup failure),
preflight checks, fresh 3.12 venv, and a single resolve installing the
workspace agnoctl editable next to agno[tests] — so a just-bumped,
not-yet-published agnoctl version can no longer break the resolve (the
race that reddened CI in the minutes around the a4 release).

Also dropped: the stale requirements.txt preseed (the [tests] resolve does
not need it) and the agno-infra install, whose own lock is currently
self-conflicting (pydantic 2.13.4 vs typing-inspection==0.4.1) and which
no job in the test matrix imports — infra keeps its own CI coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- test_jwks_parameter_takes_precedence_over_env wrote an empty-keys JWKS
  file for the env var (its own comment promised a different kid), which now
  trips JWTValidator's fail-closed construction inside AgentOS.get_app()
  before the middleware under test is even added. The env file now carries
  the real key under kid env-key-2, so precedence is still discriminated
  (env winning would 401 on the test-key-1 token) without an empty key set.
- test_search_knowledge_document_serialization imports numpy, which is not
  part of agno[dev]; importorskip keeps the dev-venv run green while CI
  (agno[tests]) still exercises it.
- The mul-tool hook tests pinned no instructions, so the model occasionally
  answered without calling the tool (the test-agents-2 failure); they now
  instruct tool use, matching their add-tool sibling. Verified live.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dependency trim moved python-multipart out of agno's core deps, and the
system-test containers install '.' plus a hand-copied package list — the A2A
server (leanest list) crashed at startup with 'Form data requires
python-multipart', taking 27 tests down as ConnectErrors; gateway/remote
only survived via transitive providers. Installing '.[os]' gives every
container exactly the server closure AgentOS declares (python-multipart,
uvicorn, websockets, scheduler deps) instead of a list that can drift.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
test.yml:
- New style-check-agnoctl job: ruff + mypy + the 153-test agnoctl suite.
  The package published four alphas with zero CI coverage.
- style-check-agno and style-check-cookbook install the workspace agnoctl
  editable alongside agno[dev], so a version bump can no longer race the
  PyPI publish and redden the checks.

test_on_release.yml:
- feat/v2.7 removed from the push triggers: per-push it burned the full
  model-API matrix on every commit and inherited main's chronic model-side
  failures (google SSL/structured-output, deepinfra tool-use, external-API
  tests), drowning real signal. Feature branches run it on demand via
  workflow_dispatch; release branches and the Saturday cron are unchanged.
- test-dev-setup verifies agnoctl (which dev_setup.sh now installs) instead
  of agno-infra (which it deliberately no longer does).
- The system-test health wait covers the A2A and ADK servers too — a downed
  A2A container previously slipped past the gate and surfaced as 27
  ConnectError test failures instead of one unhealthy-service message.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Left unsorted by the #8760 merge; picked up by format.sh.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Third layer of the #8759 aggregate fallout, unreachable until the infinity
and brave exclusions let resolution proceed: embedders references
agno[vllm], so the tests union pulls vllm, and on a CPU runner uv
backtracks through vllm's torch pins to a 2023 sdist (0.1.3) whose build
demands CUDA_HOME. The extra itself stays for direct installs on GPU hosts.

The rebuilt test_setup.sh surfaced this as a one-screen resolver error at
the setup step — the failure mode the old script hid as 194 collection
errors. Verified: uv dry-run resolve of the tests union with
--python-platform x86_64-manylinux_2_28 (the earlier macOS-host dry run is
exactly how this layer slipped past).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Release-bound branches want the full matrix on every push without manual
dispatch; the answer to its chronic reds is fixing those tests, not
descoping the trigger. Keeps the rest of the workflow hardening (agnoctl
verification in test-dev-setup, five-service health gate).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
OpenAI's images API rejects 'style' with 400 Unknown parameter (it was
dall-e-3-only and has been removed), which broke every DalleTools
generation. The parameter is now Optional[...] = None and only forwarded
when explicitly set, so default usage works against the live API while an
explicit opt-in still passes through for compat gateways. Reproduced and
verified against the live API. Also drops the dead style arg from the
dalle cookbook.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fastmcp 3.4.3 added an always-on HostOriginGuardMiddleware with
localhost-only defaults: any deployed AgentOS upgrading fastmcp would get
421 Misdirected Request on /mcp for its real hostname before our
middleware ever ran (agno connect verify included). AgentOS keeps a
single validation engine — the opt-in transport-security middleware
driven by MCPServerConfig.allowed_hosts, with its tested 400
invalid_host/invalid_origin semantics — so the built-in guard is disabled
where the parameter exists (signature-gated: fastmcp is unpinned and
older versions lack the kwarg). Full test_mcp_server suite green on
fastmcp 3.4.3 (56 tests, incl. the transport-security block).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three failure classes, three treatments — red must mean code:

- Retired model pins (was 404 'no longer available'): the 20 stale
  gemini-2.0-flash pins move to the gemini-flash-latest alias the rest of
  the google suite already uses. All 5 structured-response tests pass
  again (their 'str has no attribute' failures were error strings landing
  in response.content, not a parsing bug).
- Provider availability is a skip, not a failure: dog.ceo 5xx (Cloudflare
  520s are routine — it is down right now) skips with the status named;
  the DALL-E media tests skip when the tool result carries an
  availability-class error (model brownout/retirement window, billing,
  quota, auth) but still fail on contract errors like the removed style
  parameter. Verified live: 3 passed, 4 skipped mid-brownout.
- Known SDK weakness is xfail(strict=False), not red: the seven
  concurrent-SSL stress tests document google-genai's shared-client TLS
  corruption (DECRYPTION_FAILED / WRONG_VERSION_NUMBER, reproduced 2/16
  locally, fails on main too). They xpass when clean and stop reddening
  the branch until the adapter-level fix (thread-local sync client,
  per-loop async client) lands as its own change.

Also pins instructions on the two mul-tool hook tests (the model
sometimes answered without calling the tool), matching their add-tool
sibling.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…erence assert

- The system-test gateway registers a second (remote) database, so the two
  get_session_runs tests must pass db_id like their get_sessions/get_traces
  siblings already do — the 400 they hit is the surface's documented
  multi-db contract, discovered on these tests' first real execution (they
  had only ever died at the container crash before).
- test_agent_use_json_schema_true_keeps_dict gets the repo's
  flaky(max_runs=3) retry mark: one CI failure against many local/branch
  passes is a transient model hiccup, and a real pass-through regression
  would still fail all three runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The repo's @pytest.mark.flaky(max_runs=N) convention was a silent no-op:
the 'flaky' plugin that implements it was never a dependency, so pytest
ignored the mark and model-nondeterminism retries never ran (which is why
test_aupdate_memory_task_with_db kept failing in CI while carrying the
mark). The plugin joins agno[integration-tests].

TestGeminiStressTest::test_high_concurrency_stress gets the same
xfail(strict=False) as the other seven concurrent-SSL stress tests — its
assert message wording differed, so the earlier sweep missed it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants