ACE-048: offload blocking work off the event loop + uvicorn worker factory

[ashwin-agami]

Summary

Behaviour-preserving availability fix (audit P3, feature F12, high/auth lane). The single asyncio worker ran argon2 verify (~50–100 ms), OIDC token exchange (up to a 10 s timeout), and per-tool-call audit INSERTs synchronously on the loop → one login / callback / tool call froze all concurrent users. And uvicorn.run(build_app(), …) passed an app instance, so --workers>1 was impossible. Now the blocking calls run off-loop, and uvicorn binds a factory import-string so --workers=N works.

Changes

• New leaf async_offload.run_blocking(fn, *args, **kwargs) (functools.partial over anyio.to_thread.run_sync).
• KDF: login handlers (oauth authorize, admin_login) offload the credential check via a new user_store.authenticate_with_own_store() that opens its own Store in the worker thread — a loop-thread SQLite connection is thread-bound and can't cross threads. authenticate stays sync (20+ existing test callers unchanged).
• OIDC: oidc_callback offloads exchange_code + verify_id_token; oidc_start/admin_oidc_start offload authorize_url.
• Audit: _call_tool's finally offloads record_tool_call (actor read on-loop, passed in). Still best-effort.
• Factory: main() → uvicorn.run("mcp_http:build_app", factory=True, workers=WORKERS) (WORKERS env, default 1). The boot-migration race is already handled by store.run_migrations' pg_advisory_lock (Cloud-Run multi-instance), and the admin seed tolerates the boot race — so no new lock was needed.

Test plan

• run_blocking runs a sync fn with args and kwargs on a worker thread + propagates exceptions; the factory import-string mcp_http:build_app resolves to a Starlette app.
• Behaviour-preserving proof: all existing oauth/admin/OIDC/tool-call tests still pass (same auth outcomes + the timing-enumeration defense — the dummy-verify still runs, now off-thread).
• Full gate green: ruff + all tests + gitleaks + lib-drift (uv run dev.py check).

Out of scope (per spec)

The query-subprocess event-loop block (known issue, parked ACE-028); a pooled/queued audit writer (the fresh-Store-per-call churn stays — follow-up); rate limiting (ACE-049, depends on this).

Spec: ACE-048

[Copilot]

Pull request overview

This PR addresses event-loop blocking in the hosted HTTP MCP server by offloading synchronous/IO-heavy operations (password verification, OIDC flows, and per-tool-call audit writes) to worker threads, and updates the server entrypoint to use a uvicorn factory import string so --workers can spawn multiple processes.

Changes:

• Add async_offload.run_blocking() (anyio thread offload) and use it in OAuth/admin login, OIDC start/callback, and tool-call audit logging.
• Add user_store.authenticate_with_own_store() to safely run credential checks in worker threads without sharing loop-thread DB connections.
• Switch mcp_http.main() to uvicorn.run("mcp_http:build_app", factory=True, workers=WORKERS) and add tests covering offload + factory resolution.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/test_async_offload.py	Adds coverage for run_blocking semantics and uvicorn factory import resolution.
packages/agami-core/src/async_offload.py	Introduces the run_blocking() helper wrapping anyio thread offload with kwargs support.
packages/agami-core/src/user_store.py	Adds authenticate_with_own_store() to open/close a Store within the worker thread.
packages/agami-core/src/oauth_server.py	Offloads password auth and OIDC network/verification work off the event loop.
packages/agami-core/src/admin.py	Offloads admin password auth off the event loop via run_blocking.
packages/agami-core/src/mcp_http.py	Offloads tool-call audit INSERTs; switches uvicorn to a multi-worker-capable factory import string.
packages/agami-core/pyproject.toml	Adds async_offload to the packaged flat module list.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ashwin-agami]

Declining this one — the premise is incorrect for this code. asyncio.CancelledError subclasses BaseException, not Exception (since Python 3.8), so except Exception: pass does not catch it: cancellation propagates out of the finally normally. The audit write is intentionally best-effort and the runtime is 3.11+. Leaving this thread open for a human call rather than self-resolving a disagreement with the bot.

[ashwin-agami]

Thanks @copilot — dispositioned all three:

Fixed (2):

• anyio not in server deps — declared `anyio>=4` directly; agreed, relying on it arriving transitively through `mcp` is fragile.
• test sys.path — added `packages/agami-core/src` to `sys.path` so the test imports `async_offload`/`mcp_http` when run in isolation, matching the other server tests.

Declined (1) — the `finally`/cancellation concern:
The premise is incorrect for this code. `asyncio.CancelledError` subclasses `BaseException`, not `Exception` (since Python 3.8), so the `except Exception: pass` around the audit write does not swallow it — cancellation propagates normally. The audit write is deliberately best-effort and the runtime is 3.11+, so there is nothing to fix here. (The await in the finally is itself a valid cancellation point; if the request is cancelled it raises out, it does not get trapped.)

[Copilot]

Thanks @copilot — dispositioned all three:

Fixed (2):...

Both fixes are confirmed in commit 002c59a:

• anyio>=4 is now declared directly in the server optional deps.
• packages/agami-core/src is added to sys.path in test_async_offload.py, matching the other server tests.

On the cancellation point — agreed, asyncio.CancelledError is a BaseException subclass since Python 3.8, so except Exception: pass does not trap it. No change needed there. CI is green.