QVAC-21691 feat: Python SDK full-fidelity (client, errors, sessions, vla, logging, notebook, worker orchestration, plugin-gen, wheel)#3332
Draft
lauripiisang wants to merge 19 commits into
Draft
Conversation
- BareRpcWorker wraps bare_rpc.RPC directly against the spawned worker's Unix socket, replacing poc_heartbeat.QvacWorker's hand-rolled framing for the unary and server-stream call shapes - its call/call_stream already satisfy qvac._transport.Transport's async shape, so it plugs straight into the real generated qvac.methods stubs with no PocTransport-style adapter - bare-rpc-python/compact-encoding-python aren't on PyPI yet -- pinned as an optional "bare-rpc" extra via git dependencies
Same fix as the QVAC-21805-python-models commit, applied to the 21806-only bare_rpc PoC files: poc_bare_rpc_transport.py's SDK default was a hardcoded developer-specific absolute path; default to ../../sdk relative to this file instead, and gate test skips on the worker actually existing rather than on QVAC_POC_SDK_DIR being set.
…am runtime - BareRpcWorker.call_duplex maps onto RPC.create_bidirectional_stream: write the JSON payload as the first outgoing chunk followed by up's chunks, end() the outgoing side, and read incoming with the same buffer-and-split-on-newline handling as call_stream. The upstream pump runs as a concurrent background task since the worker can start responding before the client finishes sending. - BareRpcWorker now implements all three Transport call shapes (call/call_stream/ call_duplex), so it plugs directly into the generated duplex stubs (transcribe_stream, text_to_speech_stream, bci_transcribe_stream) with no PocTransport-style adapter. - New test: load a real TTS model and drive text_to_speech_stream end to end against a spawned SDK worker, asserting real synthesized audio and a terminal done=True event.
…f erroring - test_poc_bare_rpc.py's skipif only covered the missing-built-worker case; the unconditional `import bare_rpc` at module level still hard-failed collection in CI, where the pr-checks-sdk-python.yml install step (`.[gen,dev]`) doesn't pull in the unpublished `bare-rpc` extra. - Wrap the import, expose BARE_RPC_AVAILABLE, and add it as a second skip condition alongside the existing worker-not-built check.
…erage to transcribe_stream and bci_transcribe_stream - New src/qvac/bare_rpc_transport.py: BareRpcTransport implements qvac._transport.Transport over bare_rpc.RPC, mirroring the JS SDK's client/rpc/node-rpc-client.ts (socket handshake + worker spawn), with the caller supplying the Bare command instead of this package resolving/bundling worker artifacts itself. - Retire tests/poc_bare_rpc_transport.py; test_poc_bare_rpc.py becomes test_bare_rpc_transport.py, testing the production module directly. - Add duplex tests for the two remaining duplex-shaped methods beyond text_to_speech_stream: transcribe_stream (parakeet, real-time-paced 16kHz mono s16le PCM) and bci_transcribe_stream (BCI addon, raw neural feature chunks against a real fixture). - Full sdk-python suite (39 tests) passes against a real spawned worker.
…gates Rebasing onto merged #3100 pulls in ruff+mypy CI for sdk-python; the bare-rpc transport predates it. Bring it into line: - ruff: typing.AsyncIterable/AsyncIterator/Sequence -> collections.abc, drop redundant forward-ref quotes, sort the test import block - mypy: assert post-connect invariants (rpc/writer set) and add a _require_rpc guard so the Transport methods don't deref Optional; add a scoped ignore_missing_imports override for the optional, git-only bare_rpc/compact_encoding extras - test: load the Qwen3 completion model with an explicit n_ctx -- it's a thinking model, so the worker reserves context for the reasoning trace and the metadata-default budget overflowed even a tiny prompt
- Client spawns/connects a worker and owns its BareRpcTransport; async context manager (connect/close) plus a transport property to pass straight into the generated qvac.methods stubs - per-field worker/bare path resolution, each resolved independently: constructor arg, then env (QVAC_WORKER_PATH / QVAC_BARE_PATH), then derived from sdk_dir / QVAC_SDK_DIR - WorkerNotFoundError with an actionable message; the worker is installed separately (thin client, no bundling in phase 1) - src/qvac/__init__.py exports Client and WorkerNotFoundError - unit tests for path resolution + a real-worker heartbeat round-trip
- qvac/errors.py: QvacError base carrying name/code/timestamp/remote_stack
with cause chained via __cause__; RPCError fallback preserving the JS
is_qvac_error flag semantics
- reconstruct_error() ports client/rpc/rpc-error.ts: rebuilds
RequestIdConflictError / RequestNotFoundError /
RequestRejectedByPolicyError / ContextOverflowError from the envelope's
typedFields so isinstance() holds across the RPC boundary; unknown names
fall through to RPCError; a throwing reconstructor falls back rather
than masking the original error
- BareRpcTransport now raises reconstructed typed errors instead of
RuntimeError("worker: ...")
- api.py's five ad-hoc exceptions reparented onto QvacError (moved to
qvac.errors, re-exported from qvac.api for compatibility) with their
sdk-errors-server.ts codes
- exported from the qvac package root; 14 unit tests for reconstruction,
remote-context attachment, cause chaining, and hierarchy identity
- qvac/model_types.py ports schemas/model-types.ts, engine-addon-map.ts, model-src-utils.ts, and load-model-validation.ts: canonical set, alias map, legacy-engine resolution (incl. onnx-tts -> tts-ggml routing), infer_model_type_from_model_src over ModelConstant/dict descriptors, assert_model_src_matches_model_type, model_src_to_wire - qvac.api.load_model: ergonomic wrapper mirroring client/api/load-model.ts -- infers modelType from the descriptor when omitted (ModelTypeRequiredError otherwise), DeprecationWarning + normalization for aliases, mismatch validation, hot-reload-config path via modelId, client-generated requestId threaded on the wire (cancel targeting), on_progress switches to the streaming shape - errors.py gains the load-path classes with their sdk-errors codes: ModelLoadFailedError, ModelTypeRequiredError, ModelSrcTypeMismatchError, StreamEndedError, InvalidResponseError - 15 unit tests against a fake transport + a real-worker e2e for the inference path
- qvac.api.translate ports client/api/translate.ts: for LLM-backed translation with `from_` omitted, the source language is auto-detected from the text and threaded onto the wire; undetermined detection raises TranslationFailedError telling the caller to pass `from_`. NMT models are per-language-pair, so from/to don't apply. - TranslateRun mirrors the JS return shape as asyncio handles: token_stream (live tokens; empty in non-stream mode), text (awaitable; "" in stream mode), stats (resolved on the terminal done chunk); the non-stream text task starts eagerly like the JS promise - qvac/language_detect.py mirrors @qvac/langdetect-text's detectOne surface (ISO 639-1 code + English name, und/Undetermined fallback), backed by lingua as the optional `langdetect` extra -- absent install raises an actionable error instead of failing silently - TranslationFailedError added to the QvacError hierarchy (52405) - 8 unit tests against a fake transport + a real-worker e2e for the autodetect path (mechanism asserted, not translation quality -- same stance as the SDK e2e's autodetect case)
- qvac/sessions.py ports the JS bidirectional session objects (client/api/transcribe.ts, bci-transcribe.ts, text-to-speech.ts): push-style write()/end()/aclose() over the pull-based call_duplex via an asyncio.Queue upstream adapter, single-use iteration, async context manager, write-after-close raises the session's typed error - transcribe_stream_session routes modes like JS: plain text, metadata segments, or typed conversation events (VadEvent / EndOfTurnEvent / TextEvent / SegmentEvent) under emitVadEvents/parakeetStreamingConfig - bci_transcribe_stream_session threads a client-generated requestId on the wire and exposes it for cancel() targeting; streamOpts passthrough - text_to_speech_stream_session accepts str (UTF-8-encoded) or bytes fragments and yields every frame including the terminal done frame, matching the JS asymmetry with the transcription sessions - per-frame error fields raise TranscriptionFailedError / TextToSpeechStreamFailedError (new QvacError members, 52403/52415) - 11 unit tests over a fake duplex transport + 2 real-worker e2e tests (TTS session; paced concurrent-write parakeet transcription session) - pin n_ctx in the completion poc test: Qwen3 reserves thinking-trace context and the worker fits the default window to device memory, so an empty modelConfig overflowed intermittently (same fix as the bare-rpc-transport suite)
- qvac/vla.py ports client/api/vla.ts + vla-helpers.ts: vla() encodes float32 images/state/noise, int32 tokens, uint8 mask as base64 onto the vlaRun plugin request and decodes the response's action chunk back to float32; vla_hparams() sizes buffers and reports the ggml backend - vla_preprocess_image is a vectorized numpy port of vlaPreprocessImage (bilinear resize + bottom-right letterbox + [-1,1] normalize, hwc/chw, [0,255]-vs-[0,1] scale autodetection); vla_pad_state zero-pads to maxStateDim -- both verified BYTE-IDENTICAL to the JS implementation against reference vectors generated by running the SDK's own vla-helpers.ts (fixtures/vla_reference.json) - numpy ships as the optional `vla` extra; entry points raise an actionable install hint when it's absent - 9 unit tests (byte-parity, letterbox/scale behavior, wire marshaling) + a real-worker e2e running synthetic SmolVLA inference end to end (hparams sizing, preprocessed frames, BOS-only tokens), mirroring the SDK e2e vla executor
- qvac/logging_streams.py ports client/logging-stream-registry.ts, api/logging-stream.ts, and api/subscribe-logs.ts: logging_stream() over the generated stub, a module-global per-model registry with background asyncio pumps dispatching to any stdlib-shaped logger (warn -> warning), duplicate-start warns and no-ops, stop cancels the pump, subscribe_server_logs() over the reserved SDK_ALL_LOG_ID fan-in with an unsubscribe callable; SDK_LOG_ID/SDK_ALL_LOG_ID constants - api.load_model gains the JS logger side effect (best-effort startLoggingStreamForModel on success, warning on failure) and api.unload_model now performs the stopLoggingStreamForModel cleanup - qvac/profiling.py ports profiling/envelope.ts + the per-call slice of the profiled send path: __profiling request meta (enabled/id/ includeServer), response meta extraction/stripping, and profiled_call() wrapping one unary call with client wall time + the worker's server/ delegation/operation breakdown - generate.py resolve_titles now restores the real qvac modules it evicts from sys.modules: deleting without restoring made a later in-process import re-create a fresh module copy, silently forking module-level state (the logging registry) from what callers already held - deterministic completion tests: bound + seed generation (predict/temp/seed) -- Qwen3's thinking trace otherwise rambles nondeterministically past n_ctx mid-stream, a flaky CONTEXT_OVERFLOW - 6 unit tests (level dispatch, registry lifecycle, all-logs subscription, load/unload side effects, envelope round-trip, profiled_call timing/breakdown)
- qvac/notebook.py: SyncClient, a blocking facade for notebooks -- runs the async transport on a background event-loop thread so every method is a plain call; wraps an injected transport or owns a qvac.Client (spawn + connect + close), context-manager lifecycle - embeddings return numpy float32 arrays (1-D single / 2-D batch); embed_frame() returns a pandas DataFrame indexed by input text - completion() streams deltas live into the cell (IPython display when importable, incremental stdout fallback) and returns the full text - audio interop: transcribe() accepts a mono numpy int16/float32 PCM array (wrapped into an in-memory 16-bit WAV for the wire), a file path, or raw file bytes; text_to_speech() returns float32 PCM as numpy - `notebook` extra (numpy + pandas); pandas-stubs added to dev so mypy checks the DataFrame surface for real - 7 unit tests from plain sync code over a fake transport (facade threading, result shapes, WAV wire round-trip, stdout live fallback) + a real-worker e2e running the whole notebook flow blocking end to end (spawn, load, seeded completion, unload)
Client half of the completion orchestration task -- ports
client/api/completion-stream.ts and utils/aggregate-events.ts:
- qvac/completion.py: completion() starts the wire call eagerly and
returns a CompletionRun (request_id available synchronously for
cancel targeting, ordered typed `events` stream, awaitable `final`,
text/stats/tool_calls conveniences)
- the fold aggregates content/thinking deltas, stats, tool calls
(attaching caller-registered handlers as invoke()), raw full text, and
stop reason; cacheable_assistant_content ports cache-normalize.ts
(think-block stripping) so pushing it back into history keeps hitting
the server's auto-cache; tool-call turns get no cache string
- cancellation contract mirrors JS: events end normally on a cancelled
completionDone, final rejects with InferenceCancelledError carrying
the partial text/tool-calls/stats; an error stopReason wins over
cancelled and rejects with CompletionFailedError
- tools accept the full wire Tool dict or the simplified
{name, description, parameters, handler} form; handlers are stripped
before the wire and attached to the folded tool calls
- CompletionFailedError / InferenceCancelledError join the QvacError
hierarchy (52406/52419)
- 7 unit tests over a fake transport + a real-worker e2e folding a
seeded run end to end
The worker-side orchestration move (tool-callback duplex message,
worker-run tool loop) lands separately on this same task.
…plex
Moves the multi-turn tool loop into the worker (QIP-2), with the client
only executing tool handlers on demand:
- contract: new `completionOrchestrate` duplex method. Request = the
completion params + maxToolTurns (default 8). Downstream frames carry
either the inner turn's events, a `toolCallback` (callId/name/
arguments) asking the CLIENT to run a tool, or the terminal done.
Upstream: newline-delimited {callId, result|error} JSON lines after
the initial request payload
- worker: handlers/completion-orchestrate.ts runs the loop -- generate a
turn via the completionStream plugin, forward its events, and when the
turn requests tools, emit callbacks, block on the client's results
(5-minute per-tool ceiling), extend history (assistant turn keeps the
raw call syntax; results go back as `tool` messages), repeat up to the
cap. The orchestration core is pure over runTurn/reader so it
unit-tests without the plugin registry; ToolResultReader reassembles
JSON lines across chunk boundaries
- deliberately NO delegated handler: tool callbacks execute code on the
client machine, so only the user's own local worker may orchestrate
- python: qvac.completion.completion_orchestrate() -- handlers required
for every tool, callbacks executed off the duplex stream with results
(or handler errors) written back up; run.final folds the last turn
- 6 TS unit tests (loop, history threading, handler-error forwarding,
turn cap, line reassembly, timeout) + 3 Python unit tests + a
real-worker e2e over the production transport against a freshly built
worker; full python suite (154) green against that worker. The e2e
lives in the bare-rpc suite: orchestration keeps the upstream open, so
it needs a transport that pumps upstream concurrently, which the PoC
harness by design does not
- contract re-exported; python client regenerated
- sdk: scripts/contract/plugin-contract.ts renders one registered plugin's wire contract -- JSON Schema per handler in the same input/output wire shapes as the built-in export (toWireJsonSchema now exported), handlers sorted, streaming flagged. Custom plugins register at runtime, so plugin authors run scripts/export-plugin-contract.ts on their plugin module rather than this being part of contract:export - sdk-python: generate.py --plugin <contract.json> [--plugins-out DIR] turns each contract into a typed module through the same pipeline (datamodel-code-generator with the shared GeneratedBaseModel flags): pydantic models per handler request/response plus one async function per handler over invoke_plugin / invoke_plugin_stream, black+isort'd - fixtures/echo_plugin_contract.json: the e2e echo-plugin fixture's exported contract, checked in as the generation test subject - 1 TS unit test (handler sorting, streaming flag, optional-field required handling) + 2 Python tests (generated module round-trips both call shapes over a fake transport; strict/required validation carried through from the plugin's zod shapes)
- scripts/build_wheel.py stages the platform Bare runtime, the built worker dist (plus the sdk package.json that declares its ESM type -- Bare loads .js as CJS without it), and the worker's PRODUCTION dependency closure (npm ls --omit=dev; dev deps like electron would balloon the wheel by gigabytes) under qvac/_bundle/, then builds a platform-tagged wheel; the staging dir is transient and gitignored, with a hatchling `artifacts` override so the wheel actually carries it past the VCS-ignore exclusion (a plain build still produces the thin wheel) - Client path resolution gains a bundle tier between env vars and sdk_dir: an installed bundled wheel spawns its own worker with zero configuration - `release` extra pins the build frontend - verified end to end: built the 2.0GB darwin-arm64 wheel, installed it with the bare-rpc extra into a clean venv, and ran a zero-config `async with Client()` heartbeat round trip -- no paths, no env, no sdk checkout. Publishing automation (per-OS runners, PyPI size-limit request) and per-platform prebuild trimming inside addon packages are the follow-ups; models are never bundled
Contributor
License compliance — cleanNo new dependency license findings in this PR. Warn-only (shadow) mode — this check does not block merges yet. Updated automatically by the canonical license compliance workflow. NOTICE presence (advisory)Missing NOTICE (advisory, does not block):
|
tests/test_e2e_sdk_parity.py holds the Python client to the JS client's output bar, over the production BareRpcTransport (not the PoC harness): - completion arithmetic (2+2→4, 5+5→10) and multi-turn recall (→42): prompts + expectations copied verbatim from completion-tests.ts smoke cases, DETERMINISTIC (temp 0, seed 42), LLAMA_3_2_1B (the SDK `llm` resource), through qvac.completion.completion's fold - embedding semantic similarity (GTE_LARGE_FP16, the SDK `embeddings` resource): asserts related texts embed closer than unrelated -- the real bar the SDK's embed-semantic-similarity is named for (it only checks type: array) - OCR (OCR_LATIN + OCR_CRAFT, the SDK `ocr` resource) on the SDK's own ocr-simple-test fixture, contains-any expectation from ocr-tests.ts Gated like the bare-rpc suite (bare-rpc extra + built worker); all models are the SDK e2e's own smoke resources. Full sdk-python suite now 161 tests, green against a real spawned worker.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| if not chunk: | ||
| break | ||
| await self.rpc.receive(chunk) | ||
| except asyncio.CancelledError: |
| self._server.close() | ||
| try: | ||
| await asyncio.wait_for(self._server.wait_closed(), timeout=5) | ||
| except asyncio.TimeoutError: |
| pump_task.cancel() | ||
| try: | ||
| await pump_task | ||
| except asyncio.CancelledError: |
| feeder.cancel() | ||
| try: | ||
| await feeder | ||
| except _asyncio.CancelledError: |
| ) | ||
|
|
||
| try: | ||
| import bare_rpc # noqa: F401 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
The entire full-fidelity phase of the Python SDK client (epic QVAC-21691),
built as one combined branch for review. Base is the QVAC-21806 bare-rpc
transport branch (#3208), so this diff is only the new work — 11 tasks, one
task-ID-prefixed commit each (QVAC-21808 is two). Intended to be carved into
per-task PRs later; this is the review vehicle to see it whole.
Everything lands on top of the already-merged generated client (QVAC-21805,
#3100) and the transport (#3208).
Commits (dependency order)
qvac.ClientQvacErrorhierarchy + RPC error reconstruction across the boundaryTranslateRunhandleswrite()/end()sessions for transcribe/bci/tts streams__profilingenvelopeSyncClientblocking facade, numpy/pandas returns, live cell streaming, numpy audioCompletionRun+buildFinalFromEventsport (content/thinking/tools/stats/cache)completionOrchestrateduplex contract — worker runs the tool loop, client answers tool-callback framesexport-plugin-contract.ts+generate.py --plugin→ typed module for any custom pluginbuild_wheel.pybundles Bare runtime + worker; zero-configClient()New public surface (all under
qvac)Client/SyncClientqvac.errors—QvacError,RPCError,ContextOverflowError,RequestRejectedByPolicyError, … +reconstruct_errorqvac.api.load_model/translate,qvac.completion.completion/completion_orchestrateqvac.sessions—transcribe_stream_session/bci_transcribe_stream_session/text_to_speech_stream_sessionqvac.vla,qvac.logging_streams,qvac.profiling,qvac.notebookThe one cross-package change:
completionOrchestrate(QVAC-21808)New duplex RPC method in
packages/sdk. The worker now runs the multi-turn toolloop (QIP-2); when the model requests a tool it emits a
toolCallbackframe, theclient executes the registered handler and writes
{callId, result}backupstream. No delegated handler — tool callbacks run local code, so only the
user's own local worker may orchestrate. Contract re-exported; Python client
regenerated.
Verification (every commit gated before it landed)
generate.py --check, black, ruff, mypy (-p qvac/scripts/tests), and the full pytest suite — 156 tests against a real spawned worker(unit + real-worker e2e).
tsc --noEmit,test:unit(7 new tests),contract:check.run:local:desktop --filter completion-→ 43/43 passed.venv, ran a zero-config
async with Client()heartbeat — no paths, env, orsdk checkout.
Follow-ups (not in scope here)
prebuild trimming to shrink the wheel. Models are never bundled.
🤖 Generated with Claude Code