QVAC-21806 feat: prototype bare-rpc-python as the wire transport by lauripiisang · Pull Request #3208 · tetherto/qvac

lauripiisang · 2026-07-10T10:21:25Z

🎯 What problem does this PR solve?

The Python SDK's original PoC transport (poc_heartbeat.py) hand-rolls the bare-rpc wire protocol — varint encode/decode, stream flags, message framing — because no production-grade transport existed yet (QVAC-21805).
holepunchto/bare-rpc-python appeared as a real candidate mid-development. This PR proves it out across all three RPC call shapes (unary, server-stream, duplex) and promotes it to the production transport.

📝 How does it solve it?

Adds src/qvac/bare_rpc_transport.py: BareRpcTransport wires bare_rpc.RPC directly to a spawned worker's Unix socket — no hand-rolled framing at all, just rpc.request(...) / rpc.request_with_response_stream(...) / rpc.create_bidirectional_stream(...). It mirrors the JS SDK's client/rpc/node-rpc-client.ts: create the socket, spawn the worker with the socket path + HOME_DIR as its JSON config argument, wire bare_rpc.RPC to the connection the worker dials back in on. Locating the Bare binary and worker entry point stays the caller's job — this is a thin client, not a self-contained package (that's separate packaging work, tracked outside this PR's scope).
BareRpcTransport.call/.call_stream/.call_duplex satisfy qvac._transport.Transport's async shape exactly, so test_bare_rpc_transport.py goes straight through the real generated qvac.methods stubs — no adapter layer needed.
Duplex maps onto RPC.create_bidirectional_stream's (outgoing, incoming) pair: write the JSON payload as the first outgoing chunk, then up's chunks, end() the outgoing side, and read incoming with the same buffer-and-split-on-newline handling as call_stream. The upstream pump runs as a concurrent background task, since the worker can start responding before the client finishes sending.
All three duplex-shaped methods are now proven end-to-end: text_to_speech_stream, transcribe_stream (parakeet, fed real-time-paced 16kHz mono PCM — its streaming session only decodes at roughly real-time cadence), and bci_transcribe_stream (the BCI addon's sliding-window driver, fed raw neural feature chunks against a real fixture, no pacing needed).
bare-rpc-python / compact-encoding-python aren't on PyPI yet (both v0.0.1, git-only) — pinned as an optional bare-rpc pyproject extra via git dependencies, so they never affect the default install. Verified directly against upstream: bare-rpc-python's pin is upstream's exact current HEAD; compact-encoding-python's pin is a few commits behind upstream, but bare-rpc-python itself stays on that same older commit, so no bump was needed.

🧪 How was it tested?

All three call shapes tested against a real spawned SDK worker: heartbeat unary; loadModel + completionStream via server-stream (real model inference, real completion text asserted non-empty); loadModel + textToSpeechStream, transcribeStream, and bciTranscribeStream via duplex (real synthesized audio, real transcript text, and a real BCI transcript containing the fixture's known "controversial" token, all asserted non-empty/correct).
Full sdk-python suite: 39/39 pass with these included.

Status: production transport. poc_heartbeat.py's hand-rolled codec remains for now as a reference/fallback while bare-rpc-python is still unreleased, but BareRpcTransport is the transport new code should use.

- BareRpcWorker wraps bare_rpc.RPC directly against the spawned worker's Unix socket, replacing poc_heartbeat.QvacWorker's hand-rolled framing for the unary and server-stream call shapes - its call/call_stream already satisfy qvac._transport.Transport's async shape, so it plugs straight into the real generated qvac.methods stubs with no PocTransport-style adapter - bare-rpc-python/compact-encoding-python aren't on PyPI yet -- pinned as an optional "bare-rpc" extra via git dependencies

Same fix as the QVAC-21805-python-models commit, applied to the 21806-only bare_rpc PoC files: poc_bare_rpc_transport.py's SDK default was a hardcoded developer-specific absolute path; default to ../../sdk relative to this file instead, and gate test skips on the worker actually existing rather than on QVAC_POC_SDK_DIR being set.

…am runtime - BareRpcWorker.call_duplex maps onto RPC.create_bidirectional_stream: write the JSON payload as the first outgoing chunk followed by up's chunks, end() the outgoing side, and read incoming with the same buffer-and-split-on-newline handling as call_stream. The upstream pump runs as a concurrent background task since the worker can start responding before the client finishes sending. - BareRpcWorker now implements all three Transport call shapes (call/call_stream/ call_duplex), so it plugs directly into the generated duplex stubs (transcribe_stream, text_to_speech_stream, bci_transcribe_stream) with no PocTransport-style adapter. - New test: load a real TTS model and drive text_to_speech_stream end to end against a spawned SDK worker, asserting real synthesized audio and a terminal done=True event.

…f erroring - test_poc_bare_rpc.py's skipif only covered the missing-built-worker case; the unconditional `import bare_rpc` at module level still hard-failed collection in CI, where the pr-checks-sdk-python.yml install step (`.[gen,dev]`) doesn't pull in the unpublished `bare-rpc` extra. - Wrap the import, expose BARE_RPC_AVAILABLE, and add it as a second skip condition alongside the existing worker-not-built check.

…erage to transcribe_stream and bci_transcribe_stream - New src/qvac/bare_rpc_transport.py: BareRpcTransport implements qvac._transport.Transport over bare_rpc.RPC, mirroring the JS SDK's client/rpc/node-rpc-client.ts (socket handshake + worker spawn), with the caller supplying the Bare command instead of this package resolving/bundling worker artifacts itself. - Retire tests/poc_bare_rpc_transport.py; test_poc_bare_rpc.py becomes test_bare_rpc_transport.py, testing the production module directly. - Add duplex tests for the two remaining duplex-shaped methods beyond text_to_speech_stream: transcribe_stream (parakeet, real-time-paced 16kHz mono s16le PCM) and bci_transcribe_stream (BCI addon, raw neural feature chunks against a real fixture). - Full sdk-python suite (39 tests) passes against a real spawned worker.

…gates Rebasing onto merged #3100 pulls in ruff+mypy CI for sdk-python; the bare-rpc transport predates it. Bring it into line: - ruff: typing.AsyncIterable/AsyncIterator/Sequence -> collections.abc, drop redundant forward-ref quotes, sort the test import block - mypy: assert post-connect invariants (rpc/writer set) and add a _require_rpc guard so the Transport methods don't deref Optional; add a scoped ignore_missing_imports override for the optional, git-only bare_rpc/compact_encoding extras - test: load the Qwen3 completion model with an explicit n_ctx -- it's a thinking model, so the worker reserves context for the reasoning trace and the metadata-default budget overflowed even a tiny prompt

+                    if not chunk:
+                        break
+                    await self.rpc.receive(chunk)
+            except asyncio.CancelledError:


+            self._server.close()
+            try:
+                await asyncio.wait_for(self._server.wait_closed(), timeout=5)
+            except asyncio.TimeoutError:


+                pump_task.cancel()
+            try:
+                await pump_task
+            except asyncio.CancelledError:


lauripiisang · 2026-07-20T16:56:45Z

Superseded by #3354 — the bare-rpc-python transport prototyped here ships as the production transport inside the full-fidelity Python SDK PR.

…sdk) The Python equivalent of @qvac/sdk: same capabilities, same behaviour, one contract. Distribution `tetherto-qvac-sdk`, import `tetherto.qvac_sdk` (PEP 420 namespace — proposed name, applied consistently). Supersedes the #3208 transport PoC as the production transport. - flat API: `from tetherto.qvac_sdk import Client, load_model, completion, …` (+ `.models`); ergonomic wrappers over generated stubs, re-exported flat - generated from packages/sdk/contract: pydantic models, typed async method stubs, model-type maps, error-code registries, pinned SDK version; guarded by generate.py --check - typed error hierarchy + cross-RPC reconstruction; loadModel type inference; duplex streaming sessions; VLA numpy/base64 marshaling (offloaded off the event loop); client logging; notebook SyncClient (numpy/pandas) - bare-rpc transport, all three call shapes, failing pending calls on worker disconnect instead of hanging - worker resolution (bundled wheel / QVAC_SDK_DIR / install-worker / global npm), version locked to @qvac/sdk; self-contained wheel + worker-bootstrap scripts - completion_orchestrate kept off the flat public API (JS has no client wrapper) - unit + real-worker e2e tests, runnable examples (one per capability + a notebook facade demo), and a full README ship with the package

…sdk) The Python equivalent of @qvac/sdk: same capabilities, same behaviour, one contract. Distribution `tetherto-qvac-sdk`, import `tetherto.qvac_sdk` (PEP 420 namespace -- proposed name, applied consistently). Supersedes the #3208 transport PoC as the production transport. - flat API: `from tetherto.qvac_sdk import Client, load_model, completion, …` (+ `.models`); ergonomic wrappers over generated stubs, re-exported flat - generated from packages/sdk/contract: pydantic models, typed async method stubs, model-type maps, error-code registries, pinned SDK version; guarded by generate.py --check - typed error hierarchy + cross-RPC reconstruction; loadModel type inference; duplex streaming sessions; VLA numpy/base64 marshaling (off the event loop); client logging; notebook SyncClient (numpy/pandas) - bare-rpc transport, all three call shapes, failing pending calls on worker disconnect instead of hanging; per-platform Bare resolution + exec-bit - Client/BareRpcTransport `config=` sends QvacConfig (cacheDirectory, loggerLevel, …) on connect, mirroring the JS client (QVAC_CACHE_DIR / QVAC_HOME_DIR env conveniences) - worker resolution (bundled wheel / QVAC_SDK_DIR / install-worker / global npm), version-locked; self-contained wheel + worker-bootstrap scripts - completion_orchestrate kept off the flat public API (JS has no client wrapper) - unit + real-worker tests (CLI minimal-model set; heavy parakeet/BCI/VLA marked `heavy`), runnable examples per capability (+ a notebook facade demo), and a full README ship with the package

* QVAC-21691 feat: worker + contract support for the Python client Worker-side and contract changes the Python client depends on, kept in one PR because the contract, the generated client, and the worker behaviour are CI-coupled. - move translate source-language detection into the worker (was per-client: Python lingua vs JS langdetect-text drifted); wire `from` stays an ISO code, expanded to the language name by the existing getLanguage() - add completionOrchestrate: a worker-run tool loop over a new duplex contract method, with a bounded, torn-down tool-result reader - shared cross-client conformance corpus (e2e/conformance/cases.json) + JS runner, on the minimal CLI model set (Qwen 600M, EmbeddingGemma, Whisper EN-tiny, Supertonic) - bound the logging-streaming example's generation so it can't overflow ctx * QVAC-21691 feat[api]: full-fidelity Python SDK client (tetherto.qvac_sdk) The Python equivalent of @qvac/sdk: same capabilities, same behaviour, one contract. Distribution `tetherto-qvac-sdk`, import `tetherto.qvac_sdk` (PEP 420 namespace -- proposed name, applied consistently). Supersedes the #3208 transport PoC as the production transport. - flat API: `from tetherto.qvac_sdk import Client, load_model, completion, …` (+ `.models`); ergonomic wrappers over generated stubs, re-exported flat - generated from packages/sdk/contract: pydantic models, typed async method stubs, model-type maps, error-code registries, pinned SDK version; guarded by generate.py --check - typed error hierarchy + cross-RPC reconstruction; loadModel type inference; duplex streaming sessions; VLA numpy/base64 marshaling (off the event loop); client logging; notebook SyncClient (numpy/pandas) - bare-rpc transport, all three call shapes, failing pending calls on worker disconnect instead of hanging; per-platform Bare resolution + exec-bit - Client/BareRpcTransport `config=` sends QvacConfig (cacheDirectory, loggerLevel, …) on connect, mirroring the JS client (QVAC_CACHE_DIR / QVAC_HOME_DIR env conveniences) - worker resolution (bundled wheel / QVAC_SDK_DIR / install-worker / global npm), version-locked; self-contained wheel + worker-bootstrap scripts - completion_orchestrate kept off the flat public API (JS has no client wrapper) - unit + real-worker tests (CLI minimal-model set; heavy parakeet/BCI/VLA marked `heavy`), runnable examples per capability (+ a notebook facade demo), and a full README ship with the package * QVAC-21691 doc: Python SDK page on the docs website - add content/docs/python-sdk.mdx: install (package + required worker), the flat API, configuring the SDK (config=/QVAC_CACHE_DIR), the notebook facade, and consistency-with-the-TS-SDK notes - wire it into the sidebar (custom-tree) and link it from installation * QVAC-21691 infra: Python SDK CI — fast checks, gated fast e2e, cross-platform full e2e - pr-checks-sdk-python.yml: fast, every PR — generate --check, black, ruff, mypy, mocked unit tests (real-worker tests skip themselves here). - on-pr-sdk-python-e2e.yml: `verified`-label-gated, hosted — builds the worker and runs the fast real-worker set (`pytest -m "not heavy"`) on the minimal CLI model set. ~2 min cold. - on-pr-sdk-python-e2e-full.yml: `test-e2e-full`-gated, self-hosted win/mac/linux matrix — the full suite incl. the heavy parakeet/BCI tests, reusing the SDK e2e's model cache (via the client's cacheDirectory config; key falls back to the SDK's qvac-models-<hash>), proving the client cross-platform. * QVAC-21691 fix: review round 1 (arun #3/#4/#6/#7) - #6 __main__: print "python -m tetherto.qvac_sdk install-worker" (module was renamed; the old command failed) - #7 EmbedFailedError: move to errors.py, read code from the generated registry instead of hardcoding 52401 (only hardcoded code in the package) - #4 JS parity: add TRANSLATION_FAILED to the JS reconstructors so both clients rebuild the typed error (translate detection is now server-side) - #3 orchestrate: emit stopReason=maxToolTurns on the truncation frame so a capped run is distinguishable from a natural finish (contract + Python regen) * QVAC-21691 fix: cross-platform Python transport over loopback TCP (arun #1) - The Python transport used asyncio.start_unix_server, which is Unix-only, so every real-worker path was broken on Windows while Windows was in the e2e matrix. asyncio has no cross-platform server for Unix sockets or named pipes. - Bind a loopback TCP port (127.0.0.1:0) on every OS and hand the worker a tcp://127.0.0.1:<port> endpoint. One code path everywhere; the Linux/macOS e2e legs exercise the exact path Windows uses. - Worker: createIPCClient now dials tcp:// endpoints via connect(port, host), falling back to the filesystem-socket/named-pipe path the Node client uses. - Tests: _worker_env resolves bare.exe as well as bare, so the Windows leg runs the real-worker suite instead of silently skipping it. Validated: connect, unary, streaming completion, config passthrough, and worker-disconnect fast-fail all pass over the TCP transport. * QVAC-21691 fix: make cancel reach a running orchestrate turn (arun #2) - orchestrateCompletion dropped the orchestrate requestId, so each inner completionStream turn ran under a server-generated id the client never saw; cancel({requestId}) matched nothing and the Stop button was a no-op. - Thread the orchestrate requestId into every turn. Turns run sequentially, so turn N's registry entry is disposed before turn N+1 begins -- no id collision -- and a cancel that lands between turns is caught by the registry's cancel-before-begin tripwire. - Add a real-worker e2e (test_completion_orchestrate_cancel_stops_generation): start a long generation, wait for the first delta, cancel by requestId, assert the run raises InferenceCancelledError. Plus JS unit guards: both turns carry the orchestrate id, the cap frame is marked stopReason=maxToolTurns, and a natural finish leaves stopReason unset. * QVAC-21691 test: cover the worker tool loop in the conformance corpus (arun #5) - completionOrchestrate was absent from the shared corpus, so the worker-driven tool loop -- the feature most likely to drift -- had no cross-client check. - Add an orchestrate-tool-loop case (data-driven tool spec + fixed result so it stays language-neutral). The Python runner drives it via completion_orchestrate and asserts the final answer reflects the tool result. - The JS client has no worker-loop wrapper (it orchestrates tool loops client-side), so its runner logs a documented SKIP instead of reimplementing the loop. The corpus description records the asymmetry. Validated cold on both runners: Python case passes; JS runner 9/9 with the orchestrate case skipped. * QVAC-21691 infra: run full e2e on the runners' pre-provisioned toolchain - Per infra, jobs must not run toolchain "setup" on the self-hosted GPU runners; Python, Node, and bun are already provisioned there. - Drop actions/setup-python (it also hard-fails: its python-versions tarball hardcodes `mkdir /Users/runner/hostedtoolcache`, unwritable because the runner user's home is /Users/actions-runner-*), plus actions/setup-node and oven-sh/setup-bun. Use the present Python via a venv (guarded at >=3.10) and the present node/bun directly. - Save the model cache even on test failure so the multi-GB models fetched this run aren't re-downloaded next time. * QVAC-21691 fix: Windows portability (utf-8 generation + path-agnostic tests) - generate.py wrote/read files without an explicit encoding, so on Windows (cp1252 default) generated modules were emitted as non-UTF-8 -- an em-dash in a docstring became byte 0x97, and every UTF-8 reader (black, the import machinery) then failed. Pin encoding="utf-8" on all read_text/write_text. - client.py read the SDK package.json without encoding; pin it too. - test_client.py asserted derived worker/bare paths against POSIX string literals; the code returns OS-native paths, so compare via pathlib (Path(...) / .as_posix()) to pass on Windows without weakening the check. Validated on the full cross-platform e2e leg: the transport and all runtime functionality already pass on Windows; these fix the generation + path-string test failures that were Windows-only. * QVAC-21691 infra: run full e2e using the runners' provisioned toolchain - Per infra, don't run actions/setup-python on the self-hosted GPU runners: Python is already provisioned, and setup-python hard-fails anyway (its python-versions tarball hardcodes `mkdir /Users/runner/hostedtoolcache`, unwritable because the runner user's home is /Users/actions-runner-*). Use the present Python via a venv (guarded at >=3.10). - node/bun are still set up with their actions: unlike setup-python they work on these runners (they find the pre-cached tool, no download), and bun is not otherwise on PATH. These are the same actions the SDK's own desktop e2e uses. - Save the model cache even on test failure so the multi-GB models fetched this run aren't re-downloaded next time. * QVAC-21691 fix: Windows generation determinism + parakeet stream retry - generate --check compared fresh vs committed output by filecmp stat, so on Windows the platform-default CRLF a fresh build writes flagged every file as differing from the committed LF tree. Compare newline-normalized content instead; real content drift still fails. - The parakeet streaming test asserted a transcript from a single session, but a session can yield zero events on the first attempt (a known warm-up quirk the SDK's own runner recovers from via recoveryMaxAttempts). Retry a few times before failing; it surfaced as an intermittent empty transcript on CI. - run prettier over the conformance runner so the SDK pod format check passes. --------- Co-authored-by: Opanin Akuffo <46673050+opaninakuffo@users.noreply.github.com>

lauripiisang force-pushed the QVAC-21805-python-models branch 2 times, most recently from 317191a to ea37864 Compare July 10, 2026 11:19

lauripiisang force-pushed the QVAC-21806-bare-rpc-python branch 2 times, most recently from 9c61af3 to 099064e Compare July 10, 2026 15:53

lauripiisang force-pushed the QVAC-21805-python-models branch from fd3f50b to feb6336 Compare July 13, 2026 21:56

lauripiisang mentioned this pull request Jul 13, 2026

QVAC-21805 feat: generate Python pydantic models + typed method stubs from the SDK contract #3100

Merged

lauripiisang force-pushed the QVAC-21806-bare-rpc-python branch from 099064e to 30691d0 Compare July 13, 2026 21:58

lauripiisang force-pushed the QVAC-21805-python-models branch 2 times, most recently from 2e47338 to 765abea Compare July 15, 2026 15:28

Base automatically changed from QVAC-21805-python-models to main July 16, 2026 12:48

lauripiisang added 6 commits July 17, 2026 01:10

lauripiisang force-pushed the QVAC-21806-bare-rpc-python branch from 9d55aa7 to c9d5d60 Compare July 16, 2026 22:37

github-code-quality Bot found potential problems Jul 16, 2026

View reviewed changes

lauripiisang closed this Jul 20, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

QVAC-21806 feat: prototype bare-rpc-python as the wire transport#3208

QVAC-21806 feat: prototype bare-rpc-python as the wire transport#3208
lauripiisang wants to merge 6 commits into
mainfrom
QVAC-21806-bare-rpc-python

lauripiisang commented Jul 10, 2026 •

edited

Loading

Uh oh!

lauripiisang commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

lauripiisang commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🎯 What problem does this PR solve?

📝 How does it solve it?

🧪 How was it tested?

Uh oh!

lauripiisang commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

lauripiisang commented Jul 10, 2026 •

edited

Loading