cowork-to-code-bridge — async local execution bridge for Haystack pipelines running in sandboxes

[abhinaykrupa]

cowork-to-code-bridge — offload heavy work from a Haystack pipeline to your local machine

Hey Haystack community,

When I run Haystack pipelines in cloud sandboxes or containers, I keep running into the same friction: the pipeline can't easily reach local tools, files, or GPU resources on my development machine. I built cowork-to-code-bridge to solve this: an async, file-based RPC bridge that lets a sandboxed pipeline queue tasks for Claude Code on the local machine and collect results without opening any ports.

Integrating with a Haystack custom component

from haystack import component
from cowork_to_code_bridge.client import queue_task, poll_task_result

@component
class LocalIndexerBridge:
    """
    Haystack component that delegates document indexing to the local machine.
    Useful when your vector store or GPU lives outside the sandbox.
    """

    @component.output_types(indexed_count=int, metrics=dict)
    def run(self, documents: list[dict]) -> dict:
        # Queue the heavy work — non-blocking
        task_id = queue_task(
            tool="python",
            args={
                "script": "index_documents.py",
                "payload": documents,
            },
            idempotency_key=f"index-batch-{hash(str(documents[:3]))}",
        )

        # Poll for completion; Haystack can checkpoint here if needed
        result = poll_task_result(task_id, timeout=300)
        if result["status"] != "success":
            raise RuntimeError(f"Local indexer failed: {result.get(error)}")

        return {
            "indexed_count": result["output"]["count"],
            "metrics": result["output"].get("metrics", {}),
        }

Wire this into any Pipeline like any other Haystack component — the rest of your pipeline doesn't need to know the work ran on a different machine.

Key capabilities

Feature	Detail
queue_task / poll_task_result	Non-blocking async task dispatch + polling
Loop detection	Prevents runaway retry spirals mid-pipeline
Per-op metrics	Latency and cost tracked per delegated task
Budget cap	Hard token/cost ceiling for the whole session
No open ports	File-based transport — works in any network-restricted environment

The real-world motivation

Haystack's RAG pipelines are great for composition, but embedding models and vector store writes often need local GPU or large-RAM machines. The bridge lets you keep the orchestration in the cloud and push the expensive ops down to the metal — with idempotency keys so a crashed pipeline can resume safely.

Repo: https://github.com/abhinaykrupa/cowork-to-code-bridge

---

Question for the Haystack community: Do you run your pipelines in restricted cloud environments today? How do you currently bridge the gap to local resources — and is async file-based delegation something you'd reach for?