Pipeline Runner Implementation

[Synicix]

Summary:
Added:

• Pipeline Runner implementation under runner.rs
• Detail docs on the algorithm and logic behind running the pipeline
• Modify Pipeline Job to fit with some design changes to the runner
• Basic test to check if the pipeline runner excute a toy pipeline correctly
• Stop test to check if the pipeline runner can abort the execution correclty

Missing:

• Test for joiner node (I will get to it later tonight)

Features Deferred:

• More apporiate error handling rather than just printing it out right now. Possable solution to this is to have an log file using tokio tracing log system to write errors to a single file and store it within pipeline_results for debugging purposes
• Running the pod. Due to the fact that Minimal orchestrator agent w/o reservations #91 and Agent: Add configurable storage of PodResult #94 adds an agent that will handle the actual execution of the pod, I currently have it "simulate" excution by hard coding it to the test case.

[codecov]

Codecov Report

Attention: Patch coverage is 87.88571% with 106 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/uniffi/pipeline.rs	72.52%	61 Missing ⚠️
src/uniffi/pipeline_runner/runner.rs	94.74%	32 Missing ⚠️
src/core/error.rs	47.36%	10 Missing ⚠️
src/uniffi/pipeline_runner/mod.rs	81.25%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[Copilot]

Pull Request Overview

This PR adds a Docker-based pipeline runner, expands test coverage for pipelines and their execution, and updates related utility and error‐handling components.

• Implements DockerPipelineRunner with async task orchestration (start, get_result, stop).
• Adds fixtures and tests for pipeline creation and runner execution (basic_run, stop).
• Updates helper utilities (get), error variants (channel/send errors), and orchestrator JSON parsing.

Reviewed Changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/pipeline_runner.rs	Adds async tests for running and stopping pipelines.
src/uniffi/pipeline_runner/runner.rs	Implements core runner logic for node execution and messaging.
src/uniffi/pipeline_runner/mod.rs	Defines PipelineRunner trait and PipelineRun key type.
src/uniffi/orchestrator/docker.rs	Adjusts JSON label lookup for pod jobs.
src/core/util.rs	Generalizes get to support non-String keys.
src/core/error.rs	Introduces new error kinds for channel and sending errors.

Comments suppressed due to low confidence (2)

tests/pipeline_runner.rs:31

• The test calls get_result but doesn't assert anything about the returned PipelineResult. Add assertions to verify that the output packets match the expected results of the toy pipeline.

    runner.get_result(&pipeline_run).await?;

src/uniffi/pipeline_runner/mod.rs:14

• The PipelineRunner trait signature (start(&self) -> Result<()>) does not match DockerPipelineRunner's async signature and return type (start(&mut self, ...) -> Result<PipelineRun>). Update the trait to align with implementations or implement the trait on the runner with the correct signatures.

    fn start(&self, pipeline_job: PipelineJob) -> Result<()>;

[Synicix]

Notes from Meeting:

• Switch to Zenoh for communications
• Export unffi constructor for Pipeline
• Combine pipeline run with PipelineRunInfo

[Synicix]

Update on this PR. I fix a lot of issues since a lot of my code depending on the previous implementation of pipeline. Just merged and fix 90% of conflicts that I had when switching to the new design that was merged on Monday.

Still need to fix output recording logic to deal with the output_spec and add the correct zenoh communication with the agent.

[Synicix]

Close due to staleness and too many conflicts with recents merges