Skip to content

[INFRA] Automated Release Workflows (Change Detection + PyPI Push) #40

@scottchronicity

Description

@scottchronicity

Context

With independent versioned components and PyPI-ready packages, we need a GitHub Actions workflow that automates the release pipeline end-to-end: detect which components changed in a PR, run their specific test suites, and — upon a version tag — build and push only the affected components to PyPI. No manual release scripts, no full-repo rebuilds when one agent changes.

Dependencies

Requires: [INFRA] Independent Component Versioning System
Requires: [INFRA] PyPI Publishing Pipeline (Common + Agents)

Architecturally Significant Requirements (ASRs)

Interface (Contract):

  • Workflow file: .github/workflows/release.yml triggered on push to main and on version tags (v*.*.* or <component>/v*.*.*).
  • Change detection: scripts/changed-components.sh reads git diff --name-only and outputs the set of component paths that contain modified files.
  • Tag convention: orpheus-common/v1.2.0 releases orpheus-common; orpheus-agent-bird-detection/v0.5.0 releases that agent independently.
  • Matrix build: GitHub Actions matrix strategy over changed-components.sh output to parallelize per-component test and build jobs.

Implementation (Internal Logic):

  • changed-components.sh: maps changed file paths to component roots (e.g., platform/orpheus-common/src/...platform/orpheus-common).
  • PR workflow: for each changed component, run make test && make lint && make coverage as a matrix job.
  • Release workflow (tag push): for each tagged component, run python -m build and twine upload to PyPI using OIDC trusted publishing.
  • Dependency ordering: if a tag releases orpheus-common, trigger downstream agent builds in dependency order.

Architectural Constraints

  • Must use GitHub Actions OIDC trusted publishing to PyPI (no long-lived API tokens stored as secrets).
  • Change detection must correctly identify transitive dependencies — if orpheus-common changes, all agents' test suites run.
  • Release jobs must be idempotent — re-running a failed release job must not double-publish.
  • Must not run full-repo test suites on every PR if only one component changed (cost constraint: only run what changed).

Acceptance Criteria

Feature: Automated Release Workflows

  Scenario: PR affecting one agent only runs that agent's tests
    Given a PR modifies files only in agents/orpheus-agent-bird-detection/
    When the CI pipeline runs
    Then only the orpheus-agent-bird-detection test suite executes
    And orpheus-common and other agents are not re-tested

  Scenario: PR affecting orpheus-common runs all downstream tests
    Given a PR modifies files in platform/orpheus-common/
    When the CI pipeline runs
    Then orpheus-common tests execute first
    And all 11 agent test suites execute as downstream dependents

  Scenario: Version tag triggers PyPI publish for that component
    Given a tag "orpheus-agent-bird-detection/v0.5.0" is pushed to main
    When the release workflow runs
    Then only orpheus-agent-bird-detection is built and published to PyPI
    And other components are not touched

  Scenario: OIDC trusted publishing used for PyPI upload
    Given the release workflow is triggered by a version tag
    When the PyPI upload step runs
    Then no API token secret is used — OIDC authentication is used
    And the upload is auditable via PyPI's trusted publisher log

Definition of Done

  • .github/workflows/ci.yml updated with matrix-based per-component test jobs driven by changed-components.sh.
  • .github/workflows/release.yml created: detects tagged component, builds wheel, publishes to PyPI via OIDC.
  • scripts/changed-components.sh script: maps git diff output to component paths with correct transitive expansion for orpheus-common changes.
  • OIDC trusted publishing configured in PyPI project settings for each package.
  • Runbook in docs/RELEASING.md documenting the end-to-end release process and how to cut a hotfix.

Architectural Dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    C4: ContainerDeployable units (Agents, Broker, DB, React UI)component: devopsDeveloper tooling and workflow automationcomponent: infraBuild system, CI/CD, deployment toolingtype: infrastructureMakefiles, CI/CD, Docker, K8s

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions