Skip to content

ci: add Auths cryptographic commit verification (warn-only)#25124

Draft
bordumb wants to merge 6 commits intoBerriAI:mainfrom
bordumb:auths/add-commit-verification
Draft

ci: add Auths cryptographic commit verification (warn-only)#25124
bordumb wants to merge 6 commits intoBerriAI:mainfrom
bordumb:auths/add-commit-verification

Conversation

@bordumb
Copy link
Copy Markdown

@bordumb bordumb commented Apr 4, 2026

Relevant issues

Motivated by the March 24, 2026 supply chain incident where compromised CI credentials were used to publish malicious LiteLLM versions (v1.82.7, v1.82.8) to PyPI. LiteLLM's own SECURITY.md classifies supply chain attacks as P0 (Critical).

Pre-Submission checklist

  • Includes cookbook/security/auths_attack_simulation.py simulation script
  • Isolated to new files only — no modifications to existing code or workflows
  • Non-blocking: fail-on-unsigned: false ensures existing CI is unaffected

Changes

What this adds

Auths provides Ed25519 signatures bound to KERI-based decentralized identifiers (DIDs), creating a cryptographic chain of custody from commit to published artifact.

  1. .github/workflows/auths-verify-commits.yml — GitHub Actions workflow that runs Auths commit signature verification on PRs and pushes to main. Configured in warn-only mode (fail-on-unsigned: false) so it reports unsigned commits without blocking merges.

  2. .auths/allowed_signers — Configuration file listing authorized maintainer public keys. Currently contains placeholder entries with instructions for maintainers to add their keys.

  3. cookbook/security/auths_attack_simulation.py — Demonstrates how the March 24 attack vector (unauthorized PyPI publish via compromised CI credentials) would be detected by Auths verification. Creates a simulated repo with signed and unsigned commits, showing that the unsigned "attacker" commit is flagged.

Why this matters

The March 24 incident succeeded because there was no cryptographic binding between the published PyPI package and a verified maintainer identity. The attacker used a stolen PYPI_PUBLISH token (exfiltrated via a compromised Trivy GitHub Action) to publish v1.82.7 and v1.82.8 directly to PyPI. The source code on GitHub was never modified.

Auths addresses this gap: even if registry credentials are stolen, an attacker cannot produce a valid Ed25519 signature for commits they did not author. The signing key is bound to the maintainer's device keychain and never leaves it.

This is complementary to existing security measures (OIDC Trusted Publishing, cosign for Docker images, CodeQL, OSSF Scorecard, zizmor) — it adds the missing layer of per-commit identity verification.

To activate enforcement

  1. Install the Auths CLI: brew tap auths-dev/auths-cli && brew install auths (or cargo install auths_cli)
  2. Create identity: auths init
  3. Export signers: auths signers sync --output .auths/allowed_signers
  4. Set fail-on-unsigned: true in the workflow

Deployment notes

  • Zero risk to existing CI: warn-only mode, no modifications to any existing files
  • The action auto-installs the Auths CLI; no manual setup required in CI

@bordumb bordumb requested a review from a team April 4, 2026 02:06
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 4, 2026

CLA assistant check
All committers have signed the CLA.

@vercel
Copy link
Copy Markdown

vercel bot commented Apr 4, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Apr 5, 2026 8:22am

Request Review

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq bot commented Apr 4, 2026

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing bordumb:auths/add-commit-verification (adaf293) with main (127149c)

Open in CodSpeed

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 4, 2026

Greptile Summary

This PR adds a warn-only Auths commit-signature verification layer to LiteLLM's CI, motivated by the March 24, 2026 supply-chain incident where a stolen PYPI_PUBLISH token was used to publish malicious package versions without touching the GitHub source. It introduces three new files with no modifications to any existing code or workflows.

Key changes:

  • .github/workflows/auths-verify-commits.yml — GitHub Actions workflow that verifies Ed25519 commit signatures on PRs and pushes to main; configured with fail-on-unsigned: false so it reports but never blocks. The Auths action is now pinned to a full commit SHA (57e304ef…), addressing the unpinned-action concern from the prior review round.
  • .auths/allowed_signers — Lists authorized maintainer public keys. Currently holds a single key belonging to the PR author; per prior discussion, at least two core LiteLLM maintainers should add their own keys before the file provides meaningful enforcement signal.
  • cookbook/security/auths_attack_simulation.py + README.md — Educational simulation demonstrating the cryptographic primitive. The README correctly notes that commit-signing alone would not have caught the real attack (registry-only publish) and that a full deployment would also require auths artifact sign in the release workflow to close the gap.

Open items:

  • The simulation script places optional-dependency imports inside main() rather than at module level, which is against the project's CLAUDE.md style guide (P2 only).
  • Enforcement is effectively inert until core maintainers populate .auths/allowed_signers with their own keys.

Confidence Score: 5/5

Safe to merge — warn-only, no modifications to existing files, all prior P0/P1 concerns resolved.

The two previously critical findings (unpinned third-party action, resource leak) have been fully addressed: the action is pinned to a full commit SHA and the file-handle issue no longer exists in the current version. The only remaining findings are P2 style suggestions (inline imports in a cookbook script). The workflow is strictly non-blocking and additive.

.auths/allowed_signers — needs at least one LiteLLM core maintainer key for the workflow to provide meaningful enforcement signal before fail-on-unsigned is ever flipped to true.

Important Files Changed

Filename Overview
.github/workflows/auths-verify-commits.yml Warn-only Auths commit verification workflow; action now pinned to full commit SHA (57e304ef…) after prior review. Permissions correctly scoped at job level. Non-blocking to existing CI.
.auths/allowed_signers Contains a single Ed25519 key belonging to the PR author (external contributor). Per prior discussion, at least two LiteLLM maintainers should add their keys before the file provides meaningful enforcement signal.
cookbook/security/auths_attack_simulation.py Educational simulation of the March 24 supply-chain attack. Logic is sound and handles missing-SDK cases gracefully. Minor style issue: optional-dependency imports placed inside main() rather than at module level, against project conventions.
cookbook/security/README.md Clear documentation accurately describing the March 24 attack vector, what Auths does and does not protect against, and how to run the simulation.

Sequence Diagram

sequenceDiagram
    participant Dev as Developer
    participant GH as GitHub Actions
    participant Auths as Auths CLI<br/>(auths-verify-github-action)
    participant Signers as .auths/allowed_signers
    participant PR as PR Comment

    Dev->>GH: Push commit / open PR
    GH->>GH: Checkout repo (fetch-depth: 0)
    GH->>Auths: Run auths-verify-github-action
    Auths->>Signers: Load authorized Ed25519 public keys
    loop Each non-merge commit
        Auths->>Auths: Verify Ed25519 signature
        alt Signed by key in allowed_signers
            Auths-->>GH: Verified
        else Unsigned or unknown signer
            Auths-->>GH: Unverified (warn-only, fail-on-unsigned=false)
        end
    end
    Auths->>PR: Post verification summary comment
    GH-->>Dev: Workflow complete (never blocks merge)
Loading

Reviews (3): Last reviewed commit: "fix: clarify that simulation shows commi..." | Re-trigger Greptile

#
# Documentation: https://github.com/auths-dev/auths/blob/main/docs/guides/platforms/ci-cd.md
#
# Maintainers — add your keys below this line:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Empty allowed_signers means zero cryptographic enforcement at merge time

The file contains no public keys. Until at least one maintainer adds their key, the Auths verification step has no authorized identities to validate commits against. Depending on the Auths CLI implementation, this will either:

  • Report every commit as unsigned/unverified (warn-only, so nothing is blocked), or
  • Vacuously pass verification (nothing to check against = no violations).

Either outcome means the "cryptographic chain of custody" described in the PR description doesn't exist the moment this merges. The workflow will run but provide no actual signal until keys are populated.

Consider making this a follow-up requirement: the workflow should only be merged once at least one maintainer key is present, otherwise it creates the impression of coverage without providing any.

run(["git", "config", "user.signingkey", key_path], cwd=repo)

# Create allowed_signers file
pub_key_content = open(f"{key_path}.pub").read().strip()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 File opened without a context manager — resource leak

open() used directly without with leaves the file handle open until garbage collection. Use a context manager:

Suggested change
pub_key_content = open(f"{key_path}.pub").read().strip()
with open(f"{key_path}.pub") as fh:
pub_key_content = fh.read().strip()

# Documentation: https://github.com/auths-dev/auths/blob/main/docs/guides/platforms/ci-cd.md
#
# Maintainers — add your keys below this line:
z6MkhPJCPXd5A9VN4wScJkxTtz6de7egZQx78vsiAT1vg3PZ@auths.local namespaces="git" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICuPK6OfYp7ngZp40Q+Dsrahhks472v6gPIMD0upCRnM
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 External contributor key is the sole authorized signer

The only entry in .auths/allowed_signers is a key whose DID (z6MkhPJCPXd5A9VN4wScJkxTtz6de7egZQx78vsiAT1vg3PZ) belongs to the PR author — an external contributor, not a LiteLLM core maintainer.

With this configuration, the Auths CI will:

  • Mark every commit from LiteLLM maintainers as unverified (flagged in the warn output)
  • Mark commits from this external contributor as verified

This inverts the trust model the file is meant to establish. While fail-on-unsigned: false keeps it non-blocking today, the verification output will consistently produce false-positive warnings on all legitimate maintainer commits, and will create the appearance that the external contributor's key is the canonical signing authority for the repo.

The key should belong to at least one LiteLLM core maintainer before this file provides meaningful signal. The original PR description's "To activate enforcement" section (step 3) describes the correct flow for maintainers to export their own keys.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is expected

This PR should not be merged unless maintainers are interested in adopting Auths for an added layer of security that is device-attested (i.e. attackers can steal your identity/credentials, but will ultimately fail when pushing from a device that isn't tied to maintainer's identity)

Next steps:
Before merging, at least 2 maintainers should try adding their Auths identity to .auths/approved_signers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants