[fray] CI skips fray tests when only iris changes, hiding cross-layer breakage

[RohithKuditipudi]

Description

.github/workflows/fray-unit-tests.yaml uses a paths-filter that only runs fray tests when files under lib/fray/**, uv.lock, or the workflow itself change. Because fray depends on iris (lib/fray → lib/iris per AGENTS.md), changes to iris can break fray tests without CI noticing — on main or on the iris-only PR that caused the break.

Concrete instance: iris commit cf05d94ee (2026-04-12, "Enforce constraint/attribute case-normalization via AttributeValue") changed Constraint to store values: tuple[AttributeValue, ...] with no .value attribute. Five assertions in lib/fray/tests/test_v2_iris.py (TestConvertConstraints, TestConvertConstraintsDeviceAlternatives) started failing against the new API but went unnoticed on main for two days. The break only surfaced on PR #4725, which touched fray.

Definition of Done

• Fray tests run in CI on any change that can affect them (at minimum: lib/iris/** in addition to the existing paths), OR
• A justification is documented for why cross-layer breakage is acceptable to discover only on the next fray-touching PR.
• Same review applied to other per-subproject workflows with paths-filter that skip on upstream-layer changes (zephyr, marin, levanter) if the same pattern exists.

[rjpower]

@claude make a PR for this

[claude]

Claude finished @rjpower's task in 3m 26s —— View job

---

Blocked: .github/workflows requires workflows permission

I prepared and committed the fix locally, but the push was rejected — the Claude GitHub App on this repo does not have the workflows permission, which is required to create or update files under .github/workflows/. This is a documented limitation of this integration (FAQ). The changes exist on the runner but cannot be pushed to claude/issue-4728-20260422-2212, so I cannot open a PR from my side.

Remote error:

! [remote rejected]     claude/issue-4728-20260422-2212 -> claude/issue-4728-20260422-2212
  (refusing to allow a GitHub App to create or update workflow
   .github/workflows/fray-unit-tests.yaml without `workflows` permission)

Below is the full patch so you (or a maintainer with workflow-write) can apply it directly. ./infra/pre-commit.py --all-files --fix passed cleanly on it. No Python test changes, so no pytest run was warranted.

---

Reasoning

Actual import-level dependencies (verified via grep ^from/^import across the tree):

Subproject	Imports from
fray	iris
zephyr	iris, fray
levanter	haliax, fray, zephyr (transitively iris)
marin	haliax, iris, fray, zephyr, levanter

Current path filters vs. what each was missing:

Workflow	Added
fray-unit-tests.yaml	lib/iris/**
zephyr-unit-tests.yaml	lib/fray/** (iris was already present)
levanter-tests.yaml	lib/iris/**, lib/fray/**, lib/zephyr/**
marin-unit-tests.yaml	lib/haliax/**, lib/iris/**, lib/fray/**, lib/zephyr/**, lib/levanter/**

Patch

diff --git a/.github/workflows/fray-unit-tests.yaml b/.github/workflows/fray-unit-tests.yaml
--- a/.github/workflows/fray-unit-tests.yaml
+++ b/.github/workflows/fray-unit-tests.yaml
@@ -25,6 +25,9 @@ jobs:
           filters: |
             relevant:
               - 'lib/fray/**'
+              # fray imports from iris (fray.v2.iris_backend, tests/test_v2_iris.py);
+              # run fray tests on any iris change so cross-layer breakage is caught here.
+              - 'lib/iris/**'
               - 'uv.lock'
               - '.github/workflows/fray-unit-tests.yaml'

diff --git a/.github/workflows/zephyr-unit-tests.yaml b/.github/workflows/zephyr-unit-tests.yaml
--- a/.github/workflows/zephyr-unit-tests.yaml
+++ b/.github/workflows/zephyr-unit-tests.yaml
@@ -25,8 +25,10 @@ jobs:
           filters: |
             relevant:
               - 'lib/zephyr/**'
-              # Iris is being actively developed, to avoid bugs sneaking into zephyr, we trigger zephyr tests on iris change
+              # zephyr depends on upstream layers (imports from iris and fray); run
+              # zephyr tests on any upstream change so cross-layer breakage surfaces here.
               - 'lib/iris/**'
+              - 'lib/fray/**'
               - 'uv.lock'
               - '.github/workflows/zephyr-unit-tests.yaml'

diff --git a/.github/workflows/levanter-tests.yaml b/.github/workflows/levanter-tests.yaml
--- a/.github/workflows/levanter-tests.yaml
+++ b/.github/workflows/levanter-tests.yaml
@@ -27,6 +27,12 @@ jobs:
             relevant:
               - 'lib/levanter/**'
               - 'lib/haliax/**'
+              # levanter depends on upstream layers (imports from haliax, fray, and
+              # zephyr; transitively iris). Run levanter tests on any upstream change
+              # so cross-layer breakage surfaces here.
+              - 'lib/iris/**'
+              - 'lib/fray/**'
+              - 'lib/zephyr/**'
               - 'uv.lock'
               - '.github/workflows/levanter-tests.yaml'

diff --git a/.github/workflows/marin-unit-tests.yaml b/.github/workflows/marin-unit-tests.yaml
--- a/.github/workflows/marin-unit-tests.yaml
+++ b/.github/workflows/marin-unit-tests.yaml
@@ -28,6 +28,14 @@ jobs:
               - 'lib/marin/**'
               - 'experiments/grug/**'
               - 'tests/**'
+              # marin sits at the top of the dependency stack and imports from every
+              # upstream layer (haliax, iris, fray, zephyr, levanter). Run marin tests
+              # on any upstream change so cross-layer breakage surfaces here.
+              - 'lib/haliax/**'
+              - 'lib/iris/**'
+              - 'lib/fray/**'
+              - 'lib/zephyr/**'
+              - 'lib/levanter/**'
               - 'uv.lock'
               - 'pyproject.toml'
               - '.github/workflows/marin-unit-tests.yaml'

Notes / open questions for review

• CI time vs. coverage tradeoff. Expanding the marin filter to include every upstream layer means any lib/iris/**, lib/haliax/**, lib/fray/**, lib/zephyr/**, or lib/levanter/** change will now trigger the marin unit-test job (~15 min). This is the safest default and matches the issue's stated preference, but if that cost is too high, an alternative is to narrow to documented "public surface" directories per subproject — at the cost of re-introducing the same blind spot whenever a public API moves file.
• pyproject.toml / uv.lock. All four workflows already watch uv.lock; marin also watches pyproject.toml. Consider adding per-subproject pyproject.toml paths too if internal API version bumps matter, but that's orthogonal and not done here.
• experiments/**. Not touched. Happy to audit if desired.

Checklist

• Read relevant workflow files and confirm current filters
• Survey zephyr, marin, levanter for same pattern
• Edit the four workflow files to cover upstream deps
• Run ./infra/pre-commit.py --all-files --fix (passed)
• Commit locally
• Push — blocked by missing workflows permission on the Claude GitHub App

How this was verified

• Static: confirmed actual import edges with grep -rhE "^from (iris|fray|zephyr|levanter|haliax)" lib/{fray,zephyr,levanter,marin}/{src,tests}.
• Lint: ./infra/pre-commit.py --all-files --fix → OK (yaml format, trailing whitespace, all checks green).
• No Python source changed, so no pytest run.
  · Branch: claude/issue-4728-20260422-2212 (local commit 61093a147, not pushed)