E2E tests should exercise reusable workflow changes from the PR branch

[ralphbean]

What happens

Changes to the reusable workflow file (.github/workflows/fullsend.yaml) are not covered by e2e tests. The e2e suite does not trigger when that file is modified, and when it does run, the deployed test environment references the workflow from main rather than from the PR branch. This means workflow changes (like the OpenShell version pin in #882) merge untested.

What should happen

Two things need to change:

1. Trigger: The e2e test workflow's path filter should include the reusable workflow file so that modifications to it cause the e2e suite to run.
2. Ref override: The e2e test deployment should be configured to consume the reusable workflow from the PR's head ref instead of main, so the version under test is the one actually being changed.

How to verify

Once both pieces are in place, a smoke test would confirm coverage: open a PR that sets OPENSHELL_VERSION to a known-bad value (e.g., v0.0.0-bogus) and confirm the e2e suite fails. If it passes, the ref override isn't wired correctly.

Context

Raised during review of #882 (Renovate automation for OpenShell version pin). Could be addressed in a follow-up PR or folded into #882. See also #866 (e2e concurrency).

[github-actions]

fullsend triage is working on this — view logs

[fullsend-ai-triage]

Triage Summary

Severity: Medium · Category: Enhancement

Problem

The reusable workflow file (.github/workflows/fullsend.yaml) is not included in the e2e test suite's path triggers, and the e2e deployment always references the workflow from main rather than the PR branch. This means workflow changes merge without e2e coverage.

Root Cause

Two gaps in the CI configuration:

1. The e2e workflow's paths filter omits .github/workflows/fullsend.yaml.
2. The e2e test deployment hardcodes the workflow ref to main instead of using the PR's head ref.

Recommended Fix

1. Add .github/workflows/fullsend.yaml to the e2e workflow's paths trigger.
2. Configure the e2e deployment to consume the reusable workflow from the PR head ref.

Note: #854 follows a similar pattern (adding e2e path triggers for scaffold agent files) and may be a useful reference.

Verification

Open a PR that sets OPENSHELL_VERSION to a known-bad value
(e.g., v0.0.0-bogus) in the reusable workflow.
Verify the e2e suite triggers and fails.
If it passes or does not trigger, the fix is incomplete.

Related Issues

• feat: add Renovate automation for OpenShell version pin #882 — Renovate automation for OpenShell version pin (open, provides context)
• E2E tests cancelled when another PR triggers a concurrent run #866 — E2E concurrency issue (open, tangentially related)
• ci(e2e): trigger e2e tests on changes to internal scaffold agent files #854 — Similar path-filter enhancement for scaffold agent files

[ifireball]

/fs-prioritize

[fullsend-ai-prioritize]

RICE Priority Score: 1.8

Score breakdown

Dimension	Score	Reasoning
Reach	1	This affects all developers who contribute workflow changes to the repo. It's an internal CI/CD improvement rather than user-facing, but workflow changes like the OpenShell version pin (#882) merge untested, impacting the whole team's confidence in CI.
Impact	1	Untested workflow changes can introduce CI breakage that affects all developers. The impact is medium — it's a meaningful improvement to the development workflow and testing coverage, but manual verification is a possible (if unreliable) workaround.
Confidence	0.9	Very high confidence. The issue is exceptionally well-described with clear scope, specific files to change, verification steps, and related issues (#882, #866, #854) providing additional context. The triage summary confirms the root cause and fix.
Effort	0.5	Simple and well-scoped. Two small config changes: adding a path to the e2e workflow's trigger filter, and overriding the workflow ref to use the PR head. Similar work was done in #854 for scaffold agent files, providing a pattern to follow.

Formula: (1 x 1 x 0.9) / 0.5 = 1.8

[fullsend-ai-triage]

Meeting update — 2026-06-03

Relevant to this issue: The team identified action items to ensure e2e tests properly exercise workflow changes from the PR branch, directly relevant as ADR 0041 sync dispatch (PR #1611) changes the reusable workflow.

• Action item to ensure end-to-end tests pass for the ADR 0041 implementation
• Action item to create a PR to fix workflow issues in e2e tests if the current PR does not resolve them
• Action item to stabilize e2e tests to ensure they properly test changes on the PR branch

Related PRs: PR #1611 (ADR 41 sync dispatch) is the change that needs e2e coverage

Next steps: Verify e2e tests exercise the sync dispatch workflow changes; file a fix PR if gaps remain

Meeting notes

[fullsend-ai-triage]

Meeting update — 2026-06-10

Relevant to this issue: The team adopted using a "full send AI ref" placeholder in the scaffold to facilitate E2E testing with commit references — directly addressing the gap where E2E tests don't exercise reusable workflow changes from the PR branch.

• Decision: use a placeholder reference in the scaffold that can be swapped to the PR commit ref during E2E runs
• This enables E2E tests to exercise workflow changes from the PR branch rather than always referencing main

Next steps:

• Review the GitHub setup changes for the recent PR to ensure consistency with the new reference implementation
• Review the previous objection regarding pinning of the fullsend reference

Meeting notes

[rh-hemartin]

New evidence from PR #2412: PR #2412 changed .github/workflows/reusable-retro.yml (removing a dead env var) and e2e tests were skipped because none of the reusable workflow files are in the e2e path filters.

Two synced locations in e2e.yml need updating:

• Push trigger paths (lines 13-25): add '.github/workflows/reusable-*.yml'
• PR grep regex (line 98): add |^\.github/workflows/reusable-.*\.yml$

This covers all reusable workflows (reusable-retro.yml, reusable-dispatch.yml, reusable-code.yml, etc.) since they all orchestrate Go binary execution that e2e tests exercise.

See also #2087 re: deduplicating these two filter locations.