test(integration): refresh stale integration tests after v0.20.0 changes#1757
Merged
Conversation
Bump pyproject.toml + uv.lock to 0.20.0 and roll the [Unreleased] CHANGELOG block into [0.20.0] - 2026-06-11. Lint mirror green locally (ruff check + format, pylint R0801, auth-signals). Post-merge: tag v0.20.0 to trigger the release workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The release integration suite (which gates releases, not per-PR runs) surfaced 29 failures after v0.20.0 was tagged. Each was a stale test that had not been updated to match an intentional source behavior change merged this cycle. Source is correct throughout; only tests change here. Clusters and causing PRs: - A (#1735): download_github_file now calls _host._resolve_dep_auth_ctx; mock host helpers never stubbed it, forcing the wrong auth path. Added the stub to make_host/_make_host in 4 download test files. Also #1735 made generic raw-URL network errors raise RuntimeError instead of falling back to the Contents API -- rewrote 2 stale fallback tests to assert the raise. - B (#1742): apm compile -t copilot suppresses empty AGENTS.md shells when .github/instructions/*.md exists. Flipped 3 assertions to expect no AGENTS.md and assert the instruction files are present. - C (#1739): marketplace fetches moved to a shared requests.Session (_HTTP_SESSION.get); updated 6 tests to mock that seam. - D (#1734): optional registry env inputs are omitted without an override; updated 2 placeholder tests to assert the var is absent. - E (#1734): same omit-optional change; assert var not in result. - F (#1720): tar symlink rejection wording changed; updated the regex. Verified locally: 738 passed, 16 skipped (token-gated e2e). The two runnable cluster-B e2e tests (mixed_deps, guardrailing) pass against the v0.20.0 build; ado_e2e is identical logic, validated in CI. Full lint mirror green. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the release-gated integration test suite to match intentional behavior changes that landed around v0.20.0 (download auth-context resolution, optional registry inputs omission, Copilot compile instruction dedup, marketplace HTTP session seam, and updated archive error wording). The PR also includes version/changelog/lockfile updates that look like release metadata changes.
Changes:
- Refreshes multiple integration tests to stub
_resolve_dep_auth_ctx()and to assert generic-host raw URL network failures raise (no fallback). - Updates Copilot-target compile E2E/integration assertions to expect suppression of empty
AGENTS.mdshells when instructions exist under.github/instructions/. - Switches marketplace tests to mock
apm_cli.marketplace.client._HTTP_SESSION.getand adjusts optional-env expectations for MCP registry metadata.
Show a summary per file
| File | Description |
|---|---|
| uv.lock | Bumps apm-cli version entry to 0.20.0. |
| pyproject.toml | Bumps project version to 0.20.0 (scope/intent should match PR description). |
| CHANGELOG.md | Adds 0.20.0 section and adjusts at least one entry formatting/reference. |
| tests/integration/test_wave7_policy_registry_coverage.py | Updates MCP optional-env expectations and marketplace mocking seam. |
| tests/integration/test_wave4_pure_logic_coverage.py | Updates tar symlink/hardlink rejection assertion regex. |
| tests/integration/test_mixed_deps.py | Updates Copilot compile assertions to expect no AGENTS.md when instructions are deployed. |
| tests/integration/test_guardrailing_hero_e2e.py | Updates guardrailing E2E flow to validate Copilot instructions via .github/instructions/ rather than AGENTS.md. |
| tests/integration/test_download_strategies_selection.py | Stubs _resolve_dep_auth_ctx() and updates raw URL failure expectations to raise. |
| tests/integration/test_download_strategies_phase3w5.py | Same as above for the phase3w5 suite variant. |
| tests/integration/test_download_copilot_phase3.py | Updates optional-env behavior assertions for Copilot adapter resolution. |
| tests/integration/test_download_copilot_end_to_end.py | Same as above for end-to-end variant. |
| tests/integration/test_ado_e2e.py | Updates Copilot compile expectations for ADO scenario to avoid AGENTS.md shells. |
Copilot's findings
- Files reviewed: 11/12 changed files
- Comments generated: 3
Comment on lines
5
to
8
| [project] | ||
| name = "apm-cli" | ||
| version = "0.19.0" | ||
| version = "0.20.0" | ||
| description = "MCP configuration tool" |
| @@ -124,7 +124,7 @@ class TestMixedDependencyCompile: | |||
| """Test compiling projects with mixed dependencies.""" | |||
|
|
|||
| def test_compile_with_mixed_deps_generates_agents_md(self, temp_project, apm_command): | |||
| @@ -156,7 +156,7 @@ class TestADOCompile: | |||
| ADO_TEST_REPO = "dev.azure.com/dmeppiel-org/market-js-app/_git/compliance-rules" | |||
|
|
|||
| def test_compile_generates_agents_md(self, tmp_path): | |||
danielmeppiel
added a commit
that referenced
this pull request
Jun 12, 2026
…r changes (#1758) * chore: release v0.20.0 Bump pyproject.toml + uv.lock to 0.20.0 and roll the [Unreleased] CHANGELOG block into [0.20.0] - 2026-06-11. Lint mirror green locally (ruff check + format, pylint R0801, auth-signals). Post-merge: tag v0.20.0 to trigger the release workflow. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(integration): refresh stale integration tests after v0.20.0 changes The release integration suite (which gates releases, not per-PR runs) surfaced 29 failures after v0.20.0 was tagged. Each was a stale test that had not been updated to match an intentional source behavior change merged this cycle. Source is correct throughout; only tests change here. Clusters and causing PRs: - A (#1735): download_github_file now calls _host._resolve_dep_auth_ctx; mock host helpers never stubbed it, forcing the wrong auth path. Added the stub to make_host/_make_host in 4 download test files. Also #1735 made generic raw-URL network errors raise RuntimeError instead of falling back to the Contents API -- rewrote 2 stale fallback tests to assert the raise. - B (#1742): apm compile -t copilot suppresses empty AGENTS.md shells when .github/instructions/*.md exists. Flipped 3 assertions to expect no AGENTS.md and assert the instruction files are present. - C (#1739): marketplace fetches moved to a shared requests.Session (_HTTP_SESSION.get); updated 6 tests to mock that seam. - D (#1734): optional registry env inputs are omitted without an override; updated 2 placeholder tests to assert the var is absent. - E (#1734): same omit-optional change; assert var not in result. - F (#1720): tar symlink rejection wording changed; updated the regex. Verified locally: 738 passed, 16 skipped (token-gated e2e). The two runnable cluster-B e2e tests (mixed_deps, guardrailing) pass against the v0.20.0 build; ado_e2e is identical logic, validated in CI. Full lint mirror green. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(release): refresh release-validation harness for v0.20.0 behavior changes The release-validation shell harness carries its own copies of behavior assertions that duplicate the integration suite. Two of them went stale this cycle from the same PRs that broke the integration tests (#1757): - GH-AW compat (#1720): `apm pack --archive` now emits .zip by default; the archive check grepped only `build/*.tar.gz`. Accept either extension, testing each glob independently (a single `ls a b` exits non-zero when either pattern is unmatched, even if the other matches). - Hero scenario 2 / AGENTS.md (#1742): copilot `apm compile` omits the empty AGENTS.md shell when installed instructions already live under `.github/instructions/`. The check insisted AGENTS.md exist; now accept AGENTS.md OR a populated `.github/instructions/`, mirroring the merged pytest fix in test_guardrailing_hero_e2e.py. Same fixes applied to the Windows .ps1 (AGENTS.md only; it has no archive check). Predicates validated locally against apm v0.20.0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: danielmeppiel <danielmeppiel@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
The release integration suite (which gates releases, not per-PR runs) surfaced 29 failures right after
v0.20.0was tagged. Every failure was a stale test that hadn't been updated to match an intentional source behavior change merged this cycle. Source is correct throughout — this PR only touches tests.Warning
The current
v0.20.0tag's release run is broken because of these test failures. After this merges, the release must be re-triggered (see below).Why these slipped through
Integration tests gate releases, not per-PR CI. So several PRs this cycle (#1735, #1734, #1739, #1720, #1742) landed deliberate behavior changes whose integration tests were left stale, and the breakage only appeared in the
v0.20.0release run.Clusters and causing PRs
download_github_filenow calls_host._resolve_dep_auth_ctx(dep_ref); mock host helpers never stubbed it, sofile_ctxbecame a truthy MagicMock and forced the wrong auth path. Added the stub tomake_host/_make_hostin 4 download test files. #1735 also made generic raw-URL network errors raiseRuntimeErrorinstead of silently falling back to the Contents API — rewrote 2 stale fallback tests to assert the raise.apm compile -t copilotsuppresses emptyAGENTS.mdshells when.github/instructions/*.mdexists. Flipped 3 assertions to expect noAGENTS.mdand assert the instruction files are present.requests.Session(_HTTP_SESSION.get); updated 6 tests to mock that seam instead ofrequests.get.Symlinks and hard links are not supported); updated the regex.Validation
test_mixed_deps, guardrailing) pass against the actualv0.20.0build;test_ado_e2eis identical logic, validated in CI.After merge
The
v0.20.0release run failed, so the tag needs to be re-triggered (re-point/re-push thev0.20.0tag at the newmain, or cut a fresh tag). Leaving that as an explicit operator step.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com