feat(marketplace): add sourceBase for nested package sources (#1519)#1731
Draft
leocamello wants to merge 2 commits into
Draft
feat(marketplace): add sourceBase for nested package sources (#1519)#1731leocamello wants to merge 2 commits into
leocamello wants to merge 2 commits into
Conversation
…ft#1519) Marketplace blocks in apm.yml can now declare an optional `sourceBase` -- a git base (https://host/path, arbitrary depth) that host-less relative package `source` values compose onto. This enables deeply nested enterprise GitLab group paths that the `host.tld/owner/repo` shorthand cannot express. Semantics ("relative to base, always"): - A host-less `source` (`my-package`, `a/b/c`) composes onto the base. - Host-prefixed (`host.tld/owner/repo`) and full-URL sources override the base (absolute, base ignored); local `./` sources are untouched. - Without `sourceBase`, behavior is byte-for-byte unchanged, and a host-less relative source stays rejected (fail-closed -- no silent routing to the default host). Composition happens at parse time: a relative entry is rewritten to the equivalent host-prefixed form, so the resolver, builder, and output mappers treat it identically to a microsoft#1288 host-prefixed entry -- both the build-time `git ls-remote` and the emitted marketplace.json URL are correct with no changes to those modules. `sourceBase` is validated with microsoft#1288's security posture (https-only, FQDN host, no userinfo/port/query/fragment/.git) plus validate_path_segments on the path. `apm marketplace plugin add` accepts relative sources when a base is declared. Includes schema/authoring docs, a CHANGELOG entry, and unit tests covering validation, composition (1 and N segments), override precedence, base-less rejection, and the composed marketplace.json URL.
`apm marketplace check` built a single default-host RefResolver with no token and called list_remote_refs(entry.source) ignoring entry.host, so any package on a non-default host (self-managed GitLab, GHES, ADO, Bitbucket DC) -- including every relative source composed onto marketplace.sourceBase -- failed with `git ls-remote` exit 128 even when `apm pack` against the same apm.yml succeeded. Mirror the per-host resolver pattern that builder.py already uses: build one RefResolver per effective host, with the host's token resolved via AuthResolver. Local `./` packages now short-circuit before any network call. Default-host entries keep today's behaviour exactly. The per-host token logic is extracted into a shared `marketplace/auth_helpers.resolve_token_for_host` that both the builder and the check command use, so the two paths cannot drift again. Surfaced while testing microsoft#1519 (sourceBase always yields a non-default host); also fixes the pre-existing gap for microsoft#1288 host-prefixed sources.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds an optional
marketplace.sourceBasefield so marketplace authors onself-managed hosts (canonical case: GitLab nested subgroups) can declare a git
base once and write each package
sourcerelative to it. The per-entrysourceshorthand tops out at
host.tld/owner/repo(3 segments); enterprise GitLabnesting is routinely deeper (
.../group/sub-group/team/project/<pkg>), which theshorthand cannot express.
Semantics ("relative to base, always"): a host-less
sourcecomposes onto thebase; a
sourcecarrying its own host (host.tld/owner/repoor full URL) or alocal
./source overrides the base; with nosourceBasedeclared, behaviour isbyte-for-byte unchanged and a host-less single-segment source stays rejected
(fail-closed — no silent routing to the default host).
Implementation: composition happens at parse time (
yml_schema.py) — arelative entry is rewritten to the equivalent host-prefixed form, so the
resolver, builder, and output mappers treat it identically to a #1288
host-prefixed entry with no changes to those modules.
sourceBaseis validatedwith #1288's posture (https-only, FQDN host, no userinfo/port/query/fragment/
.git) plusvalidate_path_segments.apm marketplace plugin addacceptsrelative sources when a base is declared.
Also fixes
apm marketplace check: it was building a single default-hostresolver with no token and ignoring
entry.host, so it failed (git ls-remoteexit 128) for any non-default host — every
sourceBaseentry, and thepre-existing #1288 host-prefixed form. It now uses the same per-host resolver +
per-host token pattern as
apm pack(extracted into a sharedmarketplace/auth_helpers.resolve_token_for_host) and short-circuits localpackages, so
checkandpackagree.Validated end-to-end against a real self-managed GitLab (4-segment nested group):
apm marketplace checkexits 0, andapm packemits asource: urlentry withthe composed URL and a real resolved
sha.The consumer-side install gap for non-default-host marketplaces
(
apm install <pkg>@<marketplace>strips the host) is independent, reproduces onstock 0.18.0, and is tracked in #1010 — out of scope here.
Fixes #1519
Type of change
Testing
tests/unit/marketplace/test_source_base.py(validation, composition for 1 and Nsegments, override precedence, base-less rejection, composed marketplace.json
URL) and
tests/unit/commands/test_marketplace_check.py(per-host resolution:composed→GitLab+token, host-prefixed override→github, local→zero ls-remote).
uv run pytest tests/unit tests/test_console.pyis green except one pre-existing,environment-dependent
test_auth_scopingfailure unrelated to this change.ruff check/ruff format --checkclean.Spec conformance (OpenAPM v0.1)
(
src/apm_cli/marketplace/is not a Mode-B critical path andCONFORMANCE.{json,md}regenerate with no diff.)