fix: support wildcard file paths in project includes - #1914
Conversation
The glob expanded to several files but cp targeted a literal destination path, and the later load used that same literal path instead of the matches.
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="tests/test-cases/include-project-file-wildcard/integration.test.ts">
<violation number="1" location="tests/test-cases/include-project-file-wildcard/integration.test.ts:20">
P2: This test (and its .gitlab-ci.yml fixture) depend on a live external GitLab project `components/go` and its `templates/build.yml`/`templates/test.yml` content via a real network fetch, so it will fail when offline or whenever that external project changes. Consider hosting the included project fixtures locally (e.g. a git project-include that resolves to checkout-able content) so the test is hermetic and asserts the wildcard behavior against files the repo controls.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| stateDir: ".gitlab-ci-local-include-project-file-wildcard", | ||
| }, writeStreams); | ||
|
|
||
| const jobNames = writeStreams.stdoutLines |
There was a problem hiding this comment.
P2: This test (and its .gitlab-ci.yml fixture) depend on a live external GitLab project components/go and its templates/build.yml/templates/test.yml content via a real network fetch, so it will fail when offline or whenever that external project changes. Consider hosting the included project fixtures locally (e.g. a git project-include that resolves to checkout-able content) so the test is hermetic and asserts the wildcard behavior against files the repo controls.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/test-cases/include-project-file-wildcard/integration.test.ts, line 20:
<comment>This test (and its .gitlab-ci.yml fixture) depend on a live external GitLab project `components/go` and its `templates/build.yml`/`templates/test.yml` content via a real network fetch, so it will fail when offline or whenever that external project changes. Consider hosting the included project fixtures locally (e.g. a git project-include that resolves to checkout-able content) so the test is hermetic and asserts the wildcard behavior against files the repo controls.</comment>
<file context>
@@ -0,0 +1,26 @@
+ stateDir: ".gitlab-ci-local-include-project-file-wildcard",
+ }, writeStreams);
+
+ const jobNames = writeStreams.stdoutLines
+ .filter(l => /^(build|test)\s/.test(l))
+ .map(l => l.split(/\s+/)[0])
</file context>
There was a problem hiding this comment.
1 issue found across 1 file (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/parser-includes.ts">
<violation number="1" location="src/parser-includes.ts:153">
P3: Switching the match sort to `String#localeCompare` makes the order of included project files depend on the host runtime's locale (via the Intl collator), whereas the previous default `.sort()` was locale-independent and deterministic. Because these matches are concatenated into `includeDatas` and pipeline config can be merged/overridden in that order, a developer machine with a different locale could resolve a slightly different pipeline than CI for the same inputs. If the goal was stable ordering, keep the locale-invariant default `.sort()` (or pass an explicit `{numeric: true, sensitivity: 'base'}` if case-insensitivity is intended); otherwise project includes should order consistently regardless of locale.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| includeDatas = includeDatas.concat(await this.init(fileDoc, opts)); | ||
| const includeDir = `${cwd}/${stateDir}/includes/${gitData.remote.host}/${value["project"]}/${value["ref"] || "HEAD"}`; | ||
| const normalizedFile = fileValue.replace(/^\/+/, ""); | ||
| const matches = globbySync(normalizedFile, {cwd: includeDir, absolute: true}).sort((a, b) => a.localeCompare(b)); |
There was a problem hiding this comment.
P3: Switching the match sort to String#localeCompare makes the order of included project files depend on the host runtime's locale (via the Intl collator), whereas the previous default .sort() was locale-independent and deterministic. Because these matches are concatenated into includeDatas and pipeline config can be merged/overridden in that order, a developer machine with a different locale could resolve a slightly different pipeline than CI for the same inputs. If the goal was stable ordering, keep the locale-invariant default .sort() (or pass an explicit {numeric: true, sensitivity: 'base'} if case-insensitivity is intended); otherwise project includes should order consistently regardless of locale.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/parser-includes.ts, line 153:
<comment>Switching the match sort to `String#localeCompare` makes the order of included project files depend on the host runtime's locale (via the Intl collator), whereas the previous default `.sort()` was locale-independent and deterministic. Because these matches are concatenated into `includeDatas` and pipeline config can be merged/overridden in that order, a developer machine with a different locale could resolve a slightly different pipeline than CI for the same inputs. If the goal was stable ordering, keep the locale-invariant default `.sort()` (or pass an explicit `{numeric: true, sensitivity: 'base'}` if case-insensitivity is intended); otherwise project includes should order consistently regardless of locale.</comment>
<file context>
@@ -150,7 +150,7 @@ export class ParserIncludes {
const includeDir = `${cwd}/${stateDir}/includes/${gitData.remote.host}/${value["project"]}/${value["ref"] || "HEAD"}`;
const normalizedFile = fileValue.replace(/^\/+/, "");
- const matches = globbySync(normalizedFile, {cwd: includeDir, absolute: true}).sort();
+ const matches = globbySync(normalizedFile, {cwd: includeDir, absolute: true}).sort((a, b) => a.localeCompare(b));
const filePaths = matches.length > 0 ? matches : [`${includeDir}/${normalizedFile}`];
for (const filePath of filePaths) {
</file context>
| const matches = globbySync(normalizedFile, {cwd: includeDir, absolute: true}).sort((a, b) => a.localeCompare(b)); | |
| const matches = globbySync(normalizedFile, {cwd: includeDir, absolute: true}).sort(); |
1 participant
A wildcard in
include:project:fileexpanded to several files, butcptargeted a literal destination path so the fetch failed withis not a directory, and the subsequent load used that same literal path instead of the matches. Fixes #1642.Summary by cubic
Support wildcard file paths in project includes. Expands globs and copies matched files into the destination directory with deterministic order to avoid “is not a directory” errors.
include:project:fileglobs withglobbySync, sort matches, and load each file.cp <src> <dir>/; tests fortemplates/[bt]*.ymlverifybuildandtestjobs are included.Written for commit 4ace621. Summary will update on new commits.