feat(raycicmd): add matrix expansion types and selector resolution

[andrew-anyscale]

Rayci will expand matrix into individual Buildkite steps, allowing for fine-grained depends_on targeting of specific matrix combinations.

Matrix Formats Supported

Simple array (anonymous dimension):
matrix: ["3.10", "3.11", "3.12"]

Named dimensions:
matrix:
setup:
python: ["3.10", "3.11"]
cuda: ["12.1.1", "12.8.1"]

Key Generation

Expanded keys follow deterministic format:
{base-key}-{dim1}{val1}-{dim2}{val2}

Examples:
ray-build + python=3.11, cuda=12.1.1 → ray-build-cuda1211-python311
ray-build + ["linux", "darwin"] → ray-build-linux, ray-build-darwin

Selector Syntax for depends_on

Simple reference (waits for ALL instances via meta-step):
depends_on: ray-build

Selector syntax (waits for matching instances only):
depends_on:
- key: ray-build
matrix:
cuda: "12.1.1" # matches all python versions with cuda 12.1.1

Multi-value selector:
depends_on:
- key: ray-build
matrix:
python: ["3.10", "3.11"] # matches specific python versions

Topic: matrix-expand
Signed-off-by: andrew andrew@anyscale.com

[andrew-anyscale]

Reviews in this chain:
└#378 feat(raycicmd): add matrix expansion types and selector resolution
　└#379 feat(raycicmd): integrate matrix expansion into pipeline conversion

[andrew-anyscale]

#	head	base	diff	date	summary
0	74665960	df64f501	diff	Jan 20 13:04 PM	2 files changed, 846 insertions(+)
1	853ec652	df64f501	diff	Jan 20 13:06 PM	1 file changed, 2 insertions(+), 2 deletions(-)
2	7450d3e3	df64f501	diff	Jan 20 13:31 PM	2 files changed, 28 insertions(+), 9 deletions(-)
3	b7b5455b	df64f501	diff	Jan 20 13:39 PM	1 file changed, 11 insertions(+), 5 deletions(-)
4	b488745a	df64f501	diff	Jan 20 13:56 PM	2 files changed, 68 insertions(+), 17 deletions(-)
5	bc2e07e6	df64f501	diff	Jan 26 11:22 AM	2 files changed, 153 insertions(+), 247 deletions(-)
6	570e6e75	df64f501	diff	Jan 26 11:32 AM	2 files changed, 8 insertions(+), 8 deletions(-)
7	268d356a	df64f501	diff	Jan 26 11:42 AM	1 file changed, 26 insertions(+), 2 deletions(-)
8	24b799ad	6165f028	diff	Feb 5 17:17 PM	2 files changed, 4 insertions(+), 21 deletions(-)
9	cbb9612b	6165f028	diff	Feb 5 17:26 PM	0 files changed
10	4ce2802a	6165f028	diff	Feb 5 17:29 PM	0 files changed

[gemini-code-assist]

Summary of Changes

Hello @andrew-anyscale, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant new feature to Rayci: the ability to expand matrix definitions into distinct Buildkite steps. This enhancement provides users with fine-grained control over their CI/CD pipelines, enabling them to define dependencies that target specific combinations within a matrix. The core logic, implemented in a new raycicmd package, handles parsing various matrix formats, generating all possible combinations, substituting matrix values into step properties, and resolving matrix-aware depends_on clauses. This change aims to make CI configurations more powerful and flexible for complex build and test scenarios.

Highlights

• Matrix Expansion Logic: Introduced core functionality to expand matrix definitions into individual Buildkite steps, enabling more granular control over CI/CD pipelines.
• Flexible Matrix Definitions: Supports various matrix input formats, including simple arrays and named dimensions, parsed via parseMatrixConfig.
• Dynamic Value Substitution: Implemented substituteMatrixValues to replace {{matrix.X}} placeholders within step configurations, allowing dynamic content based on matrix combinations.
• Enhanced Dependency Management: Extended depends_on functionality to allow targeting specific matrix combinations using matrixSelector, resolved by expandMatrixSelector.
• Utility Functions: Added helper functions for type conversion (anySliceToStringSlice) and deep copying of step maps and values (deepCopyStepMap, deepCopyAnyMap, deepCopyAnyValue, deepCopyAnySlice) to ensure safe data manipulation.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new feature for expanding matrix configurations in RayCI, allowing for more granular control over build steps. The implementation in raycicmd/matrix_expand.go is well-structured and covers parsing, expansion, key/tag generation, and dependency resolution. The accompanying tests in raycicmd/matrix_expand_test.go are comprehensive and cover a wide range of scenarios, including edge cases and error conditions.

My review includes a couple of suggestions to improve performance and code clarity in the matrix expansion and value substitution logic. Overall, this is a solid addition.

[gitar-bot]

⚠️ Bug: Mismatched ordering between instances and expandedKeys in selector expansion

Details

In the matrixSelector.expand method, the code assumes that instances (generated by cfg.expand()) and expandedKeys[sel.Key] have exactly the same ordering. However, expandedKeys is populated externally and there's no guarantee it follows the same iteration order.

The code at line 325-327 matches instances by index position:

for i, inst := range instances {
    if inst.matches(sel) {
        matches = append(matches, allExpanded[i])
    }
}

If expandedKeys["ray-build"] was built with a different ordering (e.g., using a different iteration over maps), the wrong keys would be matched.

Suggested fix: Instead of relying on index-based matching, regenerate the keys directly:

for _, inst := range instances {
    if inst.matches(sel) {
        matches = append(matches, inst.generateKey(sel.Key, cfg))
    }
}

This removes the dependency on external ordering and makes the code more robust.

---

Was this helpful? React with 👍 / 👎

[gitar-bot]

Code Review ⚠️ Changes requested 2 resolved / 4 findings

Well-structured matrix expansion implementation with comprehensive tests. Two previously identified issues remain: silent character dropping in key sanitization (minor) and a potential ordering mismatch bug in selector expansion (important).

⚠️ Bug: Mismatched ordering between instances and expandedKeys in selector expansion

📄 raycicmd/matrix_expand.go:319-327

In the matrixSelector.expand method, the code assumes that instances (generated by cfg.expand()) and expandedKeys[sel.Key] have exactly the same ordering. However, expandedKeys is populated externally and there's no guarantee it follows the same iteration order.

The code at line 325-327 matches instances by index position:

for i, inst := range instances {
    if inst.matches(sel) {
        matches = append(matches, allExpanded[i])
    }
}

If expandedKeys["ray-build"] was built with a different ordering (e.g., using a different iteration over maps), the wrong keys would be matched.

Suggested fix: Instead of relying on index-based matching, regenerate the keys directly:

for _, inst := range instances {
    if inst.matches(sel) {
        matches = append(matches, inst.generateKey(sel.Key, cfg))
    }
}

This removes the dependency on external ordering and makes the code more robust.

💡 Quality: sanitizeKeyPart silently drops characters without warning

📄 raycicmd/matrix_expand.go:159

The sanitizeKeyPart function silently removes characters like periods and dashes from matrix values to create valid Buildkite keys. While this is documented in the comment, it could lead to unexpected key collisions.

For example:

• "3.10" and "310" would produce the same sanitized key
• "cuda-12" and "cuda12" would produce the same sanitized key

Consider either:

1. Replacing invalid characters with a safe substitute (e.g., period -> underscore) instead of removing them
2. Adding logging/warning when characters are stripped
3. Documenting this behavior more prominently for users of the API

This could cause hard-to-debug issues when two matrix combinations produce identical keys.

✅ 2 resolved

✅ Edge Case: Empty string dimension key creates confusing tag output

📄 raycicmd/matrix_expand.go:174
In generateMatrixTags, when the dimension is an empty string (simple array matrix), the tag is just the value itself (e.g., "3.10"). However, this could cause issues if the value happens to conflict with other tag values from named dimensions.

Additionally, in generateMatrixTags, the iteration order over the map is non-deterministic (Go map iteration is random). While the results are sorted at the end, this is fine. However, consider documenting this behavior for clarity.

This is a minor concern since the tags are sorted at the end, but worth noting for documentation purposes.

✅ Edge Case: Index bounds check in expandMatrixSelector may silently skip

📄 raycicmd/matrix_expand.go:327
In expandMatrixSelector, there's a bounds check if i < len(allExpanded) that silently skips matrix instances if they don't have corresponding expanded keys. This could lead to subtle bugs if the expandedKeys map gets out of sync with the actual expanded instances.

for i, inst := range instances {
    if matchesMatrixSelector(inst, sel) {
        if i < len(allExpanded) {  // This silently skips on out-of-bounds
            matches = append(matches, allExpanded[i])
        }
    }
}

If the expandedKeys array has fewer elements than instances, matching instances beyond the array bounds will be silently ignored. Consider either:

1. Returning an error if there's a mismatch
2. Adding a debug log or warning
3. Using a defensive assertion at the start of the function

This could indicate a programming error in the caller and should probably fail loudly rather than silently.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply	Compact
gitar auto-apply:on	gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}