feat(raycicmd): set RAYCI_ARTIFACTS_DIR for wanda steps

[andrew-anyscale]

Set RAYCI_ARTIFACTS_DIR environment variable for wanda steps based on the target platform. Uses /c/tmp/artifacts for Windows (detected via instance_type containing "windows") and /tmp/artifacts for other platforms.

This enables wanda to extract build artifacts to the correct location for Buildkite artifact upload.

Topic: raycicmd-artifacts
Relative: wanda-artifact-cli
Signed-off-by: andrew andrew@anyscale.com

[andrew-anyscale]

Reviews in this chain:
└#407 feat(wanda): add Artifact struct to spec for extraction
　└#411 feat(wanda): implement artifact extraction using docker cp
　　└#409 feat(wanda): add artifacts_dir CLI flag
　　　└#410 feat(raycicmd): set RAYCI_ARTIFACTS_DIR for wanda steps

[andrew-anyscale]

#	head	base	diff	date	summary
0	d09894c7	e45039da	diff	Feb 3 13:24 PM	5 files changed, 42 insertions(+)
1	47c8864b	3ba30cbe	diff	Feb 3 14:18 PM	4 files changed, 46 insertions(+), 21 deletions(-)
2	5583f7ce	b0cff492	rebase	Feb 3 14:24 PM	0 files changed
3	8ebf196c	2f92adb8	rebase	Feb 3 17:51 PM	0 files changed
4	2a465c21	81f41f5d	rebase	Feb 4 14:22 PM	0 files changed
5	af6ce95a	3920c222	rebase	Feb 4 14:27 PM	0 files changed
6	1a2e1f02	5f986153	diff	Feb 4 15:12 PM	0 files changed
7	54099d6f	e090ae8f	rebase	Feb 4 15:29 PM	0 files changed
8	c35f6d55	e090ae8f	diff	Feb 5 8:10 AM	1 file changed, 3 insertions(+), 1 deletion(-)
9	1feebe07	e090ae8f	diff	Feb 5 8:25 AM	2 files changed, 4 insertions(+), 4 deletions(-)
10	c8452f58	cefe4d92	diff	Feb 5 8:32 AM	2 files changed, 15 insertions(+), 7 deletions(-)
11	aaedbd8d	cefe4d92	diff	Feb 5 8:50 AM	0 files changed
12	83996e6b	bd707914	rebase	Feb 5 11:38 AM	0 files changed
13	34ea9f96	5c06796d	rebase	Feb 5 11:59 AM	0 files changed
14	92b01d31	5c06796d	diff	Feb 5 16:15 PM	2 files changed, 14 insertions(+), 6 deletions(-)
15	40bb72b5	5c06796d	diff	Feb 5 16:26 PM	1 file changed, 5 insertions(+), 3 deletions(-)

[gemini-code-assist]

Summary of Changes

Hello @andrew-anyscale, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the raycicmd tool by integrating an RAYCI_ARTIFACTS_DIR environment variable into Wanda steps. This variable provides an OS-dependent mount point for artifacts, streamlining the collection and uploading of build outputs like Python Wheels directly within the CI pipeline, thereby improving artifact management and consistency across different operating systems.

Highlights

• New Environment Variable for Artifacts: Introduced RAYCI_ARTIFACTS_DIR environment variable for Wanda steps, dynamically set based on the operating system (Windows vs. others).
• Enhanced Artifact Uploading: Enabled direct uploading of build artifacts, such as Python Wheels, within Wanda steps by providing a standardized mount point.
• New Artifact Test Case: Added a new artifact-test CI step, including a dedicated Dockerfile and Wanda configuration, to demonstrate and validate artifact handling.
• Core Logic Update: Updated raycicmd/converter.go to manage the RAYCI_ARTIFACTS_DIR environment variable based on the runtime OS.
• Test Coverage Expansion: Modified raycicmd/converter_test.go to reflect the inclusion of RAYCI_ARTIFACTS_DIR in various test scenarios, ensuring correct environment variable propagation.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• .buildkite/artifact-test.Dockerfile
  • Added a new Dockerfile to create dummy Python wheel files and a binary for artifact testing.
• .buildkite/artifact-test.wanda.yaml
  • Added a new Wanda configuration file to define how the dummy artifacts from the Dockerfile should be collected and stored, including an optional file.
• .buildkite/forge.rayci.yaml
  • A new artifact-test step was added to the main CI pipeline configuration, linking to the new Wanda artifact test.
• raycicmd/converter.go
  • Imported the runtime package.
  • Implemented logic to set the RAYCI_ARTIFACTS_DIR environment variable to /c/artifact-mount for Windows and /artifact-mount for other operating systems.
• raycicmd/converter_test.go
  • Updated all relevant test cases to assert the presence and correct value of the RAYCI_ARTIFACTS_DIR environment variable in the generated environment maps.

Activity

• The pull request is currently in a "draft" state.
• It has been signed off by andrew.
• No other review comments or activity have been recorded yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a RAYCI_ARTIFACTS_DIR environment variable for build steps, with its value dependent on the operating system. This is intended to allow artifacts like Python wheels to be uploaded from Wanda steps. While the feature is valuable, the current implementation has a critical flaw: the artifact directory path is determined by the OS of the raycicmd runner, not the target OS of the individual build step. This will lead to incorrect paths for cross-platform builds (e.g., generating Windows steps from a Linux runner). My review includes a suggestion to move this logic into the individual step converters to correctly determine the path based on the step's target environment. I've also pointed out necessary updates to test cases that will be required after this change.

[aslonnie]

use heredoc / multi-line for this, and add set -euo pipefail?

[andrew-anyscale]

Done

[aslonnie]

nit: why? can we revert this?

[andrew-anyscale]

This was a victim of go fmt on a previous iteration that updated this file more. Reverting