feat(wanda): implement artifact extraction using docker cp

[andrew-anyscale]

Extract artifacts from built images using docker cp for cross-platform reliability

• Create container from image without starting it
• List container files via docker export for glob matching
• Copy each artifact using docker cp
• Clean up container when done
• Extraction runs for root spec only, even on cache hit

Example:

❯  PYTHON_VERSION=3.13 MANYLINUX_VERSION=260128.221a193 HOSTTYPE=aarch64 ARCH_SUFFIX=-aarch64 BUILDKITE_COMMIT=b5737cefc0 IS_LOCAL_BUILD=true /Users/andrew/devel/rayci-wanda-artifacts/_release/wanda-darwin-arm64 --artifacts_dir .whl/ ci/docker/ray-wheel.wanda.yaml

...

2026/02/04 09:10:39 extracting 1 artifact(s) from localhost:5000/rayci-work:ray-wheel-py3.13-aarch64
Successfully copied 72MB to /Users/andrew/devel/ray-local-wheel-build/.whl/ray-3.0.0.dev0-cp313-cp313-manylinux2014_aarch64.whl
2026/02/04 09:10:40 extracted 1 artifact(s) in 722ms:
2026/02/04 09:10:40   /Users/andrew/devel/ray-local-wheel-build/.whl/ray-3.0.0.dev0-cp313-cp313-manylinux2014_aarch64.whl

Topic: wanda-artifact-copy
Relative: wanda-artifact-spec

Signed-off-by: andrew andrew@anyscale.com

[andrew-anyscale]

Reviews in this chain:
└#407 feat(wanda): add Artifact struct to spec for extraction
　└#411 feat(wanda): implement artifact extraction using docker cp
　　└#409 feat(wanda): add artifacts_dir CLI flag
　　　└#410 feat(raycicmd): set RAYCI_ARTIFACTS_DIR for wanda steps

[andrew-anyscale]

#	head	base	diff	date	summary
0	a3b19fc3	70e0db32	diff	Feb 3 14:18 PM	11 files changed, 349 insertions(+), 2 deletions(-)
1	80ab6e0b	25c63e1c	rebase	Feb 3 14:24 PM	0 files changed
2	81385ecc	25c63e1c	diff	Feb 3 17:51 PM	2 files changed, 6 insertions(+), 6 deletions(-)
3	1e3c63b9	f1104ff7	diff	Feb 4 14:22 PM	8 files changed, 253 insertions(+), 30 deletions(-)
4	217cc4d4	313a1f83	diff	Feb 4 14:27 PM	0 files changed
5	34ea72c0	7a60b1e6	diff	Feb 4 15:12 PM	0 files changed
6	868f0c18	7a60b1e6	diff	Feb 4 15:29 PM	2 files changed, 6 deletions(-)
7	f210797c	7a60b1e6	diff	Feb 5 8:32 AM	1 file changed, 7 insertions(+)
8	f343d80f	928b35c4	diff	Feb 5 11:38 AM	1 file changed, 7 insertions(+), 3 deletions(-)
9	44d0999e	928b35c4	diff	Feb 5 11:59 AM	1 file changed, 8 insertions(+)

[gemini-code-assist]

Summary of Changes

Hello @andrew-anyscale, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the wanda build system by enabling the extraction of specified artifacts directly from built Docker images. This new capability, implemented using docker cp, provides a robust and cross-platform compatible way to retrieve files or directories from containers. It ensures that artifacts are extracted efficiently and reliably, with options for handling missing optional artifacts and clear logging for performance insights.

Highlights

• Artifact Extraction Implementation: Introduced functionality to extract artifacts from built Docker images using docker cp.
• Cross-Platform Reliability: The chosen docker cp method ensures reliable artifact extraction across different operating systems, including Windows.
• Controlled Container Lifecycle: Artifacts are extracted by creating a temporary container from the image, copying the specified files, and then removing the container, all without starting the container.
• Root Spec Only Extraction: Artifact extraction is performed exclusively for the root specification, even when a cache hit occurs for the image build.
• Optional Artifact Handling: Support for optional artifacts has been added, allowing the system to log a warning instead of failing if an optional artifact is not found.
• Performance Monitoring: The duration of the artifact extraction process is now logged to facilitate performance monitoring.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• wanda/docker_cmd.go
  • Added createContainer function to create a Docker container without starting it.
  • Added copyFromContainer function to copy files/directories from a container to the host.
  • Added removeContainer function to remove a Docker container.
• wanda/docker_cmd_test.go
  • Imported the os package.
  • Added TestDockerCmdCopyFromContainer to verify copying a single file.
  • Added TestDockerCmdCopyFromContainer_directory to verify copying a directory.
  • Added TestDockerCmdCopyFromContainer_notFound to test error handling for non-existent files.
• wanda/forge.go
  • Imported the time package.
  • Modified the Build function to trigger artifact extraction for the root spec if ArtifactsDir is configured and artifacts are defined.
  • Implemented ExtractArtifacts function, which orchestrates the container creation, artifact copying via docker cp, and container removal, including logging and optional artifact handling.
  • Added resolveArtifactDst helper function for determining artifact destination paths.
  • Updated the Build function's documentation to mention artifact extraction.
• wanda/forge_config.go
  • Added ArtifactsDir field to the ForgeConfig struct to specify the destination for extracted artifacts.
• wanda/forge_test.go
  • Added TestForgeConfigArtifactsDir to verify the ArtifactsDir configuration.
  • Added TestBuild_WithArtifacts_exact to test the extraction of a specific artifact.
  • Added TestBuild_WithArtifacts_optional to test the handling of optional artifacts.
  • Added TestBuild_WithArtifacts_rootOnly to ensure artifacts are extracted only from the root spec.
  • Added TestBuild_WithArtifacts_cacheHit to confirm artifact extraction occurs even on a cache hit.
• wanda/testdata/Dockerfile.artifact
  • New Dockerfile used for testing artifact extraction, creating various test files.
• wanda/testdata/Dockerfile.artifact-top
  • New Dockerfile for testing artifact extraction in a multi-stage or dependent build scenario.
• wanda/testdata/artifact-dep-base.wanda.yaml
  • New Wanda YAML specification defining a base image with an artifact to be extracted.
• wanda/testdata/artifact-dep-top.wanda.yaml
  • New Wanda YAML specification defining a top-level image that depends on artifact-dep-base and has its own artifact.
• wanda/testdata/artifact-exact.wanda.yaml
  • New Wanda YAML specification for testing exact artifact extraction.
• wanda/testdata/artifact-optional.wanda.yaml
  • New Wanda YAML specification for testing optional artifact extraction.

Activity

• The pull request introduces a new feature for artifact copying.
• The author, andrew-anyscale, has implemented the core logic for docker cp operations and integrated it into the wanda build process.
• The PR is currently marked as a draft, indicating ongoing work or awaiting initial feedback.
• The changes are categorized under the wanda-artifact-copy topic and are related to wanda-artifact-spec.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a mechanism to copy artifacts from built Docker images using docker cp. The implementation is straightforward, adding new docker command wrappers and integrating artifact extraction into the build process. The changes are well-tested, covering file and directory copying, optional artifacts, and behavior on cache hits.

My review focuses on improving code style and correctness in the tests and documentation. Specifically:

• Using filepath.Join for path construction in tests to improve portability and adhere to Go idioms.
• Correcting a misleading comment in wanda/forge.go.

I also noticed some code duplication in the new test functions in wanda/docker_cmd_test.go. While I haven't added a specific comment, you might consider refactoring the common setup (pulling image, creating container) into a helper function to make the tests more concise.

Overall, this is a solid contribution that adds valuable functionality.

[aslonnie]

• why we are not using crane this time?
• what will happen when wanda cache hits?

[andrew-anyscale]

why we are not using crane this time?

I didn't see any capabilities for copying out individual files/directories outside of exporting the file system. I can rework if there is an entrypoint I am missing. https://github.com/google/go-containerregistry/blob/main/cmd/crane/doc/crane.md

what will happen when wanda cache hits?

Extraction runs for root spec only, even on cache hit. This is so that CI doesn't have missing artifacts in the case of a cache hit.