Skip to content

Emit excluded parent dirs of files re-included by dockerignore negations - #87

Open
Rogerjaytw wants to merge 2 commits into
apple:mainfrom
Rogerjaytw:fix-dockerignore-reinclude-parent-dirs
Open

Emit excluded parent dirs of files re-included by dockerignore negations#87
Rogerjaytw wants to merge 2 commits into
apple:mainfrom
Rogerjaytw:fix-dockerignore-reinclude-parent-dirs

Conversation

@Rogerjaytw

@Rogerjaytw Rogerjaytw commented Jul 4, 2026

Copy link
Copy Markdown

Motivation

Fixes apple/container#1800.

A .dockerignore that excludes a directory's contents but re-includes some of its descendants with negation patterns (the default Rails template does this):

/foo/*
!/foo/.gitkeep
/foo/bar/*
!/foo/bar/.gitkeep

fails the build during [internal] load build context with:

Error: unknown: "failed to solve: changes out of order: "foo/bar/.gitkeep" "foo/.gitkeep""

The walk filter in pkg/fssync/walk.go dropped excluded entries unconditionally. The directory foo/bar itself is excluded by /foo/* and never emitted, while foo/bar/.gitkeep is re-included by !/foo/bar/.gitkeep and streamed to BuildKit as an orphan. BuildKit's receiver (fsutil's Validator) requires every file to arrive after its parent directory and rejects the stream with exactly the error above.

Change

Replace the hand-rolled patternmatcher filtering in Walk with fsutil.NewFilterFS wrapped around the context FS in DiffCopy, mirroring how BuildKit's own filesync provider filters a local context. fsutil implements the full pattern semantics — in particular it lazily emits excluded ancestor directories before a re-included descendant — so .dockerignore handling now matches Docker exactly.

This also passes the exclude patterns from the request metadata through as-is instead of joining and re-splitting them on commas, which previously broke patterns containing a comma.

Scope note: include-pattern filtering stays delegated to the host side (followpaths) as before; only exclude-pattern handling moves into the fsutil filter.

Testing

  • Added a regression test that replays the issue's context tree and patterns through filteredFS + the sender walk, asserting both the emitted order (foo, foo/.gitkeep, foo/bar, foo/bar/.gitkeep) and acceptance by fsutil's Validator — the same check BuildKit's receiver runs. On the previous code the validator rejects the stream with the exact error from the issue.
  • go test ./... passes; gofmt clean; go mod tidy && go mod vendor in sync.

🤖 Developed in collaboration with Claude Code

A .dockerignore that excludes a directory's contents but re-includes some
of its descendants with negation patterns, such as:

    /foo/*
    !/foo/.gitkeep
    /foo/bar/*
    !/foo/bar/.gitkeep

failed the build during "load build context" with:

    changes out of order: "foo/bar/.gitkeep" "foo/.gitkeep"

The walk filter dropped excluded directories entirely, so a re-included
file could be streamed to BuildKit without its parent directory ever
being emitted. BuildKit's receiver validates that every file arrives
after its parent directory and rejected the stream.

Replace the hand-rolled patternmatcher filtering in Walk with fsutil's
NewFilterFS wrapped around the context FS in DiffCopy, mirroring how
BuildKit's own filesync provider filters a local context. fsutil
implements the full pattern semantics, including emitting excluded
ancestor directories before a re-included descendant. This also passes
the exclude patterns through as-is instead of joining and re-splitting
them on commas, which previously broke patterns containing a comma.

Fixes apple/container#1800

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
stephenlclarke added a commit to stephenlclarke/container that referenced this pull request Jul 12, 2026
Map supported Docker-compatible --gpus requests to the Apple virtio-gpu VM device while letting containerization discover guest DRM character-device metadata from the running VM. Pin the matched containerization revision from Package.resolved for Makefile-driven builds, expose builder-shim digest provenance, and pin/attest package release workflow actions.

Release-Highlight: Maps container run/create --gpus to the Apple virtio-gpu backend and projects guest DRM nodes when the guest kernel exposes them, while reporting the immutable builder-shim image digest used by the matched stack.

Refs: apple#1511, apple/containerization#480, apple/containerization#569, apple/container-builder-shim#87
stephenlclarke added a commit to stephenlclarke/container-builder-shim that referenced this pull request Jul 12, 2026
Publish builder images with max provenance and SBOM metadata, report the pushed image digest, enable Dependabot updates for pinned GitHub Actions, and route fork-specific security reports to the stephenlclarke fork.

Release-Highlight: Builder images now publish provenance/SBOM metadata and report the immutable digest that container pins into the matched stack.

Refs: apple#87
stephenlclarke added a commit to stephenlclarke/homebrew-tap that referenced this pull request Jul 12, 2026
Pin the tap workflow action by SHA, enable Dependabot updates for GitHub Actions, and advance the source submodules to the matched Stephen-owned GPU/provenance stack commits.

Release-Highlight: Homebrew tap source snapshots now point at the matched GPU/provenance stack commits and keep release automation actions pinned by SHA.

Refs: apple/container#1511, apple/containerization#480, apple/containerization#569, apple/container-builder-shim#87
stephenlclarke added a commit to stephenlclarke/container-compose that referenced this pull request Jul 12, 2026
Require stable package dispatches to run make release-gate, add a stack-consistency check for matched runtime refs and builder image identity, pin workflow actions by SHA, attest release packages, and refresh GPU docs/handoffs around the Apple virtio-gpu runtime shape.

Release-Highlight: Stable releases now require make release-gate, which runs full CI plus Docker Compose parity before package dispatch.

Release-Highlight: Records the matched container b1917fb, containerization 41252f2, and builder-shim dbe6de4 stack refs, including the digest-pinned builder image.

Release-Highlight: Documents Compose gpus as Apple virtio-gpu support with runtime-discovered guest DRM projection, not Metal/CUDA/vendor passthrough. Upstream references: apple/container#1511, apple/containerization#480, apple/containerization#569, apple/container-builder-shim#87.

Refs: apple/container#1511, apple/containerization#480, apple/containerization#569, apple/container-builder-shim#87
stephenlclarke added a commit to stephenlclarke/homebrew-tap that referenced this pull request Jul 12, 2026
Advance the container-compose source submodule to the stack-gated GPU/provenance commit used for the next matched release.

Release-Highlight: Homebrew tap source snapshots now include the stack-gated container-compose release metadata and GPU parity docs.

Refs: apple/container#1511, apple/containerization#480, apple/containerization#569, apple/container-builder-shim#87
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Dockerignore Resolution Order Differs from Docker Proper

1 participant