Skip to content

ci(sim): keep the asset bakes cached across branches - #771

Open
DavidDobas wants to merge 2 commits into
mainfrom
ci/assets-bake-cache
Open

ci(sim): keep the asset bakes cached across branches#771
DavidDobas wants to merge 2 commits into
mainfrom
ci/assets-bake-cache

Conversation

@DavidDobas

Copy link
Copy Markdown
Collaborator

Follow-up to #767. #753's asset publish still re-baked the apartment (8 min) and the backrooms (12 min); two reasons, two fixes.

  • Bake script split out. The CoACD step moves from build_environment_pack.py to sim/tools/decompose_pack.py, which is the only script the bake layer copies. Editing the finish step (visuals, nav map, viewer glb) no longer re-bakes a pack. build_environment_pack.py now runs the finish step alone.
  • Per-branch build cache. A registry cache export replaces the ref's records, so every branch publish overwrote buildcache and evicted main's bakes. Branches now read main's buildcache plus their own cache-<branch> and write only their own. Main alone writes buildcache.
  • Branch caches carry no sha- tag and are not in the keep-forever list, so cleanup-sim-images.yml ages them out after 14 days without a push.

This branch's own publish bakes the backrooms once (new bake-layer key); the apartment stays cached.

Verified: ruff, tests/ (76 passed), docker build --check -f sim/Dockerfile.assets, both tools load the backrooms scene to the same 29 parts and floor as before.

Split the CoACD bake out of build_environment_pack.py into decompose_pack.py,
the only script copied into the bake layer, so an edit to the finish step no
longer re-bakes every pack. Branch publishes read main's buildcache and write
only their own cache-<branch> ref; main alone writes buildcache, so a branch
can no longer evict main's 17-minute apartment bake.
@greptile-apps

greptile-apps Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR separates expensive environment-pack decomposition from the finishing stage and isolates registry build caches by branch.

  • Moves CoACD decomposition into decompose_pack.py so finishing-tool edits do not invalidate the bake layer.
  • Makes non-main builds read the shared main cache and a collision-resistant scoped cache while exporting only to their scoped cache.
  • Keeps main as the sole writer of the shared buildcache.
  • Documents branch-cache cleanup and the revised environment-pack workflow.

Confidence Score: 5/5

The PR appears safe to merge with no outstanding actionable findings.

The latest changes use PUSH_MAIN_TAGS rather than a ref-name comparison to protect the shared main cache and add a hash to sanitized non-main scopes. The only previous Greptile thread was manually resolved without explanation and therefore is not outstanding.

Important Files Changed

Filename Overview
ci/build_assets_image.sh Uses the authoritative main-build flag to reserve the shared cache and appends a stable hash to non-main cache scopes.
.github/workflows/publish-sim-images.yml Supplies the GitHub ref name as the asset build's cache scope.
sim/Dockerfile.assets Splits expensive hull decomposition into a cacheable layer independent of environment-pack finishing.
sim/tools/decompose_pack.py Extracts the existing scene loading, floor normalization, and CoACD decomposition behavior into a dedicated tool.
sim/tools/build_environment_pack.py Retains only the post-decomposition environment-pack finishing operations.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Publish workflow] --> B{PUSH_MAIN_TAGS?}
  B -->|true| C[Read buildcache]
  C --> D[Write buildcache]
  B -->|false| E[Read buildcache]
  E --> F[Read cache-ref-slug-hash]
  F --> G[Write cache-ref-slug-hash]
  H[Raw environment GLB] --> I[decompose_pack.py]
  I --> J[Cached CoACD hull layer]
  J --> K[build_environment_pack.py]
  K --> L[Visual meshes, nav map, viewer assets]
Loading

Reviews (2): Last reviewed commit: "ci(sim): key the shared asset cache off ..." | Re-trigger Greptile

Comment thread ci/build_assets_image.sh Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant