Rapidly updating a parent image can cause a child image to use stale base

[lianatech-samuli-yliniemi]

Expected Behavior

Editing a parent image should always result in child images having the most recently edited version of the parent image as their base.

Current Behavior

Often times Tilt can handle the update correctly and child images use the correct base image, but the bigger the project becomes and the more relations there are, Tilt will start to get confused as to what is the correct base image for a child.

I have created and attached a test scenario which can be used to reproduce the bug. As the bug scenario can be hard to trigger, since it is most likely caused by some race condition, the script may have to be run for some time.

Short description what the scenario contains:

• Tiltfile with k8s deployments and Docker builds for parent, child, child-of-child and child-of-child-of-child.
• A test.sh script which can be run, it does the following:
  • Wait for Tilt state to settle, every resource is Ready and UpToDate.
  • Write to a file on random intervals, which causes parent image to rebuild and the child images to rebuild. The child images have a sleep 10 to simulate a longer build.
  • When some of the resources, but not all, are ready, write again to cause another rebuild. Do this 5 times.
  • Wait for tilt resources to settle.
  • Check if the file from the parent image is correctly inherited to all the child images.
  • Exit if the file from parent image does not have the same value across all the images.

Steps to Reproduce

1. Download the test scenario. tilt-test.tar.gz
2. Run kind cluster, I used kind-with-registry.
3. All the image URLs point to localhost:5001, edit if needed.
4. Run tilt up
5. Run test.sh and wait for it to find the bug.

The expected output from the script should be:

Value from parent: 0
Value from child: 0
Value from child-of-child: 0
Value from child-of-child-of-child: 1
Unique values: 2
Unique values not 1!

This indicates that the parent deployment is running with the latest parent image, and has a value of 0 in /textfile.txt.
The child images inherit this image, so they should have the same value in the file, but for some reason one of the child images has pulled an old version of the parent image, which contained another value.

This can be verified from the Tilt logs, where the misbehaving child deployment has this type of log:

STEP 1/4 — Loading cached images
     - localhost:5001/parent:tilt-63bf9fc15a0e57a3
     - localhost:5001/child:tilt-09f029fe80035ee1

While a normally working deployment has this:

STEP 1/4 — Loading cached images
     - localhost:5001/parent:tilt-b7b0161a37d5c5c3
     - localhost:5001/child:tilt-35aa0936459e4683

And finally we can see the parent deployed like this:

STEP 3/3 — Deploying
     Applying YAML to cluster
     Objects applied to cluster:
       → parent:deployment

     Step 1 - 0.24s (Building Dockerfile: [localhost:5001/parent])
     Step 2 - 0.01s (Pushing localhost:5001/parent:tilt-b7b0161a37d5c5c3)
     Step 3 - 0.01s (Deploying)
     DONE IN: 0.26s 

[event: pod parent-7f55fdc75c-5wntf] Container image "localhost:5001/parent:tilt-b7b0161a37d5c5c3" already present on machine

From these logs we can see that the misbehaving child deployment got a wrong tag for the parent image.

Context

tilt doctor Output

$ tilt doctor
Tilt: v0.35.0, built 2025-06-13
System: linux-amd64
---
Docker
- Host: unix:///var/run/docker.sock
- Server Version: 28.0.4
- API Version: 1.48
- Builder: 2
- Compose Version: v2.34.0
---
Kubernetes
- Env: kind
- Context: kind-kind
- Cluster Name: kind-kind
- Namespace: default
- Container Runtime: containerd
- Version: v1.32.0
- Cluster Local Registry: &RegistryHosting{Host:localhost:5001,HostFromClusterNetwork:,HostFromContainerRuntime:,Help:https://kind.sigs.k8s.io/docs/user/local-registry/,SingleName:,}
---
Thanks for seeing the Tilt Doctor!
Please send the info above when filing bug reports. 💗

About Your Use Case

We have a pretty big Tilt-setup where this issue can be reproduced quite easily by making some changes to a base image.

The idea of the setup is to have some library images, utility images etc. that can be used as a base for other images. Making changes to these base images should reflect into other images, to be able to do seamless development.