Custom Docker image not rebuild correctly after merging

[JuJup]

Bug description

We have a very strange behavior with our Gitpod Setup using a custom Dockerfile.
We have a main branch A and a feature branch B. In B we're testing some new things so the Dockerfile differs slightly but is mainly equal to A. From time to time, we merge the main branch A into the feature branch B to keep it up to date.
In this case, the Docker Image is rebuild for B, but with the Dockerfile from A.
Sounds confusing? Yes it is :lau

I described the steps using a minimal example to show the problem:

Steps to reproduce

1. Create a project with these two files (main):

# .gitpod.Dockerfile
FROM gitpod/workspace-base:latest

RUN touch main-branch.info

# .gitpod.yml

image:
  file: .gitpod.Dockerfile

2. Start a Gitpod workspace for main
   -> ls ~ should show only main-branch.info

3. Create a Feature Branch (test-branch) and change the Dockerfile to:

# .gitpod.Dockerfile
FROM gitpod/workspace-base:latest

RUN touch main-branch.info

RUN touch test-branch.info

4. Start a Gitpod workspace for test-branch
   -> ls ~ should show main-branch.info and test-branch.info
   -> So far everything is fine

5. Make a change to Dockerfile in main:

# .gitpod.Dockerfile
FROM gitpod/workspace-base:latest

RUN touch main-branch-refactored.info

6. Start a Gitpod workspace for main
   -> ls ~ should show only main-branch-refactored.info

7. Merge main into test-branch:

8. Start a Gitpod workspace for test-branch
   -> ls ~ shows main-branch-refactored.info but not test-branch.info ⚠️

That's already confusing, as the code now doesn't represent the state anymore. So I tried rebuilding the image explicitely:

9. Open https://gitpod.io/#imagebuild/https://gitlab.com/<group>/<project>/-/tree/test-branch
   -> Image is rebuild (you can see the logs, but you can see it's the wrong Dockerfile from main)
   -> ls ~ shows main-branch-refactored.info but not test-branch.info ⚠️

The only thing that helps now is changing some line in the Dockerfile in test-branch. But this can't be the solution honestly.

Workspace affected

No response

Expected behavior

• When the Dockerfile changes (no matter if directly in the branch or through a merge from main branch), the Image should be rebuild correctly from the branch's Dockerfile
• When I use https://gitpod.io/#imagebuild/, I expect the rebuild to be base on the branch's Dockerfile

Example repository

No response

Anything else?

No response

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[JuJup]

Did anybody had a chance to take a look at that?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[shintonik]

@JuJup Thanks for bringing up the issue. We are dealing with the same problem.

Every time we merge the main branch into a feature branch it often breaks our Gitpod workspaces which then needs a manual fix. The workaround is basically making a commit with a pseudo change to gitpod.Dockerfile to break the cache and trigger a prebuild. But this is really tedious. We would really appreciate if someone could look into this bug.

Thank in advance.

[axonasif]

Hi @shintonik, are you using prebuilds?

[shintonik]

Yes, we use prebuilds.

[JuJup]

Hey @axonasif,
can you reopen the issue?

[axonasif]

Hi @JuJup

Done