Allow `if-not-present` pull policy for builds with image extensions

[MoritzWeber0]

Description

When running pack build and the builder contains image extensions, the build fails with:

ERROR: failed to build: pull policy must be 'always' when builder contains image extensions

Proposed solution

The pack build command should accept --pull-policy if-not-present as argument, also if the builder contains image extensions.

Additional context

I don't want to rely on a local registry or internet connection when building my images locally.

[jjbustamante]

Make sense to me, but no sure if @natalieparellano has some thoughts about it

[natalieparellano]

The issue with --pull-policy if-not-present is that we will not get a manifest for an image that only exists in the daemon (though with newer versions of docker using containerd as the backing store, that will hopefully one day be possible).

However, I believe some recent changes to imgutil (see https://github.com/buildpacks/imgutil/blob/main/local/v1_facade.go) should make it possible to construct a dummy manifest that may be good enough for our purposes (kaniko). We should do an exploration for it.

[hhiroshell]

Could you please assign this to me? This issue involves a slightly deeper implementation of the pack than those I've worked on before, but I’d like to take on more challenging tasks.

Anyway, I'm thinking about start with this exploration. I’d appreciate any advice you could share.

[natalieparellano]

@hhiroshell thank you for looking into this, and apologies for the slow reply - after digging into this a little further, I think this issue might make more sense on the lifecycle as the bulk of the effort would be there, although both pack and the lifecycle would require changes.

• Here is where we enforce in pack that the pull policy must be 'always' in order to use extensions
• Here is where the lifecycle assumes that the provided base image for extensions must exist in a registry; if you replace the call to remote.NewImage with local.NewImage, the returned image (because of the mentioned changes in imgutil) should still work.

That should be sufficient as an exploration. To make this a "real" feature, we'd want to inspect the -daemon flag in the restorer in order to instantiate a remote or local image depending on the type of build. And, we'd want to liberally test this, by adding to the acceptance tests for extensions in both pack and the lifecycle to ensure that builds work as expected with this change. Finally we'd probably want to clarify and update the spec as I believe the "remote only" restriction is mentioned in a couple of places.

Unfortunately, this may be a little difficult to work on without guidance from someone with a lot of context. I have worked a lot on extensions, but I am about to step away for several months. I don't want to discourage anyone from looking into this however, as it would be good for others to build context, just cautioning that a fair amount of discovery may be needed!

Hope this is helpful. Best of luck!

[hhiroshell]

Hi @natalieparellano,
I’m sorry for not making progress on this yet. I’ve been quite busy with various events, including attending KubeCon.

Thank you so much for your helpful guidance—it’s been invaluable. It seems like there’s a lot of work to do, but now that the tasks are somewhat clear, I believe we can make progress.

And I’m truly sorry to hear that you’re stepping away... 😢

[hhiroshell]

I am working on a prototype implementation focused on extensions for build images to assess the feasibility of this functionality. Although it is not yet fully functional, I would like to share changes on the lifecycle I have identified as necessary to achieve it.

• Use local.NewImage() instead of remote.NewImage() in pullSparse() of restorer.go (as @natalieparellano mentioned) (source)
• Update the Exec() function in restorer.go to write the result of imgutil.Image.Digest() as the reference for the build image in analyzed.toml. Currently, the reference value is derived from the result of imgutil.Image.Identifier() (source)
  • During the restore phase, the kanikoDir path is generated based on the value of imgutil.Image.Digest(). In the extend phase, the kanikoDir path is constructed using the reference value. For these paths to align, the reference value must match the value of imgutil.Image.Digest(). Previously, this was not an issue because Digest() and Identifier() returned the same value. However, this is no longer the case with local.NewImage()
    • 
  • While it seems possible to use Identifier() for the kanikoDir path and reference, we have to avoid this approach. This is because only Digest() matches the hash of the manifest included in the OCI layout extracted to kanikoDir. Otherwise, the image would fail to load from the kanikoDir layout during the extend phase.
• Remove the logic for extracting the digest from the reference in the ImageFor() function of dockerfile_applier.go (source)
  • The reference value has been updated to use the result of local.NewImage().Digest(), resulting in a format change from the unchanged version:
    • before: localhost:5000/extensions-builder@sha256:14ac1df2b41...
    • after: sha256:ba92dbce0...

As I mentioned, these changes are not sufficient to make the functionality work. Running pack build with the lifecycle image that includes the above changes results in the following error in the extender:

[extender (build)] ERROR: failed to extend build image: extending build image: unsupported hash: ""

The issue is suspected to be caused by the digest values for the layers in the manifest of the OCI layout extracted to kanikoDir being set to : (a colon only). If this assumption is correct, v1_facade.go will need to be updated to ensure the layers include valid digest values.

I will next verify whether the above changes to v1_facade.go can be implemented as described.