Gaps of Paketo in offline environments

[braunsonm]

From discussion on Slack, currently it seems Paketo has some gaps in how it handles offline buildpacks. I hope to outline some of the issues that exist in the Paketo ecosystem today in hopes that they might get prioritized and addressed by the Paketo team.

1. Buildpacks use different underlying libraries which changes how users need to package the buildpacks for offline use. Java buildpacks use libpak which requires using the tool create-package. Others use packit which require the use of jam.
2. There is insufficient tooling to create offline buildpacks from a language family or builder family. Currently you need to individually create-package and jam every layer in a language family, and create your own buildpack using those packaged layers
3. There is no documentation on how to create offline buildpacks
4. Due to (2) and (3), offline environments need to create their own tooling or automation in order to create these buildpacks and keep them up to date. This creates a security issue since buildpack layers get updated frequently and it is untenable to keep them up to date manually when every layer needs to be packaged individually

For the Java buildpacks you will need to use the create-package tool which can be found here https://github.com/paketo-buildpacks/libpak/tree/main/cmd/create-package. With this you will need to run create-package --output ${SOME-OUTPUT} --version ${SOME-VERSION} --include-dependencies this should produce an offline buildpack artifact.

For all of the other buildpacks you should be able to use jam which can be found here https://github.com/paketo-buildpacks/jam. With this you will need to run jam pack --buildpack ${PATH-TO-BUILDPACK.toml} --version ${SOME-VERSION} --output ${SOME-OUTPUT} --offline this should produce an artifact that will work offline.

However, this needs to be done on every layer for every job family, then combined into a custom job family builder, then combined yet again into a builder image which would be more analogous to the builder:full image that exists today.

Cloud Foundry offers a better developer experience on their own buildpacks, being able to create an offline buildpack at the language family level: https://github.com/cloudfoundry/java-buildpack#offline-package

I am hoping the Paketo maintainers can brainstorm with the community on two ways to rectify these issues:

1. Create documentation on using paketo buildpacks in an offline environment
2. Improve tooling for creating offline buildpacks. Hopefully by standardizing the tools required to create offline packages and allow entire language families to be packaged in one command rather than one of each layer, and then creating a custom buildpack.toml with those layers.

[loewenstein]

This might be a good place to pitch https://docs.google.com/document/d/1g5rRW-oE_v8Gdvz-CiCOK9z2rxg6L5XniKI25Zq2j6M/edit#heading=h.ucm3oxm9w03a as a potential part of the solution.

Not sure if this is reaching too far, but it could be an alternative to investing into specific offline buildpacks tooling.