Skip to content

0000-prepare-operation.md

Latest commit

 

History

History
374 lines (253 loc) · 13.5 KB

0000-prepare-operation.md

File metadata and controls

374 lines (253 loc) · 13.5 KB

Meta

  • Name: Prepare Operation
  • Start Date: 2022-02-07
  • Author(s): @jromero
  • Status: Draft
  • RFC Pull Request: (leave blank)
  • CNB Pull Request: (leave blank)
  • CNB Issue: (leave blank)
  • Supersedes: (put "N/A" unless this replaces an existing RFC, then link to that RFC)

Summary

Definitions

  • namespace - A table within project.toml that encompasses a set of properties.
  • Prepare operation - a new operation that occurs before the Build operation.
  • project.toml - A project descriptor file specified here.

Motivation

The motiviation for this change was to enable the following goals from being achieved:

  • Goal 1: Serializing CLI configuration

    As a user, I would like to be able to serialize and share the parameters I use with certain platforms such as pack.

  • Goal 2: A recognizable file in repositories

    As a user, I would like to be able to recognize, based on the file system, if a project is using Cloud Native Buildpacks.

  • Goal 3: Platform recognition of project.toml

    As a user, I would like to ensure that my configuration in project.toml is being used.

NOTE: This RFC directly solves for Goal 3, while enabling the solutions for Goals 1 and 2 via a supplimental RFC (Support for pack.toml).

What it is

The proposal is composed of following changes:

  1. Moving io.buildpacks properties to io.buildpacks.defaults.
  2. A replaceable new phase prepare that applies platform configuration.
  3. Supporting Cloud Native Buildpack utilities.

Changes to io.buildpacks namespace

Namespace io.buildpacks.defaults

This namespace declares default values that platforms should acknowledge. A majority of properties should be spec'd but we would allow for unspec'd additional properties.

The criteria for spec'd properties in this namespace would be based on whether they affect inputs to the lifecycle phases. For example, the order and environment variables are inputs to the lifecycle. The builder, in contrast, is not an input to the lifecycle so it wouldn't meet the criteria.

Therefore the following changes have been made from the latest schema:

  • Remove builder

Version io.buildpacks namespace

Given that the project descriptor's io.buildpacks namespace is directly tied to the platform by the fact that it's the platform that needs to consume and parse it, it makes sense to unify the version of the schema with that of the platform API.

Schema

See Platform Spec changes.

Usages Examples

Multi-Platform (pack + kpack)

In the following example we show how various platform namespaces can have their own configuration but properties can be promoted to io.buildpacks.defaults if the user's intent is to apply it to all platforms.

[io.buildpacks]
schema-version = "0.2"

###
# common buildpacks config
##

[io.buildpacks.defaults]
exclude = ["some-files/**"]

##
# pack config
##

[io.buildpacks.pack]
schema-version = "0.1"
builder = "cnbs/sample-builder:bionic"
image = "my-app"                        # (example only)
run-image = "my-run-image:latest"       # (example only)

##
# kpack config
##

[com.vmware.kpack]
schema-version = "0.1"

[[com.vmware.kpack.build.env]]
name="CUSTOM_ENV"
value="SOME_VALUE"

Notice:

  1. Both pack and kpack are expected to apply io.buildpacks.defaults.exclude.
  2. io.buildpacks.pack has additional properties that are specific to pack only.

Prepare phase

A prepare phase would be a new lifecycle phase that gets executed before create or detect phase. Similar to the rebaser, it is outside of the standard Build operation.

Responsibilities

At minimum, the expected reponsibility of the preparer would be to:

  • Apply the requested configuration.

Inputs

The prepare phase would take all the same inputs as creator plus the addition of a path to project.toml.

Outputs

The prepare phase may affect the file system, and mutation or create files that would be consumed be futher phases.

Cloud Native Buildpacks utilities

The Cloud Native Buildpacks project will provide the following functionality in the form of utilities:

go prepare function

A function that can help developers apply configuration to the filesystem based on a provided configuration.

This would be useful for builder providers or platform implementers that choose to provide their own preparer executable.

preparer executable

The Cloud Native Buildpacks project will have a preparer it ships along with the existing lifecyle image.

This would be useful for builder providers or platform implementers to not have to develop standard functionality.

The default implementation COULD take care of applying the following configuration:

  • io.buildpacks.defaults.group →
    • Download buildpacks
    • Update order.toml
  • io.buildpacks.defaults.build.env →
    • Set build env vars in <platform>/env
  • Notify users of any other properties in io.buildpacks.defaults

Drawbacks

  1. Parsing and applying the io.buildpacks namespace becomes responsibility of the platform.
    • This is mitigated by providing utilities and the fact that the prepare phase is an independant and swappable.
  2. Executing the Prepare operation may require an additional container to be spun up in some platforms; this would effectively increase the overall build process.

Alternatives

Converter RFC

The idea is to ship a binary with the lifecycle that would be responsible for translating project.toml from the schema defined in the project descriptor extension spec into something that the lifecycle knows the platform can understand i.e., a schema defined in the platform spec.

  • Benefits
    • Converting various possible user provided schemas to a platform API schema makes it easier for platforms to consume the configuration.
      • Counter-point: The "ease" of consumption could be provided by other means such as language-specific libraries that ease parsing to general models.
  • Drawbacks
    • Having a converter instead of an entire swappable prepare phase means that the process of applying configuration becomes a 2-step process for platforms.

Project Descriptor Buildpack

The idea of a buildpack that can apply configuration from project.toml has been kicked around and would work for some properties such as environment variables.

  • Benefits
    • Buildpacks are more robust. Updated versions can be used to apply the later versions of project descriptor with no platform operator/implementer intervention.
  • Drawbacks
    • Not all operations may be applied at the buildpack level. For example, buildpack order. There would need to be a higher-order operation to apply other parts of the configuration. Given a higher-order operation, it doesn't make sense to split the application of the configuration.

Prior Art

  • rebase - Another "out-of-build" operation.

Unresolved Questions

  • Should arbitrary properties be allowed in io.buildpacks.defaults?
  • How does the Prepare operation make changes to non-filesystem options such as tags, run-image, etc?
    • Ideally, the lifecycle would have a filesystem based interface that we can leverage. This would prevent the preparer from having it's own independant mechanism. A prior RFC for something similar has existed (see Add Lifecycle Config File RFC). It may be worth revisiting.

Spec. Changes

Distribution Spec

No changes necessary.

Platform Spec

Add Project Descriptor io.buildpacks namespace

project.toml (TOML)

The format of project.toml MUST adhere to version 0.2 of the project descriptor specification. Within the project.toml file the io.buildpacks namespace MAY be defined.

io.buildpacks namespace
[io.buildpacks]
schema-version = "<platform API version>"

[io.buildpacks.defaults]
include = ["<.gitignore pattern>"]
exclude = ["<.gitignore pattern>"]

[[io.buildpacks.defaults.pre.group]]
id = "<buildpack ID>"
version = "<buildpack version>"
uri = "<url or path to the buildpack"

  [io.buildpacks.defaults.pre.group.script]
  api = "<buildpack API version>"
  shell = "<string>"
  inline = "<script contents>"

[[io.buildpacks.defaults.group]]
id = "<buildpack ID>"
version = "<buildpack version>"
uri = "<url or path to the buildpack"

  [io.buildpacks.defaults.group.script]
  api = "<buildpack API version>"
  shell = "<string>"
  inline = "<script contents>"

[[io.buildpacks.defaults.post.group]]
id = "<buildpack ID>"
version = "<buildpack version>"
uri = "<url or path to the buildpack"

  [io.buildpacks.defaults.post.group.script]
  api = "<buildpack API version>"
  shell = "<string>"
  inline = "<script contents>"

[[io.buildpacks.defaults.build.env]]
name = "<name>"
value = "<value>"

Where:

  • schema-version (required): is the version of the schema which correlates with the platform API.
  • defaults (optional): is a table of default properties that all platforms should apply. include and exclude are mutually exclusive. If both are present the build process MUST result in an error.
    • include (optional): is an array of .gitignore pattern-based paths to include during the build operation.
    • exclude (optional): is an array of .gitignore pattern-based paths to exclude from the build operation and thereby produced image.
    • group (optional): is an array of buildpacks. Either a version, uri, or script table MUST be included, but MUST NOT include any combination of these elements.
      • id (optional): is the ID of the buildpack.
      • version (optional, default=latest): is the version of the buildpack.
      • uri (optional, default=urn:buildpack:<id>): is the URI to the buildpack.
      • script (optional): defines an inline buildpack.
        • api (required): is the api key defines its Buildpack API compatibility.
        • shell (optional, default=/bin/sh): defines the shell used to execute the inline script.
        • inline (required): is the build script for the inline buildpack.
    • build (optional):
      • env (optional): an array table that defines environment variables to be applied during the build phase.
        • name (required): is the name of the environment variable.
        • value (required): is the value of the environment variable.

Add Prepare operation

Prepare

Before the Build operation is executed, a platform MUST prepare the build environment.

During, the Prepare phase, the platform:

  • SHOULD apply provided Project Descriptor configuration is present.
    • SHOULD generate a warning for any io.buildpacks.defaults property not applied.
  • MAY make changes to any path in the filesystem.

Add prepare section

preparer

Usage:

/cnb/lifecycle/preparer \
  [-app <app>] \
  [-buildpacks <buildpacks>] \
  [-cache-dir <cache-dir>] \
  [-cache-image <cache-image>] \
  [-daemon] \
  [-gid <gid>] \
  [-launch-cache <launch-cache> ] \
  [-launcher <launcher> ] \
  [-layers <layers>] \
  [-log-level <log-level>] \
  [-order <order>] \
  [-platform <platform>] \
  [-previous-image <previous-image> ] \
  [-process-type <process-type> ] \
  [-project-descriptor <project-descriptor> ] \
  [-project-metadata <project-metadata> ] \
  [-report <report> ] \
  [-run-image <run-image>] \
  [-skip-restore <skip-restore>] \
  [-stack <stack>] \
  [-tag <tag>...] \
  [-uid <uid> ] \
  <image>

The preparer SHOULD accept the same inputs as the creator with the addition of the following:

Input Environment Variable Default Value Description
<project-descriptor> CNB_PROJECT_DESCRIPTOR_PATH <app>/project.toml Path to a Project Descriptor
Outputs

A preparer may make general changes to the file system, modify input files, or create input files.

Exit Code Result
0 Success
1-10, 13-19 Generic lifecycle errors

Project Descriptor Spec

Remove io.buildpacks namespace

We'll want to remove the io.buildpacks namespace since it will now be defined in the Platform spec.