Docker not recognizing `[docker].env_vars`

[rhuanbarreto]

Describe the bug
I setup pants.toml with:

[docker]
env_vars = ["GITHUB_SHA=development"]
build_args = ["GITHUB_SHA"]

[environments-preview.names]
docker_linux = "//:docker_linux"
local_mac = "//:local_mac"
local_linux = "//:local_linux"

And BUILD file:

local_environment(
    name="local_mac",
    compatible_platforms=["macos_arm64"],
    fallback_environment="local_linux",
)

local_environment(
    name="local_linux",
    compatible_platforms=["linux_x86_64"],
    fallback_environment="docker_linux",
)

docker_environment(
    name="docker_linux",
    platform="linux_x86_64",
    image="python:3.11.2-slim-bullseye",
    python_bootstrap_search_path=["/usr/local/bin"],
)

I get an error when I run pants package :: :

[ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

DockerBuildContextError: Undefined value for build arg on the <redacted>:app target: The Docker environment variable 'GITHUB_SHA' is undefined. You may provide a value for this variable either in `[docker].env_vars` or in Pants's own environment.

If you did not intend to inherit the value for this build arg from the environment, provide a default value with the option `[docker].build_args` or in the `extra_build_args` field on the target definition. Alternatively, you may also provide a default value on the `ARG` instruction directly in the `Dockerfile`.

Pants version
2.15.0

OS
Mac OS M1 Max

[stuhood]

I think that there is some missing context here: the error that you've included is for building a docker_image target, and docker_image targets don't necessarily use / rely-on docker_environments.

Can you include the definition of the relevant docker_image target?

[rhuanbarreto]

Here it goes:

def streamlit_app(name: str) -> None:
    """Define the build targets for a Stremlit PoC in Pants."""
    python_version = open(".python-version", "r").readline().rstrip("\n")
    pex_binary(
        name="bin-deps",
        entry_point="streamlit",
        environment="docker_linux",
        layout="packed",
        include_sources=False,
        execution_mode="venv",
        include_tools=True,
    )

    pex_binary(
        name="bin-srcs",
        entry_point="streamlit",
        environment="docker_linux",
        layout="packed",
        include_requirements=False,
        execution_mode="venv",
        include_tools=True,
    )

    python_sources()
    files(name="sources", sources=["*.py", "!*_test.py"])

    docker_image(
        name="app",
        dependencies=[":sources"],
        # TODO: Waiting for https://github.com/pantsbuild/pants/issues/18386
        # restartable=True,
        image_tags=["{build_args.GITHUB_SHA}"],
        instructions=[
            f'FROM --platform="linux/amd64" python:{python_version}-slim-bullseye as deps',
            f"COPY src.pocs.{name}/bin-deps.pex /binary.pex",
            "RUN PEX_TOOLS=1 /usr/local/bin/python /binary.pex venv --scope=deps --compile /bin/app",
            f'FROM --platform="linux/amd64" python:{python_version}-slim-bullseye as srcs',
            f"COPY src.pocs.{name}/bin-srcs.pex /binary.pex",
            "RUN PEX_TOOLS=1 /usr/local/bin/python /binary.pex venv --scope=srcs --compile /bin/app",
            f'FROM --platform="linux/amd64" python:{python_version}-slim-bullseye',
            "RUN apt-get update && apt-get -y upgrade && rm -rf /var/lib/apt/lists/*",
            "RUN groupadd --gid 1000 appuser && useradd --uid 1000 --gid 1000 -ms /bin/bash appuser",
            "USER appuser",
            "ENV STREAMLIT_BROWSER_GATHER_USAGE_STATS False",
            "ENV STREAMLIT_SERVER_ENABLE_CORS True",
            "ENV STREAMLIT_SERVER_FILE_WATCHER_TYPE none",
            "ENV STREAMLIT_SERVER_PORT 80",
            "ENV STREAMLIT_GLOBAL_DEVELOPMENT_MODE False",
            "COPY --from=deps /bin/app /bin/app",
            "COPY --from=srcs /bin/app /bin/app",
            f"COPY src/pocs/{name} /home/appuser/app",
            "EXPOSE 80",
            'ENTRYPOINT ["/bin/app/pex", "run", "/home/appuser/app/app.py"]',
        ],
    )

[d-tw]

I have encountered the same issue, where env_vars are not used for docker commands when I'm using the environment-aware features.

Here's my config:

# pants.toml

[docker]
build_verbose = true
# These are the env vars that need to be made available for the docker
# engine to publish
# The platform env is a workaround for https://github.com/pantsbuild/pants/issues/17539
env_vars = [
  "USER",
  "HOME",
  "DOCKER_DEFAULT_PLATFORM=linux/amd64",
  "AWS_ACCESS_KEY_ID",
  "AWS_SECRET_ACCESS_KEY",
  "BUILDX_BUILDER",
  "DOCKER_BUILDKIT",
  "AWS_PROFILE"
]

[environments-preview.names]
linux = "//:linux_local"
linux_docker = "//:linux_docker"
macos = "//:macos_local"

Root BUILD file:

local_environment(
  name="linux_local",
  compatible_platforms=["linux_x86_64"],
  fallback_environment="linux_docker",
)

local_environment(
  name="macos_local",
  compatible_platforms=["macos_x86_64"],
)

docker_environment(
  name="linux_docker",
  platform="linux_x86_64",
  image="python:3.10",
  python_bootstrap_search_path=["<PATH>"]
)

And the BUILD file for the Docker

docker_image(
    name="remco"
)

When I run pants package locally on my mac, the various env vars are correctly propagated. When pants package is run on a linux machine (in this case in CI), the env vars are missing, leading a failing build. The issue is identical to #18764.

When I remove all of the environment-aware settings, the builds work both locally on my mac and on linux.

[davidbeers]

I'm seeing the same bug as @rhuanbarreto . I see that the ticket is still open, but is there a workaround at least?

[davidbeers]

I'm seeing the same bug as @rhuanbarreto . I see that the ticket is still open, but is there a workaround at least?

I guess the workaround is adding the default ARG to the Dockerfile, which is mentioned in the error message.