Editable packages only for specific features/environments

[geoHeil]

When using editable local installations of pip packages I run into issues when building docker containers.
As a software engineer I try to build containers using layer caching in mind to reduce build times.
There are multiple environments (like a static one with CI tools like testing) and a dynamic / app one with my code. I want to be able to develop from the container (must use editable installation and map the volumes).
Pixi however requires the dynamic content to be solvable also for installing the static environment. How can this be changed to facilitate layer caching?

My pixi file:

[project]
name = "foo"
version = "1.0.0"
description = "bar"
authors = [
  "baz",
]
platforms = ["linux-64", "osx-arm64"]
channels = ["conda-forge"]

[environments]
myenv-static = { features = ["myenv-static"], solve-group = "myenv-static" }
myenv-dynamic = { features = ["myenv-dynamic"], solve-group = "myenv-dynamic" }


[dependencies]
python = "3.11"
pip = ">=24.2"


[feature.myenv-static.dependencies]
pytest = ">=8.3.2,<9"

[feature.myenv-static.pypi-dependencies]
moto = "~=5.0.12"

[feature.myenv-dynamic.dependencies]
pandas = ">2,<3"

[feature.myenv-dynamic.pypi-dependencies]
mylib = { path = "./mylib", editable = true }

My dockerfile:

# syntax=docker/dockerfile:1.9.0

FROM ghcr.io/prefix-dev/pixi:0.27.1-bookworm

WORKDIR /app
COPY pixi.toml .
COPY pixi.lock .

# this is a bad -> layer caching is colliding with editable packages
# static env is NOT using any dynamic packages
#COPY mylib mylib

RUN pixi install -e myenv-static --locked
RUN pixi shell-hook -e myenv-static -s bash > /shell-hook
RUN echo "#!/bin/bash" > /app/entrypoint_myenv_static.sh
RUN cat /shell-hook >> /app/entrypoint_myenv_static.sh
RUN echo 'exec "$@"' >> /app/entrypoint_myenv_static.sh


COPY mylib mylib
# the source code is only needed here
RUN pixi install -e myenv-editable --locked
RUN pixi shell-hook -e myenv-editable -s bash > /shell-hook
RUN echo "#!/bin/bash" > /app/entrypoint_myenv_editable.sh
RUN cat /shell-hook >> /app/entrypoint_myenv_editable.sh
RUN echo 'exec "$@"' >> /app/entrypoint_myenv_editable.sh

When executed with docker buildx build --platform=linux/amd64 -t foo:xx . I receive the following error:

RUN pixi install -e myenv-static --locked:
0.253   × lock-file not up-to-date with the project

This can be remediated when enabling/pushing up the COPY mylib mylib before the install statement - however, this invalidates/reduces the layer caching

See https://github.com/geoHeil/pixi-editable-reproducible-quesetion-v1 for details and folder structure

[ruben-arts]

Nice setup, to skip the sanity checks in the installation, you can add --frozen to all commands that want to install.

# syntax=docker/dockerfile:1.9.0

FROM ghcr.io/prefix-dev/pixi:0.27.1-bookworm

WORKDIR /app
COPY pixi.toml .
COPY pixi.lock .

RUN pixi install -e myenv-static --frozen -vvv
RUN pixi shell-hook --frozen -e myenv-static -s bash > /shell-hook
RUN echo "#!/bin/bash" > /app/entrypoint_myenv_static.sh


COPY mylib mylib
RUN pixi install -e myenv-dynamic --frozen -vv
RUN pixi shell-hook --frozen -e myenv-dynamic -s bash > /shell-hook

[geoHeil]

I.e. for dynamic I would want to keep the validation and only for static use --frozen however ideally this would be feature/env specific

[geoHeil]

And even when setting both as frozen - the shell-hook fails

[geoHeil]

RUN pixi shell-hook -e myenv-static -s bash > /shell-hook:
2.698 × failed to solve the pypi requirements of 'myenv-dynamic' 'osx-arm64'
2.698 ├─▶ failed to resolve pypi dependencies
2.699 ├─▶ Failed to build mylib @ file:///app/mylib
2.699 ╰─▶ Source distribution directory contains neither readable pyproject.toml
2.699 nor setup.py: mylib

when manually creating the directory

[geoHeil]

but this is exactly what I do not want

[geoHeil]

Indeed, I overlooked the hell-hook --frozen. Works fine then