Add environment variables for client OS/arch and client user UID/GID

[arnested]

Description

When using Docker Compose for development setup, you often need to tweak the configuration to vary on Linux, macOS, and Windows. And some services you would like to run with the same UID/GID as your current client user.

Our developers use both Linux and Mac (and we have one on Windows as well).

Currently, we cannot vary our Docker Compose files to take the platform differences into account.

To work around that, we have tried to introduce shell scripts that need to be run before using Docker Compose so that it can add .env files or compose.override.yaml. And we tried using conventions like “only a minority uses Linux, and they are quite experienced, so if they export their UID we can use it if set and default to 501 otherwise”.

Neither of those solutions are elegant or error proof, and they remove some intended flexibility of env files and override configs.

To help with that, I suggest introducing four new environment variables while parsing the configuration files:

• COMPOSE_CLIENT_OS: set to Go's runtime.GOOS
• COMPOSE_CLIENT_ARCH: set to Go's runtime.GOARCH
• COMPOSE_CLIENT_UID: set to the current users UID
• COMPOSE_CLIENT_GUID: set to the current users GID

This way, we can now have a Docker Compose setup like this:

compose.yaml:

services:
  php:
    image: php
    volumes:
      - .:/code
    user: ${COMPOSE_CLIENT_UID}:${COMPOSE_CLIENT_GID}
  web:
    image: apache
    extends:
      file: compose.${COMPOSE_CLIENT_OS}.yaml
      service: web

compose.linux.yaml:

services:
  web:
    environment:
      VIRTUAL_HOST: mysite.local

compose.darwin.yaml:

services:
  web:
    environment:
      VIRTUAL_HOST: mysite.docker

The change in #11821 is rather simple. But I'll acknowledge my lack of familiarity with the docker Compose code base and therefore there could be unintended side effects, and maybe it fits better into other parts of the code base. Possibly, it also needs test coverage. And as usual, we can discuss one of the two hard problems: naming things (there might be better names for the environment variables).

At least, I hope this can be given some thought. I know it would make our setups more elegant.

[ndeloof]

do you have a real-world example to require compose.${COMPOSE_CLIENT_OS}.yaml ? Compose has been designed to enforce portability, so introducing such a feature/pattern sounds like we failed.

about support for UID/GID, see #7853

[arnested]

Our primary use case for Docker Compose is running a development setup with websites on the developers' own machine.

To add working HTTPS on the development setup, we use mkcert to add a root certificate to the host machine and add mount it into our containers like this:

    volumes:
      - '${HOME}/.local/share/mkcert:/rootCA:ro'
      - '${HOME}/.local/share/dev_certificates:/cert:rw'

Unfortunately, this location is for Linux only and mkcert places the certificates elsewhere on MacOS.

Our current workaround is to tell Mac users to create a symlink mimicking the location on Linux.

Over the years, we have also used various workarounds for increasing file system performance of volumes on MacOS. Usually, this has been using NFS or some variations, but again only on Mac (I'm not on Mac myself anymore, so I'm not sure of the current state, but I think most of them use Orbstack now).

So we have had all sorts of workarounds (“copy this file to compose.override.yml”, “export these variables”, “create a symlink”, ...).

Our goal has always been to be able to just say “just clone the repo and run docker compose up”.

Compose does a fantastic job for portability, but often we need to interact with the host system outside of Docker -- and they differ.

I'm aware of the other issue regarding UID/GID. It sneaked in to this because when I created the #11821 pull request I realized the two suggestions were very similar in how they could be implemented.

In our day-to-day life, we do vary our setups both regarding OS and UID/GID. But we've had no other choice than to do this through wrapper scripts, README's, and conventions.