[BUG] docker-compose service with bind propagation set to rshared and create_host_path set to true not working as expected

[hs-siso]

Description

I have a docker-compose file (test.yml) with the following content:

# cat test.yml
---
services:
  test_db:
    image: postgres:15.10-bullseye
    volumes:
      - type: bind
        source: /tmp/postgres
        target: /var/lib/postgresql/data
        bind:
          create_host_path: true
          propagation: rshared

which I am trying to start but errors out with:

# docker-compose -f test.yml up -d
[+] Running 0/0
 ⠋ Container bundle_install-test_db-1  Creating                                                                                                                                                                  0.0s
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/postgres

As per this doc I am expecting that the source path will be created by docker stating:

create_host_path: Creates a directory at the source path on host if there is nothing present. Compose does nothing if there is something present at the path. This is automatically implied by short syntax for backward compatibility with docker-compose legacy.

However, this does not seem to be the case and it seems that create_host_path defaults to true and works only with the basic access modes - rw or ro.

As this is nowhere to be explained in the official docs my supposition is it is a bug.

Steps To Reproduce

1. Create a test.yml with contents such as (make sure that the bind propagation is set to rshared):

---
services:
  test_db:
    image: postgres:15.10-bullseye
    volumes:
      - type: bind
        source: /tmp/postgres
        target: /var/lib/postgresql/data
        bind:
          create_host_path: true
          propagation: rshared

2. Try to start the service with docker-compose -f test.yml up
3. You should see the same error I am hitting, i.e.:

Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/postgres

Compose Version

./docker-compose --version
Docker Compose version v2.32.2

Docker Environment

# docker info
Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 162
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version:
 runc version:
 init version:
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.2.0-1016-aws
 Operating System: Ubuntu 22.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.32GiB
 Name: devops-testing-master01
 ID: 0c39d9d9-1baf-4fbf-96e3-35996d09750d
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

With docker-compose v1.29.2 this worked OK.

Here is a dummy example:

• test.yml contents

$ cat test.yml                                                                                                                                                                                                    
---
services:
  test_db:
    image: alpine:3.21.2
    volumes:
      - /tmp/postgres:/var/lib/postgresql/data:rshared
    command: >
      sh -c "echo test > /var/lib/postgresql/data/testfile.txt

• which creates the /tmp/postgres dir

$ sudo ls /tmp/postgres                                                                                                                                                                                          
testfile.txt

• docker-compose's version

$ ./docker-compose --version                                                                                                                                                                                     
docker-compose version 1.29.2, build 5becea4c

[glours]

Hello @hs-siso

I'm not able to reproduce with your example, here the steps I have done

ubuntu@ravishing-pinniped:~$ cat compose.yml 
services:
  test_db:
    image: alpine:3.21.2
    volumes:
      - /tmp/postgres:/var/lib/postgresql/data:rshared
    command: >
      sh -c "echo test > /var/lib/postgresql/data/testfile.txt"
ubuntu@ravishing-pinniped:~$ ls /tmp/postgres
ls: cannot access '/tmp/postgres': No such file or directory
ubuntu@ravishing-pinniped:~$ docker compose up
[+] Running 1/1
 ✔ Container ubuntu-test_db-1  Created                                                                                                                               0.0s 
Attaching to test_db-1
test_db-1 exited with code 0
ubuntu@ravishing-pinniped:~$ ls /tmp/postgres
testfile.txt
ubuntu@ravishing-pinniped:~$ docker compose version
Docker Compose version v2.32.3
ubuntu@ravishing-pinniped:~$ 

Can you check if this isn't a permission issue by running your command using sudo?

[ndeloof]

Can't reproduce either

$ cat compose.yaml
services:
  test_db:
    image: alpine
    volumes:
      - type: bind
        source: /tmp/postgres
        target: /var/lib/postgresql/data
        bind:
          create_host_path: true
          propagation: rshared

$ docker compose version
Docker Compose version v2.32.2

$ docker compose up
[+] Running 1/0
 ✔ Container truc-test_db-1  Created                                                                                                                                   0.0s 
Attaching to test_db-1

[hs-siso]

Sorry for the delayed response.

It doesn't look like a permission issue to me (updated):

~ % cat test-coposefile.yaml
services:
  test_db:
    image: alpine:3.21.2
    volumes:
      - /tmp/postgres:/var/lib/postgresql/data:rshared
    command: >
      sh -c "echo test > /var/lib/postgresql/data/testfile.txt"

~ % wget -O docker-compose-linux-x86_64 https://github.com/docker/compose/releases/download/v2.32.4/docker-compose-linux-x86_64
--2025-01-28 06:58:33--  https://github.com/docker/compose/releases/download/v2.32.4/docker-compose-linux-x86_64
Resolving github.com (github.com)... 140.82.112.3
Connecting to github.com (github.com)|140.82.112.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/15045751/f3a985c2-d572-4b10-b678-943fc714dc56?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20250128%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250128T065833Z&X-Amz-Expires=300&X-Amz-Signature=f15ff5978d4d3ee9391a961b35b989c7d02e0a17a99b034222a716e9a518f965&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Ddocker-compose-linux-x86_64&response-content-type=application%2Foctet-stream [following]
--2025-01-28 06:58:34--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/15045751/f3a985c2-d572-4b10-b678-943fc714dc56?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20250128%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250128T065833Z&X-Amz-Expires=300&X-Amz-Signature=f15ff5978d4d3ee9391a961b35b989c7d02e0a17a99b034222a716e9a518f965&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Ddocker-compose-linux-x86_64&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.109.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 64694701 (62M) [application/octet-stream]
Saving to: ‘docker-compose-linux-x86_64’

docker-compose-linux-x86_64                    100%[===================================================================================================>]  61.70M   327MB/s    in 0.2s

2025-01-28 06:58:34 (327 MB/s) - ‘docker-compose-linux-x86_64’ saved [64694701/64694701]

~ % file docker-compose-linux-x86_64
docker-compose-linux-x86_64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=K7BPsaK8GSzyMTCSHf3B/DxBc5ReQqS1nCywyjZVt/Vsvzo_oxHkYrLvcgwTbg/1l0C77R-6jQCS0LWxDcX, not stripped

~ % sudo chmod 777 docker-compose-linux-x86_64
~ % ls -l docker-compose-linux-x86_64
-rwxrwxrwx 1 root root 64694701 Jan 15 17:36 docker-compose-linux-x86_64

% ./docker-compose-linux-x86_64 --version
Docker Compose version v2.32.4

~ % ls -ld /tmp
drwxrwxrwt 27 root root 28672 Jan 28 06:51 /tmp

% ./docker-compose-linux-x86_64 -f test-coposefile.yaml up
[+] Running 0/1
 ⠋ Container stoimenhristov-test_db-1  Creating                                                                                                                                       0.0s
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/postgres

~ % sudo ./docker-compose-linux-x86_64 -f test-coposefile.yaml up
[+] Running 0/1
 ⠋ Container stoimenhristov-test_db-1  Creating                                                                                                                                       0.0s
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/postgres

It's worth noting that I download and install docker-compose from here

Any other thoughts/ideas?

[ndeloof]

Can you please try to reproduce with a plain docker run command ?

$ docker run -v /tmp/postgres:/var/lib/postgresql/data:rshared alpine:3.21.2 ls -al /var/lib/postgresql
total 8
drwxr-xr-x    3 root     root          4096 Jan 28 07:28 .
drwxr-xr-x    1 root     root          4096 Jan 28 07:28 ..
drwxr-xr-x    2 root     root            64 Jan 28 07:28 data
```

[hs-siso]

docker run -v /tmp/postgres:/var/lib/postgresql/data:rshared alpine:3.21.2 ls -al /var/lib/postgresql

This worked just fine:

% docker run -v /tmp/postgres:/var/lib/postgresql/data:rshared alpine:3.21.2 ls -al /var/lib/postgresql
total 12
drwxr-xr-x    3 root     root          4096 Jan 28 07:36 .
drwxr-xr-x    1 root     root          4096 Jan 28 07:36 ..
drwxr-xr-x    2 root     root          4096 Jan 28 07:36 data

% ls -ld /tmp/postgres
drwxr-xr-x 2 root root 4096 Jan 28 07:36 /tmp/postgres

% docker ps -a | grep alpine
CONTAINER ID   IMAGE                                                                                                                COMMAND                  CREATED          STATUS                      PORTS     NAMES
4297a58e4fed   alpine:3.21.2                                                                                                        "ls -al /var/lib/pos…"   39 seconds ago   Exited (0) 39 seconds ago             vigorous_heisenberg

[hs-siso]

For what is worth, I am on an Ubuntu 22 system:

~ % cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04.3 LTS"

I downloaded docker-compose from the official releases page like so:

wget -O docker-compose-linux-x86_64 https://github.com/docker/compose/releases/download/v2.32.4/docker-compose-linux-x86_64

And then moved the binary to /usr/bin:

mv docker-compose-linux-x86_64 /usr/bin/docker-compose
chmod a+x /usr/bin/docker-compose

Making it executable by anybody on the system:

% which docker-compose
/usr/bin/docker-compose

% docker-compose --version
Docker Compose version v2.32.4

[hs-siso]

Latest update from me.

I just ran the same test on a very vanilla Ubuntu22:

• with this test compose file:

services:
  test_db:
    image: alpine:3.21.2
    volumes:
      - /tmp/postgres:/var/lib/postgresql/data:rshared
    command: >
      sh -c "echo test > /var/lib/postgresql/data/testfile.txt"

• Installed docker from sources - version 24 (described in here)
• Ran docker compose and things failed the same way as reported above:

ubuntu@ip-10-16-64-57:~$ docker --version
Docker version 24.0.9, build 2936816
ubuntu@ip-10-16-64-57:~$
ubuntu@ip-10-16-64-57:~$ sudo docker compose -f test-composefile.yaml up
[+] Running 1/1
 ✔ Container ubuntu-test_db-1  Created                                                                                                                                                0.0s
Attaching to test_db-1
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/postgres
ubuntu@ip-10-16-64-57:~$

---

• I uninstalled docker v24 and install latest (v27)
• /tmp does not contain a postgres dir

ubuntu@ip-10-16-64-57:~$ docker --version
Docker version 27.5.1, build 9f9e405
which in return creates the container (when the host dir /tmp/postgres is missing it creates it too)
ubuntu@ip-10-16-64-57:~$ sudo docker compose -f test-composefile.yaml up
[+] Running 2/2
 ✔ test_db Pulled                                                                                                                                                                     1.1s
   ✔ 1f3e46996e29 Pull complete                                                                                                                                                       0.6s
[+] Running 2/2
 ✔ Network ubuntu_default      Created                                                                                                                                                0.1s
 ✔ Container ubuntu-test_db-1  Created                                                                                                                                                0.1s
Attaching to test_db-1
test_db-1 exited with code 0

• To extend this test I even downloaded the latest docker-compose binary and with a clean /tmp I re-ran up which passed again OK:

ubuntu@ip-10-16-64-57:~$ sudo ./docker-compose-linux-x86_64 -f test-composefile.yaml up
[+] Running 1/1
 ✔ Container ubuntu-test_db-1  Created                                                                                                                                                0.1s
Attaching to test_db-1
test_db-1 exited with code 0

ubuntu@ip-10-16-64-57:~$ sudo docker --version
Docker version 27.5.1, build 9f9e405

ubuntu@ip-10-16-64-57:~$ ./docker-compose-linux-x86_64 --version
Docker Compose version v2.32.4

It is the older docker version (v24) being the culprit here.

[ndeloof]

nice catch.
v24 being EoL, I guess we can close this issue as "not planned"