Docker for Mac: Portainer Stack does not work

[jnehlmeier]

Bug description
Using Docker Desktop for Mac 4.16.2 (95914) on MacOS Ventura 13.1 the provided docker compose file from your documentation does not work correctly. Not sure if it is relevant but Docker for Mac is configured to use the MacOS Virtualization Framework so that VirtioFS can be used as file sharing implementation for containers (Settings -> General).

I can not access any of the published ports (8000, 9000, 9443) and after some time portainer shuts down.

Expected behavior
Portainer should start up correctly and be accessible via browser

Portainer Logs (fresh install with empty volume)

portainer_portainer.1.t805tqryyxiz@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
portainer_portainer.1.t805tqryyxiz@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
portainer_portainer.1.t805tqryyxiz@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
portainer_portainer.1.t805tqryyxiz@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/internal/ssl/ssl.go:80 > no cert files found, generating self signed SSL certificates |
portainer_portainer.1.t805tqryyxiz@docker-desktop    | 2023/02/13 04:31PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:673 > failed initializing environment | error="Get \"https://tasks.agent:9001/_ping\": dial tcp: lookup tasks.agent on 127.0.0.11:53: no such host"
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:534 > encryption key file not present | filename=portainer
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:553 > proceeding without encryption key |
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/datastore/backup.go:109 > creating DB backup |
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/datastore/backup.go:45 > copying DB file | from=/data/portainer.db to=/data/backups/common/portainer.db.2.17.0.20230213163121
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/database/boltdb/db.go:124 > loading PortainerDB | filename=portainer.db
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/datastore/migrate_data.go:108 > migrating database from version 2.17.0 to 2.17.0 |
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/datastore/migrator/migrate_dbversion80.go:69 > transfer type field to details field for edge stack status |
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/datastore/migrator/migrate_dbversion80.go:27 > updating existing endpoints to not detect metrics API for existing endpoints (k8s) |
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/datastore/migrator/migrate_dbversion80.go:48 > updating existing endpoints to not detect metrics API for existing endpoints (k8s) |
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/datastore/migrator/migrate_ce.go:79 > db migrated to 2.17.0 |
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 16:31:21 server: Reverse tunnelling enabled
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 16:31:21 server: Fingerprint 4f:40:be:88:b7:f1:97:32:03:a7:74:38:c6:e1:77:0e
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 16:31:21 server: Listening on 0.0.0.0:8000...
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM WRN github.com/portainer/portainer/api/platform/platform.go:67 > failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/cmd/portainer/main.go:810 > starting Portainer | build_number=27403 go_version=1.19.4 image_tag=linux-amd64-2.17.0 nodejs_version=18.14.0 version=2.17.0 webpack_version=5.68.0 yarn_version=1.22.19
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/http/server.go:342 > starting HTTPS server | bind_address=:9443
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:31PM INF github.com/portainer/portainer/api/http/server.go:327 > starting HTTP server | bind_address=:9000
portainer_portainer.1.8r02exb2qb7c@docker-desktop    | 2023/02/13 04:36PM INF github.com/portainer/portainer/api/adminmonitor/admin_monitor.go:61 > the Portainer instance timed out for security purposes, to re-enable your Portainer instance, you will need to restart Portainer |

Most notably is the line

failed to retrieve docker info | error="Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

I have inspected the LinuxKit VM that Docker for Mac uses internally using docker run -it --rm --privileged --pid=host justincormack/nsenter1. It seems like /var/run/docker.sock does indeed not exist inside that VM.

Steps to reproduce the issue:

1. Install Docker Desktop for Mac 4.16.2 (95914) (or newer) on MacOS Ventura 13.1 (or newer)
2. Might not be needed: Open Docker Desktop for Mac settings, go to General and make sure VirtioFS is selected (which should automatically select use Virtualization framework).
3. Initialize Docker swarm using docker swarm init
4. curl -L https://downloads.portainer.io/ce2-17/portainer-agent-stack.yml -o portainer-agent-stack.yml
5. docker stack deploy -c portainer-agent-stack.yml portainer

Technical details:

• Portainer version: 2.17
• Docker version (managed by Portainer): 20.10.22
• Kubernetes version (managed by Portainer): v1.25.4
• Platform (windows/linux): macOS Ventura 13.1
• Command used to start Portainer (docker run -p 9443:9443 portainer/portainer): see above
• Browser: Any
• Have you reviewed our technical documentation and knowledge base? Yes

[jnehlmeier]

Installing Portainer 2.17.0 using the Docker for Mac extension seems to work though.

[tamarahenson]

@jnehlmeier

I wanted to follow up on this request. You are wanting to install Docker Swarm on Mac? And then install Portainer on the single node Swarm cluster. Why do you want Swarm instead of Standalone here?

Thanks!

[jnehlmeier]

@tamarahenson

Yes, I am always installing Docker for Mac and then activate Docker swarm on it. Then I use a mixture of normal containers and swarm services depending on the use case.

I try hard to mimic production environments during development and thus I usually deploy portainer as swarm services because I do so as well in production. Also I found it easier to update portainer if it is running as swarm service. For this I usually modify your provided docker swarm compose file and replace the fixed version with latest before deploying portainer the first time. Then I just have to force update the swarm service using the new portainer image. With a standalone container I have to stop it, remove it, recreate it with all the configuration options I have used when I first created it.

This has always worked for me until recently.

[jnehlmeier]

@jamescarppe Any reason why this has been moved to discussion? Technically it is a regression compared to earlier versions of portainer (or possibly earlier versions of Docker, in case Docker caused the issue)

[jamescarppe]

We're moving to using discussions for questions and feature requests going forward and keeping issues for bug reports exclusively. This lets us gauge community feedback easier and facilitates wider discussion on ideas and implementation.

In this case, since the issue was erroneously flagged as both kind/bug and kind/question, it was moved into a discussion when it should have stayed a bug. I'll sort that out.

Edit: Created a new issue based on this discussion: #9945