macOS bind mount permission issues: libkrun (❌) vs. applehv (✅ )

[jreades]

Background

Having migrated over from Docker to Podman this year, I have been teaching reproducible computing/Python to students on a mix of Macs and PCs. They tend to set up their working iPython notebooks on their host machines in all sorts of different places so we've been using -v $(pwd):/home/jovyan/work to connect the host directory to the JupyterLab container. The image is built on top of quay.io/jupyter/minimal-notebook:notebook-7.4.4 and the full build file is here though that's only for completeness as I don't think it's relevant.

Bindmount Issues

My own migration went seamlessly (well, until later), but about 25% of students using the macOS were unable to use bindmounts unless they set the permissions on the host directory and all subfolders to 777 (and 666 for all files) which is hardly desirable. I couldn't reproduce this until I rebuilt my VM this morning at which point I had the exact same problem.

I went down all sorts of paths and the only approach that seemed to have any effect was adding --userns=keep-id:gid=20 to the run command as that adds the jovyan user to a group with the same GID as the staff group in macOS. Adding uid=501 to remap the user has no effect (for fairly obvious reasons, I assume). This at least changed the permissions required for write access from 777 to 770/775 for folders and 666 to 664/0 so that was an improvement.

Workaround

However, I suddenly realised that there were two ways to configure the machine: with Apple's hypervisor or with the newer default Krunkit. I rebuilt the machine for the nth time but this time forcing it to use applehv (export CONTAINERS_MACHINE_PROVIDER="applehv"). Suddenly bindmounts worked again with zero permissions issues.

I've now tested this by initialising a second VM with the same specs but using the newer libKrun default and have confirmed that this VM does not allow me to successfully mount host directories that are writeable/accessible as if by the user running podman.

I don't believe it's anything specific to my image, but just in case here's the command that works with hypervisor and not with libKrun:

podman run --rm -d -p 8888:8888 \
  -v $(pwd):/home/jovyan/work \
  jreades/sds:2025-arm start.sh jupyter lab \
     --NotebookApp.notebook_dir='/home/jovyan/work' \
     --ServerApp.password='' \
     --ServerApp.token=''

Krunkit Validation

As a final step I did also try to ensure that it wasn't a missing library by following an apparent suggestion to:

brew tap slp/krunkit
brew install krunkit

That didn't fix anything so I've now brew uninstalled it.

Am I missing something? I will confess that the container <-> VM <-> host UID/GID mapping element is really at the limit of my understanding but I do feel like I chased down every viable lead for solving this with libKrun and kept having to come back to using Apple Hypervisor as the only way forward.

Suggestions gratefully received. This discussion builds on an exchange on BlueSky which was really helpful but a little constrained for exploring the issue since I couldn't give a full background or really understand the options.

Relevant Hardware/Software Information

Mac:

MBP 14" 2023
M2 Max
macOS Sequoia 15.7.2 (was waiting until end of term to upgrade)

Podman:

podman version 5.7.0

[jreades]

Should I have listed this as a bug? I'm unsure whether it's something I've not understood, or something that is actually tied to how Podman is/is not running on OSX.

[DeMaBo]

Because I have the same issue, and before investigating more into libkrun, I wanted to ask if you already opened a bug for this?

I also enable the applehv provider on podman (~/.config/containers/containers.conf).

[containers]

[engine]
env = []

[machine]
#provider = "libkrun" # disabled for testing
provider = "applehv"

[network]

[secrets]

[configmaps]

Afterwards I could access and configure files inside the container without permission issues, so there is something odd with the permissions while using the libkrun.

[jreades]

I haven’t as I didn’t know if it was a bug or something that I was overlooking in my configuration. It seemed unlikely that I was the only person to have this problem but I couldn’t find any other posts on the topic. — Jon Reades ***@***.***

…

On 30 Dec 2025 at 19:25 +0000, Dennis Borchardt ***@***.***>, wrote: Because I have the same issue, and before investigating more into libkrun, I wanted to ask if you already opened a bug for this? I also enable the applehv provider on podman (~/.config/containers/containers.conf). [containers] [engine] env = [] [machine] #provider = "libkrun" # disabled for testing provider = "applehv" [network] [secrets] [configmaps] Afterwards I could access and configure files inside the container without permission issues, so there is something odd with the permissions while using the libkrun. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[jakecorrenti]

This doesn't seem familiar to me at all, could be a regression. I'll dive into it today.

cc @tylerfanelli @slp just in case they've seen something that I missed

[slp]

We know macOS virtiofs implementation is very lax when it comes to file permissions, while libkrun's virtiofs is closer to the correct (and more strict) semantics. There's a chance this could break some existing workflows that rely on such relaxed behavior.

We need more details to know exactly if that's the case or if there's actually a problem somewhere.

[DeMaBo]

I will add more details to my setup, which is actually quite simple, but maybe it helps to reproduce the error.

Working environment:

MBP 16" 2024
M4 Pro
macOS 26.2
Podman Desktop v1.24.2
Podman 5.7.1

Running the Podman machine as rootless in libkrun 

Fedora CoreOS 43.20251110.3.1

podman machine info
host:
    arch: arm64
    currentmachine: ""
    defaultmachine: ""
    eventsdir: /run/user/501/podman
    machineconfigdir: /var/home/core/.config/containers/podman/machine/qemu
    machineimagedir: /var/home/core/.local/share/containers/podman/machine/qemu
    machinestate: ""
    numberofmachines: 0
    os: linux
    vmtype: qemu
version:
    apiversion: 5.7.1
    version: 5.7.1
    goversion: go1.25.4 X:nodwarf5
    gitcommit: f845d14e941889ba4c071f35233d09b29d363c75
    builttime: Mon Dec  8 21:00:00 2025
    built: 1765238400
    buildorigin: 'Copr: packit/containers-podman-27732'
    osarch: linux/arm64
    os: linux

This is my directory tree of my little project which I started to learn golang:

ls -lah -R ./
total 0
drwxr-xr-x@ 5 debo  staff   160B  1 Jan. 21:19 .
drwxr-xr-x  5 debo  staff   160B 16 Dez. 15:03 ..
drwxr-xr-x@ 4 debo  staff   128B  9 Jan. 11:07 .devcontainer
drwxr-xr-x@ 3 debo  staff    96B 16 Dez. 15:04 .vscode
drwxr-xr-x@ 4 debo  staff   128B  1 Jan. 21:19 src

./.devcontainer:
total 16
drwxr-xr-x@ 4 debo  staff   128B  9 Jan. 11:07 .
drwxr-xr-x@ 5 debo  staff   160B  1 Jan. 21:19 ..
-rw-r--r--@ 1 debo  staff   373B  9 Jan. 11:08 devcontainer.json
-rw-r--r--@ 1 debo  staff   140B  5 Jan. 21:59 Dockerfile

./.vscode:
total 8
drwxr-xr-x@ 3 debo  staff    96B 16 Dez. 15:04 .
drwxr-xr-x@ 5 debo  staff   160B  1 Jan. 21:19 ..
-rw-r--r--@ 1 debo  staff    46B 16 Dez. 15:04 settings.json

./src:
total 16
drwxr-xr-x@ 4 debo  staff   128B  1 Jan. 21:19 .
drwxr-xr-x@ 5 debo  staff   160B  1 Jan. 21:19 ..
-rwxr-xr-x@ 1 debo  staff    15B  9 Jan. 11:08 main.go
-rw-------@ 1 debo  staff     1B  1 Jan. 21:18 test.go

#devcontainer.json
{
  "image": "mcr.microsoft.com/devcontainers/go:2.0.6-bullseye",
  
  "customizations": {
        "vscode": {
            "settings": {
            },
            "extensions": [
            ]
        }
    },

  "updateRemoteUserUID": true,  
  "remoteUser": "vscode",
  "runArgs": [
	      "--cap-add=SYS_PTRACE",
        "--security-opt",
        "seccomp=unconfined"
  ],
}

As soon as I start my devcontainer to build with libkrun engine using the podman-machine, I cannot create any file, this is the error which I get:

vscode ➜ /workspaces/aoc-2025 $ ls -lah 
total 0
drwxr-xr-x. 4 root nogroup 128 Jan  9 14:07 .devcontainer
drwxr-xr-x. 4 root nogroup 128 Jan  2 00:19 src
drwxr-xr-x. 3 root nogroup  96 Dec 16 18:04 .vscode

vscode ➜ /workspaces/aoc-2025 $ touch main2.go
touch: cannot touch 'main2.go': Permission denied

the same is with adding content to a existing file:

Failed to save 'test.go': Unable to write file 'vscode-remote://dev-container+7b22686f737450617468223a222f55736572732f6465626f2f436f64652f416476656e744f66436f64652f616f632d32303235222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c22667350617468223a222f55736572732f6465626f2f436f64652f416476656e744f66436f64652f616f632d323032352f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2265787465726e616c223a2266696c653a2f2f2f55736572732f6465626f2f436f64652f416476656e744f66436f64652f616f632d323032352f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2270617468223a222f55736572732f6465626f2f436f64652f416476656e744f66436f64652f616f632d323032352f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a2266696c65227d7d/workspaces/aoc-2025/src/test.go' (NoPermissions (FileSystemError): Error: EACCES: permission denied, open '/workspaces/aoc-2025/src/test.go')

Maybe there is also something mandatory which I just overlook, Thank you for your help.

EDIT: Okay I found for my case this related issues #18691 #13814

[jreades]

could you provide the init command you used?

Apologies, I've been snowed under with marking and academic stuff. I (personally) was using a fairly vanilla init command:

podman machine init --cpus 4 --memory 8192 --rootful test

And

podman machine init --cpus 4 --memory 8192 --rootful --provider applehv test

Along the way I tried many other variations (some of which were detailed in my original post), including with/without rootful, with/without a mount point specified at init, and so on. None of those worked for me in terms of resolving the ability to write to the host HD, the only thing that did was specifying applehv as the provider instead of libkrun.

There is one other thing that I may not have checked (I don't recall one way or the other) which has literally only just occurred to me: the image I was trying to run is built with docker compatibility. I don't have time to test this right now (need to survive the coming week) but will try to run a 'native' open image under libkrun instead to see if I still have the same issue.

HTH.

[jreades]

We need more details to know exactly if that's the case or if there's actually a problem somewhere.

What more would help you to narrow this down? Happy to provide additional reports from podman, but my own setup and init line seemed fairly straightforward and not tied to relying on a lax security setup... I thought. :-P

[slp]

@jreades I suspect this issue should have been fixed by this commit containers/libkrun@0c1ddbd

I'll propose updating krunkit on podman.

[l0rd]

I still see the problem with krunkit 1.2.1. A simple reproducer:

mkdir -p ~/tmp/foo
podman run -ti --rm \
    -v ~/tmp/foo:/foo/ \
   --security-opt label=disable --userns=keep-id \
   --user podman \
   quay.io/podman/stable:latest \
   touch /foo/bar

This command works fine with AppleHV (relaxed file permissions), but on Fedora with SELinux enabled (not relaxed), too.

[l0rd]

$ brew list --versions | grep libkrun
libkrun 1.17.4
libkrunfw 5.3.0

[jreades]

OK, so if I run brew install krunkit then running the above command (brew list --versions...) gives me:

libkrun-efi 1.16.0

The output of /opt/homebrew/bin/krunkit --version is

krunkit 1.1.1

I get the same version now running /opt/podman/bin/krunkit --version. I can't seem to find 1.2.1 in Homebrew listings so am presumably missing something.

[slp]

I get the same version now running /opt/podman/bin/krunkit --version. I can't seem to find 1.2.1 in Homebrew listings so am presumably missing something.

You need to switch from the old to the new tap, PTAL here https://github.com/containers/krunkit?tab=readme-ov-file#installation

[jreades]

OK, so I'm now showing that the version of krunkit in /opt/homebrew/bin is 1.2.1 but my attempt to 'apply' this update in Podman using either a symlink (move the 'old' libkrun out of the way and sudo ln -s /opt/homebrew/bin/krunkit /opt/podman/bin/krunkit) or direct copy (sudo cp /opt/homebrew/bin/krunkit /opt/podman/bin/krunkit) both failed in the sense that I can no longer start up a Podman machine that uses libkrun at all.

I had a suspicion it might be a code signing issue so did a little googling and found this thread which prompted me to try signing and setting xattrs but that has not improved things.

For reference:

1. Grabbed the entitlements file from: https://github.com/containers/krunkit/blob/main/krunkit.entitlements
2. Ran sudo codesign --deep --entitlements ~/Downloads/krunkit.entitlements --sign - /opt/podman/bin/krunkit
3. Ran sudo xattr -r -d com.apple.quarantine /opt/podman/bin/krunkit

In the Desktop the error is reported as:

Error: Command execution failed with exit code 125 Command execution failed with exit code 125 Error: krunkit exited unexpectedly with exit code 1

I can still see in the extended attributes that the new version and old version differ slightly (the latter has com.apple.provenance set, but am wondering if there's a simpler way to resolve this?

[slp]

OK, so I'm now showing that the version of krunkit in /opt/homebrew/bin is 1.2.1 but my attempt to 'apply' this update in Podman using either a symlink (move the 'old' libkrun out of the way and sudo ln -s /opt/homebrew/bin/krunkit /opt/podman/bin/krunkit) or direct copy (sudo cp /opt/homebrew/bin/krunkit /opt/podman/bin/krunkit) both failed in the sense that I can no longer start up a Podman machine that uses libkrun at all.

I had a suspicion it might be a code signing issue so did a little googling and found this thread which prompted me to try signing and setting xattrs but that has not improved things.

For reference:

1. Grabbed the entitlements file from: https://github.com/containers/krunkit/blob/main/krunkit.entitlements
2. Ran sudo codesign --deep --entitlements ~/Downloads/krunkit.entitlements --sign - /opt/podman/bin/krunkit
3. Ran sudo xattr -r -d com.apple.quarantine /opt/podman/bin/krunkit

In the Desktop the error is reported as:

Error: Command execution failed with exit code 125 Command execution failed with exit code 125 Error: krunkit exited unexpectedly with exit code 1

I can still see in the extended attributes that the new version and old version differ slightly (the latter has com.apple.provenance set, but am wondering if there's a simpler way to resolve this?

I think it'd be easier to clone upstream podman, and then doing something like this:

cd contrib/pkginstaller/
make ARCH=aarch64 NO_CODESIGN=1 pkginstaller
open out/podman-installer-macos-arm64.pkg