Permission errors when running in rootless mode

[mariusor]

Issue Description

With latest podman release on an archlinux CI runner I keep getting permission denied errors on various files/directories. (see podman info example below)

I don't think this is a packaging error as the changes in the PKGBUILD from v1.43 -> v1.44 contains just the version update and associated metadata.

Steps to reproduce the issue

Steps to reproduce the issue

1. on a fresh archlinux CI image from builds.sr.ht
2. install buildah
3. run podman info

Describe the results you received

Permission errors.

Describe the results you expected

Successful command execution.

buildah version output

$ buildah version
Version:         1.44.0
Go Version:      go1.26.3-X:nodwarf5
Image Spec:      1.1.1
Runtime Spec:    1.3.0
image Version:   5.40.0
Git Commit:      30a4189415e5d0a3a0c5971307a6432d8ed1a097
Built:           Thu May 28 08:33:02 2026
OS/Arch:         linux/amd64
BuildPlatform:   linux/amd64

buildah info output

$ buildah info
Error: open /var/lib/containers/storage/overlay-containers/volatile-containers.json: permission denied
WARN[0000] failed to shutdown storage: "open /var/lib/containers/storage/overlay-containers/volatile-containers.json: permission denied" 
$ ls -lF /var/lib/containers/storage/overlay-containers/volatile-containers.json
-rw------- 1 root root 1825 May 29 10:14 /var/lib/containers/storage/overlay-containers/volatile-containers.json

Provide your storage.conf

# $ grep -v "^#" /etc/containers/storage.conf 
[storage]
driver = "overlay"
runroot = "/run/containers/storage"
graphroot = "/var/lib/containers/storage"
[storage.options]
additionalimagestores = [
]
[storage.options.pull_options]
[storage.options.overlay]
mountopt = "nodev"

Upstream Latest Release

Yes

Additional environment details

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

[nalind]

Yes, that picked up a reworking of how the storage configuration is read in our dependencies - it's picking up the location that's set in the global storage.conf file, when the newer version of the storage library expects it to be separated between drop-in files read by root and drop-in files read by non-root users if it's specified at all (locations listed here). A newer version of the package that provides the configuration should include suitable corresponding changes.

[mariusor]

suitable corresponding changes

Where should I look what these suitable changes are to make v1.44 compatible with v1.43 ?

Until the package maintainer solves this, I'd like to make them manually for my CI pipeline.

[nalind]

My understanding is that it's essentially moving or removing the "graphroot" setting, as "rootless_storage_path" is no longer consulted in its place for non-root users. Values other than the default, for root, would go into a similarly-named file in /etc/containers/storage.rootful.conf.d/ on an end-system, or /usr/share/containers/storage.rootful.conf.d/ in a package. Non-root users would look in their own .config/containers/storage.conf, /etc/containers/storage.rootless.conf.d, /usr/share/containers/storage.rootless.conf.d, and lastly the global storage.conf.

In practice, I've been prepending env CONTAINERS_STORAGE_CONF=/dev/null to a number of commands to override where the global settings are read from, as none of the other files are on my development system, so there's nothing in them that overrides the default, either.

[nalind]

Oh, but to actually answer your question, it follows from the descriptions of the "graphroot" and "rootless_storage_path" settings in the updated man page.

[mariusor]

So, I think this can be closed.

As long as the project doesn't offer defaults for the /usr/share/containers/ configuration tree and relies on packagers to create them, this is an error generated through a packaging issue, and I'm talking with the Archlinux maintainer to update the default configuration files.

Please let me know if there's any more information I can provide to help with these kind of issues in the future.

Thank you.

[doctorcolossus]

I experienced the same issues, also on Arch, with the same version migration.

Simply creating ~/.config/containers/storage.conf with the following contents seems to have resolved the issues for me:

[storage]
driver = "overlay"

[nalind]

Alright then, closing.

[Nephyrin]

So, I think this can be closed.

As long as the project doesn't offer defaults for the /usr/share/containers/ configuration tree and relies on packagers to create them, this is an error generated through a packaging issue, and I'm talking with the Archlinux maintainer to update the default configuration files.

Please let me know if there's any more information I can provide to help with these kind of issues in the future.

Thank you.

It's worth noting that podman handles this case fine and picks appropriate rootless defaults -- if a version upgrade here broke working versions until every packager echos hopefully-the-same-defaults into a file that does seem worth addressing

[nalind]

It's worth noting that podman handles this case fine and picks appropriate rootless defaults -- if a version upgrade here broke working versions until every packager echos hopefully-the-same-defaults into a file that does seem worth addressing

Which version of podman are you referring to? I expect the upcoming 6.x, which will use the same updated versions of the dependencies that are used by both buildah and podman, will end up behaving similarly.

[Nephyrin]

I was indeed testing 5.8.2. It would be nice if packagers didn't need to follow config format changes (e.g. ship defaults/templates as mentioned above), but I admittedly don't understand all the nuances here.

[viperML]

I've also run into this problem in NixOS. The distribution ships the following /etc/containers/storage.conf:

[storage]
driver = "overlay"
graphroot = "/var/lib/containers/storage"
runroot = "/run/containers/storage"

My impression is that both graphroot and runroot should be either:

• Unset at /etc/containers/storage.conf, such that it is derived automatically which one to use.
• Set at ~/.config/containers/storage.conf, explicitely for the user's path.

Is this right?