docker now uses containerd, and /var/lib/docker has moved to userdata, so why hasn't /var/lib/containerd moved too?

[samarium]

/var/lib/docker has moved to /mnt/deitpi_userdata/docker_data, presumably to move userdata off root file system.
Since docker now uses containerd which also stores images in /var/lib/containerd which is also bulk userdata, it seems that /var/lib/containerd should be moved to /mnt/dietpi_userdata/containerd_data, and /etc/containerd/config.toml adjusted with root=/mnt/dietpi_userdata/containerd-data.

I guess that dietpi_software needs to be hacked appropriately too.

[time passes ...]

moving /var/lib/containerd and updating config.toml worked

Looking at dietpi-software it already does config injection on config.toml for a different reason, so no reason why it should not do the relocation at installation like it does the /var/lib/docker relocation.

Currently not sure what happens when I relocate dietpi_userdata and I have already moved /var/lib/containerd to userdata.

dietpi-drive_manager does a dietpi-services stop which stops docker, but not containerd, so needs to know the dependency if moving containerd to userdata. I'll need to manually stop dietpi-servers and then systemctl stop containerd. Would have been simpler if waited to relocated containerd until after I had relocated userdata.

Also seems like dietpi-software doesn't handle relocated userdata, as lots of hardcoded /mnt/dietpi-userdata paths.
... Or it does by symlinking the default location to the new location :-)

Checking sizes shows containerd has the bulk of the userdata for me, which I want out of root, but then I use bind mounts for the majority of my docker volumes for my data to allow easier backups.

root@DietPi:/# du -hs /mnt/dietpi_userdata/*-data
505M	/mnt/dietpi_userdata/containerd-data
236K	/mnt/dietpi_userdata/docker-data

Maybe dietpi-services needs to know the docker is dependent on containerd and they should be stop/started in sync. Could probably do something in systemd, but maybe too complicated, and just have dietpi-services know and handle the dependency would be better.

[MichaIng]

Docker now uses /var/lib/containerd by default for images? That would be new to me. While it uses containerd since a long time, it still used /var/lib/docker (or in our case /mnt/dietpi_userdata/docker-data) for images pulled with docker pull or docker run, if not manually configured otherwise. However, while /var/lib/containerd may contain the images (which can be re-downloaded any time), the Docker data still contains the volumes created with docker volume create foo. Having images and the default volume path separated would even better match our typical /mnt/dietpi_userdata usage.

/mnt/dietpi_userdata is primarily meant to contain userdata and configs one might want to backup separately, move to a different/new system, cannot restore easily etc, which does not apply to Docker images. What benefit do you expect from having container images in /mnt/dietpi_userdata resp. outside of the rootfs?

Hardcoded /mnt/dietpi-userdata paths are not an issue: When moving it to a different drive via dietpi-drive_manager, a symlink is left behind to keep /mnt/dietpi-userdata paths intact.

AFAIK dietpi-services not controlling containerd was an oversight at first. But IIRC running containers can stall in a bad state if containerd is stopped for longer. The Docker daemon itself is only used for managing containers, creating new ones, starting/stopping them etc, but is not needed by running containers. But containerd ... yeah, I did a quick search and containers stop functioning and even data may be lost when stopping containerd without stopping containers first, so that is not a good idea and the way we handle it right now is correct.

[samarium]

I think you should look at where things are stored after pulling a few images, as from what I can see the image data is managed by containerd. I'm not sure where things split out, but for my example above I have containerd data 500M+, docker data < 1M. IIRC I measured while services and containers were stopped. I agree that previously all the data was in docker data, and I have been isolating that from rootfs for years. This is my first exposure to containerd and I was surprised to find all the data locked up in the container images, and not much in docker data.

You don't see true numbers when containers are active. I guess overlayfs or bind mounts from containerd to docker. See below for the major chunks of data in each of docker and containerd in various states of activity.

FYI, I'm using dietpi trixie 9.20+ and installed docker using dietpi-software.

# docker stop dns-server;  docker stop portainer_agent
# systemctl stop docker.socket docker containerd
# du -hs /mnt/dietpi_userdata/*-data/*{overlayfs,rootfs}
364M	/mnt/dietpi_userdata/containerd-data/io.containerd.snapshotter.v1.overlayfs
0	/mnt/dietpi_userdata/docker-data/rootfs

# systemctl start containerd docker docker.socket
# du -hs /mnt/dietpi_userdata/*-data/*{overlayfs,rootfs}
364M	/mnt/dietpi_userdata/containerd-data/io.containerd.snapshotter.v1.overlayfs
0	/mnt/dietpi_userdata/docker-data/rootfs

# docker start portainer_agent; docker start dns-server
# du -hs /mnt/dietpi_userdata/*-data/*{overlayfs,rootfs}
364M	/mnt/dietpi_userdata/containerd-data/io.containerd.snapshotter.v1.overlayfs
355M	/mnt/dietpi_userdata/docker-data/rootfs

Again if you are moving docker-data off rootfs to isolate ot from rootfs and to make it easier to backup seperately, then conainterd-data seems to be integral, and you can't meaningfully backup one without the other. I can snapshot a btrfs or lvm userdata and then make a stable backup even while containers are running, no worse that a system crash/reboot. I would prefer to avoid rootfs as I usually minimise the backups of that as it a relatively easily rebuilt with your dietpi automated install and a little other config data like from /etc and my normal ansible system setup scripts.

I have a small rootfs, and try to seperate most userdata from the system, and to stop userdata potentially filling rootfs, so it is on another disk, just like I would do with kvm/qemu images and other large data hog apps that aren't really part of the base system, and I like how you have seperated things like docker, syncthing, samba etc into userdata.