Remote snapshotter in podman

[siscia]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description Remote snapshotter in podman

We are really interested in the possibility to use a remote snapshotter like the one provide in containerd also in podman.

A remote snapshotter is a piece a containerd plugin which is provided as input the name of the layer that containerd need and it either mounts the correct directory or return an error.

All this process is manage by "user-level" code/plugin.

Is something like this even possible in podman_

[baude]

care to contribute?

[siscia]

Maybe I can find the time myself or maybe we can find some resource to have somebody working on it.
It is something interesting for podman? If I had a PR ready, it would be merged?
I haven't really find the time to explore the codebase, who is responsible for this particular part of the codebase?

[AkihiroSuda]

cc @ktock

[mheon]

I somewhat suspect this world be part of the containers/storage library.

…

On Sun, Dec 22, 2019, 13:52 Akihiro Suda ***@***.***> wrote: cc @ktock <https://github.com/ktock> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4739>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3AOCFP33XCXPZAK73N4LTQZ6ZNFANCNFSM4J6JEESQ> .

[ktock]

Thanks a lot for opening it! I'm keen to contribute to it.

I'm currently working on the implementation of remote snapshotter plugin which enables us to mount layers without pulling the actual contents. We can plug any filesystem into it so we currently support CRFS's stargz-based filesystem and we discuss to support CernVM-FS and other filesystems.
https://github.com/ktock/remote-snapshotter

Remote snapshotter currently supports containerd's snapshotter API but I think it's not hard to support graphdriver API as well.

I'll look deep into the codebase.

@siscia How do you think about this implementation strategy?

[ktock]

I opened the discussion on containers/storage#498 .

[giuseppe]

low level bits are being worked in https://github.com/giuseppe/crfs-plugin and fuse-overlayfs.

The rest of the implementation, as @mheon said, should go into containers/storage

Other remote file systems can be added in a similar way to crfs-plugin that can be used with fuse-overlayfs to lookup files from the image/lower layers

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@giuseppe Any more progress on this?

[siscia]

I am still trying to figure out what is the best approach for this.

I was expecting the storage to be layer-digest based, like each layer was indexed by its own hash.
Something like:

- storage
|- 123...
|- abc...
|- ....

Where 123... and abc... where the hash of the layer itself.

This does not seems to be the case.

Unfortunately I still haven't understood what hash is used to index the layer.

However this is quite a complication, a read-only remote snapshotter need to know how podman is looking for layer.

In my understanding this is managed by containers/image codebase that I am exploring now. However it is a big codebase and it is taking time.

Any feedback or help is very well appreciated.

I may be missing something in my analysis as well!

[rhatdan]

@siscia Are you still working on this?

[siscia]

Honestly we were planning to propose this as GSoC project.

Progress on this front are a little slow at the moment since the whole program didn't start yet.

Regarding this project I am busy with the bureaucracy from our side (mostly sorted out) and coming up with a good test for possible students.