ostree backend: Fix duplicate objects without /ostree in container

[jlebon]

Edited by @cgwalters

Goal: Ensure that when a container image doesn't have /ostree we don't end up with duplicate ostree objects (one labeled in a default way and one with the final label).

Mitigations:

• payload-link
• Change the commit process to be two phase: save layers but do it in way that we avoid duplicate objects
• Unified storage #20

Hard constraint: Must not break rpm-ostree (or ostree) when used directly

Original:

---

We've now moved to client-side SELinux labeling. I think we should still though support server-side labeling, I guess through ostree container commit/bootc build commit? I think this today though conflicts with wanting to move away from /ostree in the container image, but it could be implemented differently of course.

The main argument is simply reproducibility. Notably, coreos/fedora-coreos-tracker#2030 happened which is a great example of why doing this server-side would be better for those that want to opt in (like FCOS/RHCOS).

[cgwalters]

This also came up in ostreedev/ostree#3523 (comment)

I think this today though conflicts with wanting to move away from /ostree in the container image, but it could be implemented differently of course.

Yes though the other obvious implementation is to just support having security.selinux xattrs in the tar stream. This would need some validation that it doesn't break/conflict with container runtimes though.

[jlebon]

Yes OK, definitely quite surprising I think that even just encapsulated commits still get relabeled (coreos/fedora-coreos-tracker#2030 (comment)).

And in the new buildah path, while we do generate an OSTree commit as part of build-chunked-oci, I tend not to think of it that way and I'm not sure if it's where relabeling should happen. E.g. what I initially had in mind for this issue is something applicable to both FROM scratch flows and simple derivations.

[cgwalters]

We had a discussion about this one today and I guess it is a soft blocker for dropping /ostree from the container for the ostree backend.

I edited the top issue here.

[jlebon]

And in turn this is a blocker for moving over to chunkah which doesn't understand /ostree (nor wants to :) ).

Change the commit process to be two phase: save layers but do it in way that we avoid duplicate objects

Yeah, I think that's probably the cleanest. And it feels less risky than turning on payload links. I had an idea of e.g. ostree container commit setting user.selinux xattrs, but we don't want to make that mandatory.

[cgwalters]

Change the commit process to be two phase: save layers but do it in way that we avoid duplicate objects

I think the simplest way to do this is: Always use the "ambient" (currently booted) policy when writing layers, then loop back and relabel the fetched layers and recommit. This will result in some objects that get GC'd where there's drift, but I don't think that's a big deal.

It's messy because that will result in "recommit cycles" in the case where two images that have a distinct label for a file in a shared layer, and if the user switches between these images we will "flap" the layer commits too. But I dunno I guess it doesn't matter.

[jlebon]

Why not just:

• have no labels on imported layers
• checkout merge commit and recommit with base layer

?

The recommit should get reflinks, and the layer hashes remain canonical/don't represent multiple on-disk states.