Avoid invalidating all caches on mid-build-failure recovery

Previously, repository_metadata_needs_sync() returned True whenever any
Cacheonly=auto image lacked a cache, which caused run_clean() to wipe
every image cache to avoid partial-upgrade scenarios. As a result, if
mkosi was building several images for the first time and one of them
failed mid-way, the next run would re-sync metadata, wipe the caches of
the images that had already succeeded, and rebuild everything from
scratch.

Use the manifest file as a "previously cached" marker — it's written
when an image builds successfully and survives even after the cache
contents go stale — and only re-sync metadata (and invalidate all
caches) when every incremental image was previously cached and at least
one of them is now out of date. In the mid-build-failure case, the image
that failed never wrote a manifest, so we don't trigger the re-sync and
the surviving caches are preserved.

For all-non-incremental setups we keep the previous behavior of always
syncing — there are no caches to preserve and every build downloads
packages. In a mixed setup, the non-incremental images are ignored by
the heuristic and may eventually fail because of stale metadata, but
that's the cost of mixing them with incremental images we want to
preserve.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: daandemeyer

mkosi/__init__.py

@@ -4798,14 +4798,38 @@ def ensure_directories_exist(config: Config) -> None:
 
 
 def repository_metadata_needs_sync(images: Sequence[Config]) -> bool:
-    for config in images:
-        if config.cacheonly == Cacheonly.never:
-            return True
+    if any(c.cacheonly == Cacheonly.never for c in images):
+        return True
 
-        if config.cacheonly == Cacheonly.auto and not have_cache(config):
-            return True
+    if not (auto := [c for c in images if c.cacheonly == Cacheonly.auto]):
+        return False
 
-    return False
+    # If there are no incremental images at all, we have no caches to preserve, so
+    # always sync to avoid failures from stale metadata. In a mixed setup, ignore
+    # the non-incremental images and apply the heuristic to the incremental ones only.
+    # Non-incremental images in a mixed setup may eventually fail because of stale
+    # metadata, but that's the cost of mixing them with incremental images we want
+    # to preserve.
+    if not (incremental := [c for c in auto if c.is_incremental()]):
+        return True
+
+    # Use the manifest file as the "previously cached" marker: it's written when an
+    # image builds successfully and survives even after the image cache content goes
+    # stale. If no incremental image has a manifest, no image has ever been built
+    # before, so there's nothing to be consistent with and we have to sync to get
+    # any metadata at all. If every incremental image has a manifest and at least
+    # one of those caches is now out of date, the user is incrementally rebuilding
+    # from a fully-coherent state, so re-sync metadata and let run_clean() wipe
+    # everything to avoid partial upgrades. Otherwise, some images have manifests
+    # and some don't (e.g. recovering from a mid-build failure), and the existing
+    # metadata is by construction consistent with the caches that do exist, so we
+    # don't need to re-sync.
+    if all(not cache_tree_paths(c)[2].exists() for c in incremental):
+        return True
+
+    return all(cache_tree_paths(c)[2].exists() for c in incremental) and not all(
+        have_cache(c) for c in incremental
+    )
 
 
 def sync_repository_metadata(

mkosi/resources/man/mkosi.1.md

@@ -1599,9 +1599,12 @@ boolean argument: either `1`, `yes`, or `true` to enable, or `0`, `no`,
     as the package cache is already fully populated. If set to `metadata`,
     the package manager can still download packages, but we won't sync the
     repository metadata. If set to `auto`, the repository metadata is
-    synced unless all images are cached (see `Incremental=`) and packages can
-    be downloaded during the build. If set to `never`, repository metadata
-    is always synced and packages can be downloaded during the build.
+    synced when no image has ever been built before, or (and all cached
+    images are invalidated in this case) when every image was previously
+    cached (see `Incremental=`) and at least one of those caches is now
+    out of date. Packages can be downloaded during the build. If set to
+    `never`, repository metadata is always synced and packages can be
+    downloaded during the build.
 
 `SandboxTrees=`, `--sandbox-tree=`
 :   Takes a comma-separated list of colon-separated path pairs. The first