Fix multi-platform soci standalone convert

[prafgup]

Issue #, if available:

Fixes - #1948

Description of changes:

The standalone layout loader parsed only index.json.manifests[0], so any input whose top-level index listed multiple per-platform manifests directly (e.g. from go-containerregistry's layout.Write) lost every platform after the first silently when using --all-platforms, or with a misleading "manifest not found" when using --platform.

Replace the single-element parse with resolveLayoutRoot, which handles all three shapes index.json (single image, nested manifest list, flat per-platform list), filters out children whose blobs are missing, and promotes the top-level index to a content-addressable blob when it is itself the manifest list.

Running the reproduction steps here now gives correct results - #1948

SOCI_BIN=/Users/prafulg/projects/prafulg/soci-snapshotter/out/soci ./reproduce.sh


=== INPUT platforms ===
{
  "platform": {
    "architecture": "arm64",
    "os": "linux"
  },
  "digest": "sha256:7094b474ca994e198cf92414e0c552a315eb48237053292756425c1f24e819ce"
}
{
  "platform": {
    "architecture": "amd64",
    "os": "linux"
  },
  "digest": "sha256:965a4be6edf9b4658f854d042d21f92fc4158871547fc769be7a1150cc677ad8"
}
{
  "platform": {
    "architecture": "arm",
    "os": "linux",
    "variant": "v7"
  },
  "digest": "sha256:2381361da9d2c687488804c23e41b3aaf99101609d6d985b9c4b8f26eb5517f6"
}

=== convert --all-platforms ===
layer sha256:fee30871cd8cc89f6c3e6ed53bede46c1f08bb1d8051cd6384517d756099d454 -> ztoc sha256:34d0ce07a2d20dc49ac1459cf1448d2a2dafc1f9660f4d149ac44b6475333cdd
layer sha256:552ed30d0b0dfa9ae77aab6a08c2e1dbc8b3cef6243e9c9fc9715f566b08f202 -> ztoc sha256:4396be51bf34bb3d58192c7a4adb355585e4fbe770c3248949d4b235a9b76122
layer sha256:9fa8a8506df39d5a95b9b2e12bd982f79f31a71e8b238b757fc04e3b53e06673 -> ztoc sha256:8c5a21917a338c59324b0c66d1f5bdadf3342a5a6a76c7309da9cbe4ed9bb869
(exit 0)
{
  "platform": {
    "architecture": "arm64",
    "os": "linux"
  },
  "artifactType": null
}
{
  "platform": {
    "architecture": "amd64",
    "os": "linux"
  },
  "artifactType": null
}
{
  "platform": {
    "architecture": "arm",
    "os": "linux",
    "variant": "v7"
  },
  "artifactType": null
}
{
  "platform": {
    "architecture": "arm64",
    "os": "linux"
  },
  "artifactType": "application/vnd.amazon.soci.index.v2+json"
}
{
  "platform": {
    "architecture": "amd64",
    "os": "linux"
  },
  "artifactType": "application/vnd.amazon.soci.index.v2+json"
}
{
  "platform": {
    "architecture": "arm",
    "os": "linux",
    "variant": "v7"
  },
  "artifactType": "application/vnd.amazon.soci.index.v2+json"
}

=== convert --platform linux/amd64 (not first) ===
layer sha256:552ed30d0b0dfa9ae77aab6a08c2e1dbc8b3cef6243e9c9fc9715f566b08f202 -> ztoc sha256:4396be51bf34bb3d58192c7a4adb355585e4fbe770c3248949d4b235a9b76122
(exit 0)
{
  "platform": {
    "architecture": "arm64",
    "os": "linux"
  },
  "artifactType": null
}
{
  "platform": {
    "architecture": "amd64",
    "os": "linux"
  },
  "artifactType": null
}
{
  "platform": {
    "architecture": "arm",
    "os": "linux",
    "variant": "v7"
  },
  "artifactType": null
}
{
  "platform": {
    "architecture": "amd64",
    "os": "linux"
  },
  "artifactType": "application/vnd.amazon.soci.index.v2+json"
}

Testing performed:

Added Integration test for multiple platform image.

lima sudo GO_TEST_FLAGS="-run TestStandalone -count=1" make integration 2>&1 | tail -25)
  ⎿  === RUN   TestStandaloneConvertBasic/tar-to-tar
     --- PASS: TestStandaloneConvertBasic/tar-to-tar (0.22s)
     === RUN   TestStandaloneConvertBasic/dir-to-dir
     --- PASS: TestStandaloneConvertBasic/dir-to-dir (0.28s)
     === RUN   TestStandaloneConvertBasic/tar-to-dir
     --- PASS: TestStandaloneConvertBasic/tar-to-dir (0.27s)
     --- PASS: TestStandaloneConvertBasic (24.86s)
     === RUN   TestStandaloneConvertSpecificPlatform
     --- PASS: TestStandaloneConvertSpecificPlatform (21.11s)
     === RUN   TestStandaloneInvalidConversion
     === RUN   TestStandaloneInvalidConversion/nonexistent_input
     --- PASS: TestStandaloneInvalidConversion/nonexistent_input (0.03s)
     === RUN   TestStandaloneInvalidConversion/invalid_format
     --- PASS: TestStandaloneInvalidConversion/invalid_format (0.03s)
     === RUN   TestStandaloneInvalidConversion/missing_destination
     --- PASS: TestStandaloneInvalidConversion/missing_destination (0.03s)
     --- PASS: TestStandaloneInvalidConversion (1.58s)
     === RUN   TestStandaloneConvertIdempotent
     --- PASS: TestStandaloneConvertIdempotent (20.65s)
     === RUN   TestStandaloneConvertAllPlatforms
     --- PASS: TestStandaloneConvertAllPlatforms (38.93s)
     PASS
     ok         github.com/awslabs/soci-snapshotter/integration 200.444s

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[prafgup]

Hey @sondavidb @Shubhranshu153 @coderbirju can you take a look at this fix 😄 .
(Also would appreciate a minor release if possible so we can consume it directly)

[prafgup]

@KlwntSingh replied to your comments 😄
Also the the new integration test uses nerdctl to pull and verify all-platforms, previously this integration test would have failed, while other integration tests are untouched in the standalone path which I added previously as a part of #1881

[prafgup]

Pinging @sondavidb regrading - #1943 (comment)

[prafgup]

Seems like one of the checks failed 🤔 "Pre-build / check (pull_request)Pre-build / check (pull_request)"

Might be due to me not having Signed-off-by: Praful Gupta <prafulgupta6@gmail.com> in the commit 🤔 can add that once the build check succeeds.

[sondavidb]

Will look at the test a little later

[prafgup]

Addressed comments :) and added the signed-off in the commit.

[sondavidb]

Thanks for the fix!

[prafgup]

Hey maintainers 🙂! Can someone else review/merge this (#1947) and #1943 (already approved by @\sondavidb) , and potentially release a minor version this with the critical vulnerability fix (already merged) - #1953 , would really help in consuming this package from releases on our end.

(cc.ing a couple devs to take a look as well) @KlwntSingh @Swapnanil-Gupta @coderbirju

[Swapnanil-Gupta]

Thanks.

[prafgup]

Resolved comments :) for merge compatibility !

Thanks for the review !