Open Data catalog Lasagna attach always fails: prepareOpenDataLasagna validates before writing the lasagna-remote.json marker that validation needs to resolve remote channels

[Bullo27]

Summary

Opening a catalog sample with Lasagna resources currently attaches zero Lasagna datasets, deterministically, for every remote artifact. The prepare step has a circular dependency:

prepareOpenDataLasagna (apps/VC3D/OpenDataLasagna.cpp:437) on a fresh cache:

1. stages the manifest via cacheRemoteFile (line ~480) — this writes the RemoteFileCache sidecar, not the Lasagna marker;
2. builds the lasagna-remote.json marker object in memory (lines ~482-496);
3. calls validatePrepared(info, cached.path) at line 497 — before the marker is written at line 498.

Inside validation, LasagnaDataset::open → loadRemoteMarker (core/src/lasagna/Dataset.cpp:472) looks for lasagna-remote.json next to the staged manifest. It does not exist yet, so the manifest stays fully local, resolveGroupLocations resolves the groups' relative zarr keys against the cache directory (which holds only the 848-byte manifest), and validateGroupDescriptor → openLasagnaChannelArray fails with:

Failed to prepare Lasagna for 20251211183505 at coordinate level L2: zarr: cannot detect version at:
~/.VC3D/remote_cache/open_data/lasagna/PHerc0332/20251211183505/<hash>/PHerc0332-20251211183505-lasagna-20260724_cos.ome.zarr/3
(no .zarray or zarr.json found)

The exception is caught at line 501, so the marker is never written, and the warm-cache path (line 448) requires the marker — every retry repeats identically. Validation needs the marker; the marker is only written after validation passes.

Reproduction (measured on main 8b7c9df, via the agent bridge)

catalog.open_sample {"sampleId": "PHerc0332", "resources": {"kinds": ["lasagna"], "volumeIds": ["20251211183505"]}}

• Run 1 (cold cache): job succeeds overall, attached: {"lasagnaDatasets": 0, ...}, with the message above. Cache afterwards contains only …lasagna.json (848 B) + its .vc-remote-file.json sidecar — no channel data, no lasagna-remote.json.
• Run 2 (warm cache): identical failure — deterministic.
• Control that isolates the ordering as the only defect: hand-writing the correct lasagna-remote.json (the exact object the code builds at lines 482-496) into the cache dir and re-running the same open yields "Attached 1 manifest-backed Lasagna dataset(s)", lasagnaDatasets: 1, and all four channel volumes (cos, grad_mag, nx, ny) attached as remote volumes. Everything downstream works once the marker exists before validation.

The parent CT volume attaches fine either way (including the rebased #vc-base-scale=2 source view), so the sample open looks successful; the Lasagna failure is one line inside the job's message list.

Scope

• The validate-before-marker shape was introduced in Fiber 3d tracing #1294 (Aug 1) as far as git log -S shows, so the catalog Lasagna path has plausibly been failing for all remote artifacts since then. The catalog currently lists Lasagna prediction artifacts for 11+ scrolls (see Remote zarr open binds OME multiscale datasets by array index, not path — scaledown artifacts (tiled_predict3d first_level>0) get silently wrong geometry over HTTP, correct locally #1346 for the survey), so this blocks all of them in the Open Data flow. Manual/local Lasagna attach paths (LasagnaDataset::openLocation) are unaffected.
• Related but separate: once this ordering is fixed, the attached channel volumes open through the remote pyramid reader, which currently misregisters these scaledown artifacts by 2^level (Remote zarr open binds OME multiscale datasets by array index, not path — scaledown artifacts (tiled_predict3d first_level>0) get silently wrong geometry over HTTP, correct locally #1346, fix in fix(render): bind remote multiscale datasets by numeric path, not array index #1347). My verification above ran with fix(render): bind remote multiscale datasets by numeric path, not array index #1347 applied locally; on plain main, fixing only the ordering would make the attach "succeed" with silently wrong geometry. The two fixes are complementary.

Suggested fix

Write the marker before validating (move lines 498-499 above 497). Failure semantics are preserved: if validation then fails, the marker correctly describes the staged artifact and the warm path re-validates on every open, so a failing artifact keeps failing loudly with the real reason (now about actual remote channels instead of an empty local directory). I have this change building and verified end-to-end on a fresh sample and will PR it shortly.

---

Found by driving the catalog Open Data flow headlessly through the agent bridge against the live catalog. All transcripts are from actual runs; the marker hand-write control is what isolates the ordering as the root cause. Analysis and write-up done with AI assistance (Claude).