vc_render_tifxyz: rendering at an absent pyramid level produces an all-black image with exit 0 — reachable via the sparse volumes the Las-volume flow now attaches

[Bullo27]

Summary

Since #1302/#1310, volumes whose pyramid is sparse (e.g. only level 2 exists — the shape a lasagna prediction zarr has after the Las-volume attach path resolves foo.ome.zarr/2 to its root) are first-class Volumes: test_lasagna_project_volumes.cpp ("leaves scale handling to native volume loading") explicitly blesses a volume with hasScaleLevel(0) == false and only level 2 present.

vc_render_tifxyz however still accepts any --group-idx and, when the requested level is absent, renders a fully black image and exits 0 — no warning, no error. The same segment against the same volume at a present level renders correctly, so the black output looks like "the prediction/scan has nothing here" rather than "you asked for a level that does not exist". For presence/fiber prediction volumes this is a silently wrong scientific conclusion, not a cosmetic defect.

Reproduction (self-contained, no data downloads)

The script builds a scene with one bright sheet (value 220 on a dim background), writes it as

• dense.zarr — normal pyramid, levels 0/1/2, and
• sparse.zarr — only level 2 present (base shape 128×192×160, so levels 0/1 are absent, exactly like the fixture in test_lasagna_project_volumes.cpp),

plus a tifxyz segment lying exactly on the sheet, then renders the 2×2 matrix:

#!/usr/bin/env python3
# deps: numpy, tifffile.  Set VC_BIN to your vc_render_tifxyz build.
import json, subprocess, numpy as np, tifffile
from pathlib import Path

BIN = "VC_BIN"; ROOT = Path("./repro"); BASE = (128, 192, 160)  # z,y,x

def sheet_z(y, x):
    return BASE[0]/2 + 10.0*np.sin(x/25.0) + 6.0*np.cos(y/30.0)

def scene(shape, scale):
    z, y, x = np.meshgrid(*(np.arange(s) for s in shape), indexing="ij")
    vol = np.full(shape, 20, np.float32)
    vol[np.abs(z*scale - sheet_z(y*scale, x*scale)) <= 4.0] = 220.0
    return vol.astype(np.uint8)

def write_v2_array(path, arr, chunks):
    path.mkdir(parents=True, exist_ok=True)
    (path/".zarray").write_text(json.dumps({"zarr_format":2,"shape":list(arr.shape),
        "chunks":list(chunks),"dtype":"|u1","compressor":None,"fill_value":0,
        "order":"C","filters":None}))
    cz, cy, cx = chunks
    for iz in range(-(-arr.shape[0]//cz)):
        for iy in range(-(-arr.shape[1]//cy)):
            for ix in range(-(-arr.shape[2]//cx)):
                blk = np.zeros(chunks, np.uint8)
                sl = arr[iz*cz:(iz+1)*cz, iy*cy:(iy+1)*cy, ix*cx:(ix+1)*cx]
                blk[:sl.shape[0], :sl.shape[1], :sl.shape[2]] = sl
                (path/f"{iz}.{iy}.{ix}").write_bytes(blk.tobytes())

def down2(a):
    a = a[:a.shape[0]//2*2, :a.shape[1]//2*2, :a.shape[2]//2*2].astype(np.float32)
    return np.round(a.reshape(a.shape[0]//2,2,a.shape[1]//2,2,a.shape[2]//2,2)
                    .mean(axis=(1,3,5))).astype(np.uint8)

l0 = scene(BASE, 1); l1 = down2(l0); l2 = down2(l1)
write_v2_array(ROOT/"sparse.zarr"/"2", l2, (16,16,16))          # ONLY level 2
(ROOT/"sparse.zarr"/".zgroup").write_text('{"zarr_format": 2}')
for lvl, arr, ch in ((0,l0,64),(1,l1,32),(2,l2,16)):
    write_v2_array(ROOT/"dense.zarr"/str(lvl), arr, (ch,)*3)     # control
(ROOT/"dense.zarr"/".zgroup").write_text('{"zarr_format": 2}')

gy, gx = np.meshgrid(np.arange(20,172,2.0), np.arange(20,140,2.0), indexing="ij")
seg = ROOT/"segment"; seg.mkdir(exist_ok=True)
tifffile.imwrite(seg/"x.tif", gx.astype(np.float32))
tifffile.imwrite(seg/"y.tif", gy.astype(np.float32))
tifffile.imwrite(seg/"z.tif", sheet_z(gy, gx).astype(np.float32))
(seg/"meta.json").write_text(json.dumps({"type":"seg","uuid":"sheet-demo",
    "name":"sheet-demo","format":"tifxyz","scale":[0.5,0.5]}))

for vol in ("dense","sparse"):
    for g in (0,2):
        out = ROOT/f"render_{vol}_g{g}"
        r = subprocess.run([BIN,"-v",str(ROOT/f"{vol}.zarr"),"-s",str(seg),
            "--tif-output",str(out),"--scale","1","-g",str(g)],
            capture_output=True, text=True)
        img = tifffile.imread(sorted(out.parent.glob(out.name+"*/**/*.tif"))[0])
        print(f"{vol:6s} g={g}: exit={r.returncode} min={img.min()} "
              f"max={img.max()} mean={img.mean():.1f} "
              f"nonzero={np.count_nonzero(img)}/{img.size}")

Measured on current main (8b7c9df, Ubuntu 24.04, gcc Release build):

volume	--group-idx	exit	output
dense	0	0	correct render of the sheet (mean 207.1, max 220)
dense	2	0	correct level-2 render (mean 179.1)
sparse	0 (absent)	0	all-black: min=max=0, 0/18240 nonzero, no warning
sparse	2	0	correct — identical stats to dense g=2 (mean 179.1)

The sparse volume renders perfectly at its present level, so nothing is wrong with the data — only the absent-level request fails, and it fails silently.

Root cause

The renderer samples through Chunked3d → Volume::readZYX → readFromChunkedArrayZYX (core/src/Volume.cpp:270), with level = group_idx. Two independent mechanisms make an absent level indistinguishable from empty data:

1. ChunkCache::shape(level) (core/src/render/ChunkCache.cpp:278) returns the {0,0,0} placeholder that prepareLasagnaProjectVolumes-style sparse pyramids store for absent levels, so every sample is treated as out-of-bounds and the read buffer keeps its zero fill.
2. Even where a chunk is actually requested, an absent level has a null fetcher and tryGetChunk/getChunkBlocking return ChunkStatus::Missing (ChunkCache.cpp:303-315, 370-380). Missing is also the legitimate status for an absent chunk file inside a present level (ZarrChunkFetcher.cpp:173,187), which consumers correctly map to fill — so the else → allFill branch in readFromChunkedArrayZYX (Volume.cpp:~385) silently zero-fills absent levels too.

In other words: at the ChunkCache API, "this chunk is fill value" and "this whole level does not exist" produce the same observable result, and the CLI never checks Volume::hasScaleLevel(group_idx) up front.

Notably the interactive viewer already treats missing levels as a first-class condition (scale fallback, #926; ChunkedPlaneSampler handles ChunkStatus::Missing explicitly) — it is the batch tools that predate sparse pyramids.

Reach

• The agent-bridge render.tifxyz method defaults groupIdx to 0 (apps/VC3D/agent_bridge/AgentBridgeHandlers_seeding.cpp:583), so vc3d_render_tifxyz via MCP against an attached Las/lasagna volume hits the black-render path with default arguments.
• The GUI render dialog forwards the user-chosen group index to vc_render_tifxyz with no presence check (SegmentationCommandHandler.cpp:2057 → CommandLineToolRunner.cpp:655). I verified this in the code, not by clicking through the dialog.
• The documented render workflow uses --group-idx 0 (docs/07_tutorial5.md), which is exactly the level a scaledown-style prediction zarr does not have.

Secondary observation (same underlying gap)

vc_grow_seg_from_seed on the same sparse volume reads the seed value as 0 through the absent level 0 (and prints zarr dataset size for scale group 0 [128, 192, 160] and chunk shape [1, 1, 1], both describing a level that does not exist) before aborting with the unrelated-sounding segment growth failed: Volume::shape level is not present. It fails loudly only because the growth pipeline happens to call Volume::shape() later; the reads themselves are silent zeros.

Suggested fix

Minimal and local: after opening the volume, both CLIs should validate the requested level and fail fast with an actionable message, e.g.

Error: --group-idx 0 refers to a scale level that is not present in this volume
       (present levels: 2). Sparse volumes (e.g. lasagna predictions) only
       contain their scaledown level.

Volume::hasScaleLevel() / presentScaleLevels() already exist, so this is a few lines per tool. I have this fix building and passing the suite locally and will submit it as a PR shortly.

A deeper option — giving "absent level" a distinct ChunkStatus (or asserting on it in readFromChunkedArrayZYX) so no consumer can silently zero-fill an absent level — touches shared semantics with the viewer's designed fallback, so I'd leave that call to you.

---

Found by driving the newly merged Las-volume/sparse-pyramid support end-to-end against synthetic fixtures. Analysis, reproduction and write-up were done with AI assistance (Claude); all numbers above are from actual local runs on 8b7c9df.