Support 32-bit BMFF item IDs for iloc v2

Add bounded 32-bit item-id insertion for BMFF targets that use iloc v2 while preserving 16-bit behavior for iloc v0/1. Expand metadata_transfer to use 32-bit item_id storage/allocation, emit infe v3 when wider IDs are needed, and select iref version/id-widths appropriately (with safe failure when ID space is exhausted). Fix container scanning to read iinf version 1 entry counts as 32-bit. Update iloc/infe/iref read/write paths to honor input id-widths and avoid truncation, and adjust casts where needed. Add unit tests exercising iloc v2 targets with high item IDs and update multiple docs (CHANGES, development, metadata backend/plan, writer target contract) to document the new behavior and limits.

Author: ssh4net

docs/development.md

@@ -321,11 +321,14 @@ Current v1 behavior is:
       merge, replace, or strip bounded Exif/XMP/JUMBF/C2PA items by extending
       `iinf`, `iloc`, `idat`, and `iref` with `cdsc` references to the primary
       item. This path requires a single parseable `iinf`, `iloc` version
-      0/1/2, `pitm`, and at most one `idat`. Bounded ICC transfer removes
-      prior ICC `colr/prof` and `colr/rICC` properties from `iprp/ipco`,
-      compacts/remaps existing `ipma` associations, appends the transferred
-      `colr/prof` property, and associates it with the primary item.
-      Arbitrary BMFF scene/property graph rewrite remains unsupported.
+      0/1/2, `pitm`, and at most one `idat`. Inserted item IDs follow the
+      existing `iloc` item-id width: `iloc` version 0/1 stays 16-bit, while
+      `iloc` version 2 can use 32-bit item IDs and emits wider `infe`/`iref`
+      records when needed. Bounded ICC transfer removes prior ICC `colr/prof`
+      and `colr/rICC` properties from `iprp/ipco`, compacts/remaps existing
+      `ipma` associations, appends the transferred `colr/prof` property, and
+      associates it with the primary item. Arbitrary BMFF scene/property graph
+      rewrite remains unsupported.
     - CLI/Python `metatransfer` wrappers expose both BMFF summaries and this
       bounded edit path; `--target-heif`, `--target-avif`, and `--target-cr3`
       now accept `--source-meta PATH` plus `--output PATH` for metadata

docs/metadata_backend_matrix.md

@@ -153,6 +153,9 @@ For the public per-target preserve/replace guarantees, see
   Exif/XMP/JUMBF/C2PA metadata items by extending `iinf`, `iloc`, `idat`, and
   `iref` with `cdsc` references to the primary item. It requires a single
   parseable `iinf`, `iloc` version 0/1/2, `pitm`, and at most one `idat`.
+  Inserted item IDs follow the existing `iloc` item-id width: `iloc` version 0/1
+  remains a 16-bit insertion path, while `iloc` version 2 can use 32-bit item IDs
+  and emits wider `infe`/`iref` records when needed.
 - Foreign-`meta` ICC property merge is bounded to
   `bmff:property-colr-icc`. OpenMeta removes prior ICC `colr/prof` and
   `colr/rICC` properties from `iprp/ipco`, compacts/remaps existing `ipma`

docs/metadata_transfer_plan.md

@@ -55,7 +55,7 @@ The first public write-side sync controls are also in place:
 | WebP | Bounded but real | Prepared bundle, compiled emit, bounded chunk rewrite/edit, file-helper roundtrip | Not a general WebP chunk editor |
 | JP2 | Bounded but real | Prepared bundle, compiled emit, bounded box rewrite/edit, file-helper roundtrip | `jp2h` synthesis is still out of scope |
 | JXL | Bounded but real | Prepared bundle, compiled emit, bounded box rewrite/edit, file-helper roundtrip | Still narrower than JPEG/TIFF |
-| HEIF / AVIF / CR3 | Bounded but real | Prepared bundle, compiled emit, OpenMeta-managed BMFF item/property edit, constrained foreign-`meta` item merge/replacement/strip plus bounded ICC property merge, file-helper roundtrip | Not broad BMFF writer parity; arbitrary foreign `meta` scene/property-graph rewrite is still unsupported |
+| HEIF / AVIF / CR3 | Bounded but real | Prepared bundle, compiled emit, OpenMeta-managed BMFF item/property edit, constrained foreign-`meta` item merge/replacement/strip plus bounded ICC property merge, file-helper roundtrip | Not broad BMFF writer parity; arbitrary foreign `meta` scene/property-graph rewrite is still unsupported; 32-bit inserted item IDs require existing `iloc` v2 |
 | EXR | Bounded but real | Prepared bundle, compiled emit, direct backend attribute emit, prepared-bundle to `ExrAdapterBatch` bridge, CLI/Python transfer surface | No file rewrite/edit path yet; current transfer payload is safe string attributes only |
 
 ## What Is Already Implemented
@@ -304,6 +304,9 @@ Implemented as a bounded BMFF target family:
 - constrained foreign top-level `meta` item merge for parseable `iinf`,
   `iloc` version 0/1/2, `pitm`, optional single `idat`, and primary-item `cdsc`
   references
+- bounded 32-bit item-id insertion for foreign item graphs that already use
+  `iloc` version 2; `iloc` version 0/1 targets remain constrained to 16-bit
+  inserted item IDs
 - bounded foreign top-level `meta` ICC property merge by replacing prior ICC
   `colr/prof` and `colr/rICC` properties, remapping `ipma`, and associating the
   transferred `colr/prof` property with the primary item
@@ -785,7 +788,13 @@ parity across every workflow.
 - [x] lock explicit unmanaged-metadata preservation rules for unrelated chunks, boxes, items, and tails per target family
 - [x] add compare-backed read-back gates for each first-class target instead of relying mainly on API-shape regression coverage
 - [x] make CLI and Python surfaces describe the same writeback behavior and path-derivation rules as the C++ helper
-- [ ] reduce remaining target differences to documented limits instead of accidental implementation details
+- [x] reduce remaining target differences to documented limits instead of accidental implementation details
+
+Evidence: `docs/writer_target_contract.md` defines per-target preserve/replace
+rules and the remaining bounded limits. The BMFF section now makes the item-id
+width rule explicit: `iloc` version 0/1 insertion remains 16-bit, while `iloc`
+version 2 can use 32-bit item IDs and fails safely when the graph shape or ID
+space is outside that contract.
 
 #### 2. Bounded EXIF / IPTC / XMP Sync Layer
 
@@ -810,7 +819,13 @@ embedded/sidecar writeback across the primary writer target family.
 - [x] cover embedded-only, sidecar-only, and dual-write XMP flows in release-facing compare validation
 - [x] add compare-backed validation for explicit sidecar-base overrides and destination-sidecar cleanup behavior
 - [x] gate the primary writer family on deterministic read-back of managed metadata after edit/apply
-- [ ] keep public parity claims tied to compare-backed evidence instead of only unit or smoke coverage
+- [x] keep public parity claims tied to compare-backed evidence instead of only unit or smoke coverage
+
+Evidence: this plan keeps parity language at the supported-lane level and ties
+release claims to `openmeta_gate_transfer_release`, compatibility-dump
+read-back, and image-usability gates. BMFF high-item-ID insertion is covered by
+a release-gated API roundtrip that writes into an `iloc` v2 graph and scans the
+result back as one Exif item and one XMP item.
 
 #### 4. TIFF / DNG Deeper Rewrite Guarantees
 
@@ -830,9 +845,9 @@ embedded-versus-sidecar precedence, and `DNGVersion` read-back.
 
 #### Done-When Readout
 
-- [ ] the first-class target family has one explicit public writer contract
-- [ ] the bounded sync-policy layer is documented and regression-gated
-- [ ] release-facing compare validation covers the main still-image writer set
+- [x] the first-class target family has one explicit public writer contract
+- [x] the bounded sync-policy layer is documented and regression-gated
+- [x] release-facing compare validation covers the main still-image writer set
 - [x] `TIFF/DNG` rewrite guarantees are strong enough to stop being a primary parity blocker
 - [x] the next work slice can move to bounded `BMFF` depth instead of still backfilling the writer baseline
 

docs/sphinx/writer_target_contract.rst

@@ -111,7 +111,8 @@ Target Summary
        in parseable foreign top-level ``meta`` graphs by extending
        ``iinf``/``iloc``/``idat``/``iref``; replace prior ICC ``colr``
        properties and remap ``iprp``/``ipco``/``ipma`` for bounded ICC.
-     - Does not rewrite arbitrary BMFF scene/property graphs.
+     - Does not rewrite arbitrary BMFF scene/property graphs; 32-bit item-id
+       insertion requires existing ``iloc`` v2.
    * - ``EXR``
      - safe string header attributes through the EXR transfer emitter or
        adapter batch
@@ -258,12 +259,18 @@ item graphs it merges, replaces, or strips bounded Exif/XMP/JUMBF/C2PA
 metadata items in the existing ``meta`` by extending ``iinf``, ``iloc``,
 ``idat``, and ``iref`` with ``cdsc`` references to the primary item. This
 constrained merge requires a single parseable ``iinf``, ``iloc`` version
-0/1/2, ``pitm``, and at most one ``idat``. For bounded ICC transfer,
-OpenMeta removes prior ICC ``colr/prof`` and ``colr/rICC`` properties from
-``iprp/ipco``, compacts/remaps existing ``ipma`` associations, appends the
-transferred ``colr/prof`` property, and associates it with the primary item.
-Arbitrary non-ICC property replacement and broader BMFF scene/property graph
-rewriting remain out of scope.
+0/1/2, ``pitm``, and at most one ``idat``. For ``iloc`` version 0/1 targets,
+inserted item IDs remain in the 16-bit item-id space. For ``iloc`` version 2
+targets, OpenMeta can allocate 32-bit item IDs and uses ``infe`` version 3 plus
+``iref`` version 1 when a newly inserted item needs that wider ID space. If the
+existing graph has exhausted the usable item-id space or mixes item-table widths
+outside this shape, the edit fails instead of truncating IDs.
+
+For bounded ICC transfer, OpenMeta removes prior ICC ``colr/prof`` and
+``colr/rICC`` properties from ``iprp/ipco``, compacts/remaps existing ``ipma``
+associations, appends the transferred ``colr/prof`` property, and associates it
+with the primary item. Arbitrary non-ICC property replacement and broader BMFF
+scene/property graph rewriting remain out of scope.
 
 If sidecar-only writeback asks to strip embedded XMP, OpenMeta can remove XMP
 from its own OpenMeta-authored metadata ``meta`` box and from parseable

docs/writer_target_contract.md

@@ -59,7 +59,7 @@ integration checks.
 | `WebP` | `EXIF`, XMP RIFF chunk, `ICCP`, bounded `C2PA` | Replace matching managed RIFF chunks; preserve unrelated chunks; patch `VP8X` feature bits | EXIF/XMP/ICC edits require an existing `VP8X` chunk |
 | `JP2` | top-level `Exif`, top-level XMP `xml` box, `jp2h/colr` ICC | Replace matching top-level metadata boxes; rewrite `jp2h` only to replace/insert `colr`; preserve unrelated boxes and unrelated `jp2h` children | Does not synthesize `jp2h`; requires one existing `jp2h` for ICC |
 | `JXL` | top-level `Exif`, XMP `xml` box, `jumb`, `c2pa` | Replace matching top-level boxes; preserve signature and non-managed boxes; classify `jumb` as generic JUMBF or C2PA | ICC is encoder handoff only; file edit emits uncompressed prepared metadata boxes |
-| `HEIF` / `AVIF` / `CR3` | BMFF metadata items (`Exif`, XMP `mime`, JUMBF/C2PA), bounded `colr/prof` ICC properties, plus OpenMeta-authored metadata-only `meta` boxes | Preserve non-`meta` top-level boxes; replace prior OpenMeta-authored metadata `meta`; merge/replace/strip item metadata in parseable foreign top-level `meta` graphs by extending `iinf`/`iloc`/`idat`/`iref`; replace prior ICC `colr` properties and remap `iprp`/`ipco`/`ipma` for bounded ICC | Does not rewrite arbitrary BMFF scene/property graphs |
+| `HEIF` / `AVIF` / `CR3` | BMFF metadata items (`Exif`, XMP `mime`, JUMBF/C2PA), bounded `colr/prof` ICC properties, plus OpenMeta-authored metadata-only `meta` boxes | Preserve non-`meta` top-level boxes; replace prior OpenMeta-authored metadata `meta`; merge/replace/strip item metadata in parseable foreign top-level `meta` graphs by extending `iinf`/`iloc`/`idat`/`iref`; replace prior ICC `colr` properties and remap `iprp`/`ipco`/`ipma` for bounded ICC | Does not rewrite arbitrary BMFF scene/property graphs; 32-bit item-id insertion requires existing `iloc` v2 |
 | `EXR` | safe string header attributes through the EXR transfer emitter or adapter batch | No file rewrite contract today; host applies prepared attributes through its own EXR writer | Attribute-emitter target, not a file edit path |
 
 ## JPEG
@@ -192,6 +192,12 @@ graphs it merges, replaces, or strips bounded Exif/XMP/JUMBF/C2PA metadata
 items in the existing `meta` by extending `iinf`, `iloc`, `idat`, and `iref`
 with `cdsc` references to the primary item. This constrained merge requires a
 single parseable `iinf`, `iloc` version 0/1/2, `pitm`, and at most one `idat`.
+For `iloc` version 0/1 targets, inserted item IDs remain in the 16-bit item-id
+space. For `iloc` version 2 targets, OpenMeta can allocate 32-bit item IDs and
+uses `infe` version 3 plus `iref` version 1 when a newly inserted item needs
+that wider ID space. If the existing graph has exhausted the usable item-id
+space or mixes item-table widths outside this shape, the edit fails instead of
+truncating IDs.
 For bounded ICC transfer, OpenMeta removes prior ICC `colr/prof` and
 `colr/rICC` properties from `iprp/ipco`, compacts/remaps existing `ipma`
 associations, appends the transferred `colr/prof` property, and associates it

src/openmeta/container_scan.cc

@@ -2111,20 +2111,22 @@ namespace {
         uint64_t p            = payload_off + 4;
 
         uint32_t entry_count = 0;
-        if (version < 2) {
+        if (version == 0) {
             uint16_t ec16 = 0;
             if (!read_u16be(bytes, p, &ec16)) {
                 return false;
             }
             entry_count = ec16;
             p += 2;
-        } else {
+        } else if (version == 1 || version == 2) {
             uint32_t ec32 = 0;
             if (!read_u32be(bytes, p, &ec32)) {
                 return false;
             }
             entry_count = ec32;
             p += 4;
+        } else {
+            return false;
         }
 
         const uint32_t kMaxEntries = 4096;