USDZ sub-asset mmap multiplied per asset, exhausting virtual address space

[dgovil]

Summary

When CrateFile opens a .usdc embedded in a .usdz using mmap, or when
GetBuffer() is called on any ArAsset from within a .usdz, the entire
.usdz file is mapped into virtual address space — even though only a small
sub-range is needed. With a package containing N embedded .usdc layers or
N long-lived texture/asset buffers, this creates N separate full-file mappings,
consuming roughly N × size(.usdz) bytes of virtual address space.

On platforms where virtual address space is limited — notably Apple embedded
platforms (iOS, tvOS, visionOS) — this makes the mmap optimisation
unacceptably costly and can cause allocation failures.

Note: Chloe started an email thread with Pixar circa 2020 on this topic, but I think we dropped the thread unfortunately.

I have a repro script here for example

demo_usdz_mmap.py

--- (I asked Claude to generate the analysis below this line for convenience use as an FYI)

Details

Path 1 — CrateFile::_MmapAsset (dominant case)

When USD opens a .usdc asset and a valid FILE* is available, CrateFile
prefers mmap over pread. The relevant call site is _MmapAsset
(pxr/usd/sdf/crateFile.cpp:2261):

FILE *file; size_t offset;
std::tie(file, offset) = asset->GetFileUnsafe();   // crateFile.cpp:2264
auto mapping = _FileMapping(
    ArchMapFileReadOnly(file, &errMsg),             // crateFile.cpp:2266
    offset, asset->GetSize());                      // crateFile.cpp:2267

For a .usdc embedded in a .usdz, GetFileUnsafe() is implemented in
usdzResolver.cpp (_Asset::GetFileUnsafe) and returns the FILE* of the
outer .usdz with offset set to the sub-file's byte position within it.

ArchMapFileReadOnly(file) maps the file from byte 0 to EOF — the entire
.usdz. The offset is then only used to position a pointer inside that
mapping (_start = mapping.get() + offset, pxr/usd/sdf/crateFile.h:380).

Result: each embedded .usdc that is loaded causes one complete mmap of
the .usdz, regardless of the sub-file's actual size.

Path 2 — SdfZipFile::Open / ArAsset::GetBuffer (textures and other assets)

When OpenAsset is called for any sub-file within a .usdz,
Sdf_UsdzResolverCache::_OpenZipFile (pxr/usd/sdf/usdzResolver.cpp:61)
calls:

result.second = SdfZipFile::Open(result.first);   // usdzResolver.cpp:66

SdfZipFile::Open(asset) calls asset->GetBuffer() (pxr/usd/sdf/zipFile.cpp:609).
For an ArFilesystemAsset this executes:

ArchConstFileMapping mapping = ArchMapFileReadOnly(_file);  // filesystemAsset.cpp:68

GetBuffer() is not cached — each call produces a new mapping of the full
.usdz. The mapping lives as long as the returned shared_ptr<const char> is
alive, which for texture/asset buffers can be the duration of the render.

When no Sdf_UsdzResolverCache scope is active (e.g. direct
Sdf.Layer.FindOrOpen() calls from Python rather than full stage composition),
each OpenAsset invocation also creates a fresh ArFilesystemAsset, so Path 2
alone produces N full-file mappings for N sub-file opens.

Combined cost

Scenario	Mmap count	Virtual address space
N .usdc sub-layers, no cache scope	2N	~2N × size(.usdz)
N .usdc sub-layers, cache active	N + 1	~N × size(.usdz)
N texture/asset buffers, no cache scope	N	~N × size(.usdz)
N texture/asset buffers, cache active	1	~1 × size(.usdz) ✓

Even with the cache active, CrateFile::_MmapAsset produces one additional
full-file mmap per .usdc sub-layer loaded. The cache fully solves Path 2 for
textures but only halves the problem for .usdc layers.

---

Impact

• Apple embedded platforms (iOS, tvOS, visionOS): virtual address space is
  tightly constrained. A modest .usdz with tens of embedded .usdc layers can
  exhaust it, causing mmap to fail and forcing a fallback to slower read paths or
  outright crashes.
• Large packages on 64-bit desktops: a .usdz with hundreds of embedded
  layers wastes significant VA space and degrades TLB/page-table performance.
• File descriptors (secondary): without a cache scope, N sub-file opens also
  consume N FILE* handles to the same .usdz, which can hit per-process fd
  limits (ulimit -n, default 256 on macOS).

---

Proposed fixes

Option A — Sub-range mapping in CrateFile::_MmapAsset (targeted fix)

Replace the full-file mmap with a region-scoped mapping:

// crateFile.cpp, _MmapAsset:
// Before (maps entire .usdz):
auto mapping = _FileMapping(ArchMapFileReadOnly(file, &errMsg), offset, size);

// After (maps only the sub-file's byte range):
auto mapping = _FileMapping(ArchMapFileRegion(file, offset, size, &errMsg), 0, size);

This reduces VA cost per .usdc sub-layer from size(.usdz) to
size(.usdc_within_usdz).

Option B — Single shared mmap at the Sdf_UsdzResolverCache level

Cache a single ArchConstFileMapping of the .usdz in Sdf_UsdzResolverCache
and hand sub-assets aliased shared_ptr views into it (the pattern used by
ArInMemoryAsset::FromBuffer). All assets from the same package share one
mapping independent of whether a cache scope is active. This solves both Path 1
and Path 2, but requires CrateFile to accept a pre-existing mapping rather
than creating its own.

Option C — Prefer pread over mmap for offset-based sub-assets

CrateFile::Open could detect that GetFileUnsafe() returns a non-zero
offset (indicating the asset lives inside a container) and prefer pread in that
case, avoiding the full-file mmap entirely at the cost of sequential-read
performance for that asset. This is a conservative safe-harbour while a
structural fix is developed.

[jesschimein]

Filed as internal issue #USD-12130

(This is an automated message. See here for more information.)