Rendering large outputs when OIT is involved can exceed SW imposed buffer size limits

[dgovil]

Using USDRecord to render an output image exceeding 64M (e.g. 4096x2160 with 16-bits per component) will cause data loss sometimes leading to crashes or rendering corruption. We can reproduce this across multiple USD versions and have tested on macOS and Linux.

Steps To Reproduce:

1. Clone USD 25.08 from the OpenUSD repository (Note: problem still exists on dev branch as of Sept. 10, 2025):
   git clone https://github.com/PixarAnimationStudios/OpenUSD
   cd OpenUSD
   git checkout v25.08
2. Use this command to render an 8192x5977 image:
   usdrecord --purposes render --complexity medium --imageWidth 8192 --renderer Storm pxr/usdImaging/bin/testusdview/testenv/testUsdviewOIT/test.usda /tmp/test.exr

That sample file is part of the USD repo

Results:
This image will likely render okay (others, like the ALab scene may produce corruption). However, errors will be logged:

Warning: in ReallocateAll at line 163 of /Users/dean/Source/ThirdParty/OpenUSD_25.08/pxr/imaging/hdSt/bufferArrayRegistry.cpp -- Number of elements in the buffer array range (0x17590000) is _larger_ than the maximum number of elements in the buffer array (0x4000000). 0x13590000 bytes of data will be skipped.
Warning: in ReallocateAll at line 163 of /Users/dean/Source/ThirdParty/OpenUSD_25.08/pxr/imaging/hdSt/bufferArrayRegistry.cpp -- Number of elements in the buffer array range (0x17590000) is _larger_ than the maximum number of elements in the buffer array (0x10000000). 0x7590000 bytes of data will be skipped.
Warning: in ReallocateAll at line 163 of /Users/dean/Source/ThirdParty/OpenUSD_25.08/pxr/imaging/hdSt/bufferArrayRegistry.cpp -- Number of elements in the buffer array range (0x17590000) is _larger_ than the maximum number of elements in the buffer array (0x10000000). 0x7590000 bytes of data will be skipped.
Recording aborted due to the following failure at time code 1.0: 
	Error in 'pxrInternal_v0_25_8__pxrReserved__::HdStVBOSimpleMemoryManager::_SimpleBufferArray::Reallocate' at line 253 in file /Users/dean/Source/ThirdParty/OpenUSD_25.08/pxr/imaging/hdSt/vboSimpleMemoryManager.cpp : 'Failed verification: ' curRangeOwner == shared_from_this() ''
	Error in 'pxrInternal_v0_25_8__pxrReserved__::HdStVBOSimpleMemoryManager::_SimpleBufferArray::Reallocate' at line 254 in file /Users/dean/Source/ThirdParty/OpenUSD_25.08/pxr/imaging/hdSt/vboSimpleMemoryManager.cpp : 'HdStVBOSimpleMemoryManager can't reassign ranges'
	Error in 'pxrInternal_v0_25_8__pxrReserved__::HdStVBOSimpleMemoryManager::_SimpleBufferArray::Reallocate' at line 253 in file /Users/dean/Source/ThirdParty/OpenUSD_25.08/pxr/imaging/hdSt/vboSimpleMemoryManager.cpp : 'Failed verification: ' curRangeOwner == shared_from_this() ''
	Error in 'pxrInternal_v0_25_8__pxrReserved__::HdStVBOSimpleMemoryManager::_SimpleBufferArray::Reallocate' at line 254 in file /Users/dean/Source/ThirdParty/OpenUSD_25.08/pxr/imaging/hdSt/vboSimpleMemoryManager.cpp : 'HdStVBOSimpleMemoryManager can't reassign ranges'
	Error in 'pxrInternal_v0_25_8__pxrReserved__::HdStVBOSimpleMemoryManager::_SimpleBufferArray::Reallocate' at line 253 in file /Users/dean/Source/ThirdParty/OpenUSD_25.08/pxr/imaging/hdSt/vboSimpleMemoryManager.cpp : 'Failed verification: ' curRangeOwner == shared_from_this() ''
	Error in 'pxrInternal_v0_25_8__pxrReserved__::HdStVBOSimpleMemoryManager::_SimpleBufferArray::Reallocate' at line 254 in file /Users/dean/Source/ThirdParty/OpenUSD_25.08/pxr/imaging/hdSt/vboSimpleMemoryManager.cpp : 'HdStVBOSimpleMemoryManager can't reassign ranges'

Notes:
The issue arises from two limits:

1. In pxr/imaging/hgiMetal/capabilities.mm, _maxShaderStorageBlockSize is hard-coded to 1GB (110241024*1024)
2. In pxr/imaging/hdSt/vboMemoryManager.cpp, HD_MAX_VBO_SIZE defaults to 1GB
   The second item can be worked around (for some cases) by setting the environment variable HD_MAX_VBO_SIZE. However, the value is stored in a 32-bit, signed integer, so the actual limit can only be increased to just shy of 2GB. The first issue could/should be modified to get the value from the Metal device (e.g. using [device maxBufferLength]). However, this runs into another issue through USD which is that size_t is used for buffer sizes and ends up as an unsigned 32-bit integer. So, for example, on my M1 Max with 64GB of memory, the value returned from [device maxBufferLength] is 44GB but would have to be truncated to just shy of 4GB to fit in a size_t.

(Reported on behalf of Dean Macri)

[jesschimein]

Filed as internal issue #USD-11658

(This is an automated message. See here for more information.)