Fork a small set of nanoVDB headers to share PyTorch's allocator

Add an override directory at `src/fvdb/nanovdb_overrides/` that holds
modified copies of a few upstream nanoVDB headers. The build prepends
this directory to the include search path so that, e.g.,
`#include <nanovdb/cuda/DeviceBuffer.h>` resolves to the forked copy
under `src/fvdb/nanovdb_overrides/`, while the rest of nanoVDB
continues to come from the upstream source tree.

Forked headers:

- `nanovdb/cuda/DeviceBuffer.h` and `nanovdb/cuda/DeviceResource.h`
  route device-side allocations through
  `c10::cuda::CUDACachingAllocator` (PyTorch's caching allocator)
  instead of `cudaMallocAsync` / `cudaFreeAsync`. This keeps
  transient scratch allocations made by nanoVDB internals
  (MergeGrids, TopologyBuilder, DilateGrid, ...) inside the same
  pool that fvdb / PyTorch tensors use. Without this, the two
  pools fragment each other and large workloads such as
  multi-frame TSDF integration hit a clean OOM even when the GPU
  still has free memory in aggregate.

- `nanovdb/tools/cuda/TopologyBuilder.cuh` adds an opt-in debug
  trace gated on the `FVDB_NANOVDB_TRACE_ALLOCS` env var that
  prints tile count and per-call scratch size from
  `allocateInternalMaskBuffers`. Useful for diagnosing topology-op
  memory blowup on large scenes; off by default and otherwise
  byte-identical with upstream.

Each forked header carries a short `FVDB FORK:` banner documenting
what diverges from upstream and why, with inline `// FVDB FORK:`
tags on every non-trivial change so `git blame` and text search make
the delta obvious. `src/fvdb/nanovdb_overrides/README.md` documents
the resync and add-an-override procedures.

CMake wiring lives in `src/cmake/get_nanovdb.cmake`, with mirror
include-path orderings in `src/CMakeLists.txt` (fvdb target) and
the top-level `CMakeLists.txt` (_fvdb_cpp pybind module).

Signed-off-by: Francis Williams <francis@fwilliams.info>

Author: fwilliams

CMakeLists.txt

@@ -169,6 +169,11 @@ set_target_properties(_fvdb_cpp PROPERTIES
 target_include_directories(_fvdb_cpp PUBLIC
     ${CMAKE_CURRENT_SOURCE_DIR}/src
     ${TORCH_INCLUDE_DIRS}
+    # Overrides directory must come BEFORE the upstream nanoVDB source so that
+    # forked headers (e.g. nanovdb/cuda/DeviceBuffer.h) win the include search.
+    # Mirror the ordering used in src/CMakeLists.txt for the fvdb target.
+    # See src/fvdb/nanovdb_overrides/README.md.
+    ${FVDB_NANOVDB_OVERRIDES_DIR}
     ${nanovdb_SOURCE_DIR}/nanovdb
     ${NANOVDB_EDITOR_INCLUDE_DIR})
 target_link_libraries(_fvdb_cpp PRIVATE

src/CMakeLists.txt

@@ -170,6 +170,10 @@ message(STATUS "fvdb: TORCH_INCLUDE_DIRS: ${TORCH_INCLUDE_DIRS}")
 
 target_include_directories(fvdb PUBLIC
     $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}>
+    # Overrides directory must come BEFORE the upstream nanoVDB source so that
+    # forked headers (e.g. nanovdb/cuda/DeviceBuffer.h) win the include search.
+    # See src/fvdb/nanovdb_overrides/README.md.
+    $<BUILD_INTERFACE:${FVDB_NANOVDB_OVERRIDES_DIR}>
     $<BUILD_INTERFACE:${nanovdb_SOURCE_DIR}/nanovdb>
     $<BUILD_INTERFACE:${nvtx3_SOURCE_DIR}/include>
     $<BUILD_INTERFACE:${NANOVDB_EDITOR_INCLUDE_DIR}>

src/cmake/get_nanovdb.cmake

@@ -11,7 +11,24 @@ CPMAddPackage(
 
 # NanoVDB is header only, so we don't build it. Instead, we just add the headers
 # to the include path and create an interface target.
+#
+# We also prepend an override directory that contains modified copies of a small
+# number of upstream nanoVDB headers. The override directory comes first in the
+# include search path so that, e.g., `#include <nanovdb/cuda/DeviceBuffer.h>`
+# resolves to our forked copy under
+# `src/fvdb/nanovdb_overrides/nanovdb/cuda/DeviceBuffer.h`. The rest of nanoVDB
+# is still picked up from the upstream source tree, so the patch surface stays
+# minimal and easy to resync.
+#
+# See src/fvdb/nanovdb_overrides/README.md for the rationale and the procedure
+# for adding new overrides.
 if(nanovdb_ADDED)
+    get_filename_component(_fvdb_src_dir "${CMAKE_CURRENT_LIST_DIR}" DIRECTORY)
+    set(FVDB_NANOVDB_OVERRIDES_DIR
+        "${_fvdb_src_dir}/fvdb/nanovdb_overrides"
+        CACHE INTERNAL "Directory with fvdb-local overrides for nanoVDB headers")
     add_library(nanovdb INTERFACE)
-    target_include_directories(nanovdb INTERFACE ${nanovdb_SOURCE_DIR}/nanovdb)
+    target_include_directories(nanovdb INTERFACE
+        ${FVDB_NANOVDB_OVERRIDES_DIR}
+        ${nanovdb_SOURCE_DIR}/nanovdb)
 endif()

src/fvdb/nanovdb_overrides/README.md

@@ -0,0 +1,59 @@
+# fvdb-local overrides for nanoVDB headers
+
+This directory holds **modified copies of a small number of upstream nanoVDB
+headers**. The build prepends this directory to the include search path before
+the upstream nanoVDB source tree, so any `#include <nanovdb/...>` that matches
+a file in this tree resolves here. Everything else falls through to upstream.
+
+The wiring lives in `src/cmake/get_nanovdb.cmake`.
+
+## Why not just patch the upstream checkout?
+
+We previously tried `CPM`'s `PATCH_COMMAND` to fix a specific bug (nanoVDB
+scratch allocations going through `cudaMallocAsync` instead of PyTorch's
+caching allocator, which fragments into two pools and OOMs). Patch-based
+approaches are fragile: they silently stop applying if upstream moves the
+surrounding lines, and they make it hard to edit nanoVDB during development.
+
+Forking the exact headers we care about into this tree gives us full edit
+access with a narrow, reviewable patch surface and a clean diff-against-upstream
+workflow.
+
+## Layout
+
+The directory structure **mirrors upstream** starting from the `nanovdb/`
+include root:
+
+```
+nanovdb_overrides/
+  nanovdb/
+    cuda/
+      DeviceBuffer.h     # forked: uses c10::cuda::CUDACachingAllocator
+    ...                  # add more as needed
+```
+
+## Adding a new override
+
+1. Copy the upstream header from
+   `build/<...>/_deps/nanovdb-src/nanovdb/nanovdb/<rel_path>` into
+   `nanovdb_overrides/nanovdb/<rel_path>`, preserving the relative path.
+2. Add a short `FVDB FORK:` banner at the top documenting *what* diverges from
+   upstream and *why*. Keep the rest of the file byte-identical so a future
+   resync with upstream is a clean 3-way merge.
+3. Every non-trivial code change inside the file should be tagged with an
+   inline `// FVDB FORK:` comment pointing at the banner, so `git blame` and
+   text searches make the delta obvious.
+
+## Resyncing with upstream
+
+When bumping the nanoVDB pin in `get_nanovdb.cmake`:
+
+1. `diff` each file in this directory against its upstream counterpart.
+2. Port the upstream changes over, keeping the `FVDB FORK` deltas.
+3. Rebuild + rerun the fvdb test suite.
+
+## Current overrides
+
+| File                         | Reason                                                                                |
+|------------------------------|---------------------------------------------------------------------------------------|
+| `nanovdb/cuda/DeviceBuffer.h`| Route device allocations through PyTorch's caching allocator (avoids dual-pool OOM).   |