Skip to content

Commit 419fcb6

Browse files
swahtzclaude
andauthored
Route NanoVDB builder scratch through PyTorch's active CUDA allocator via upstream memory-resource seams (#732)
## Summary fvdb's grid builders allocate their device scratch from nanoVDB's default `DeviceResource` — a second `cudaMallocAsync` pool that partitions VRAM against PyTorch's. Large workloads (e.g. multi-frame TSDF integration) then hit a clean OOM even when the GPU has free memory in aggregate. This routes that scratch — O(N-points) sort keys, CUB temp storage, topology mask buffers — through PyTorch's CUDA allocator instead, so it shares one pool with fvdb / PyTorch tensors. **22 sites across 13 `.cu` files plus `PadGrid.cuh`.** Note this is not hardcoded to Torch's *native* caching allocator: `c10::cuda::CUDACachingAllocator` is a namespace, and its `raw_alloc_with_stream` / `raw_delete` free functions dispatch through `CUDACachingAllocator::get()` — the runtime-swappable allocator Torch itself allocates tensors from. fvdb's scratch therefore follows whatever allocator the user has installed: the native caching allocator (including `PYTORCH_CUDA_ALLOC_CONF` knobs), the `cudaMallocAsync` backend (`PYTORCH_CUDA_ALLOC_CONF=backend:cudaMallocAsync`), or a custom allocator installed via `torch.cuda.memory.change_current_allocator(CUDAPluggableAllocator(...))`. **Supersedes #655**, which vendored modified nanoVDB headers into the tree. This instead uses the injectable-memory-resource seams we developed upstream (AcademySoftwareFoundation/openvdb#2232; PRs [#2268](AcademySoftwareFoundation/openvdb#2268), [#2269](AcademySoftwareFoundation/openvdb#2269), [#2270](AcademySoftwareFoundation/openvdb#2270), [#2272](AcademySoftwareFoundation/openvdb#2272), [#2273](AcademySoftwareFoundation/openvdb#2273)) — now merged, so the pin is plain upstream master. No fork, no include-path shadowing, no resync procedure. ## What's in this PR 1. **Pin nanovdb to upstream master.** `src/cmake/get_nanovdb.cmake` → `AcademySoftwareFoundation/openvdb @ 7946f17e`, which includes the small-builder `ResourceT` seams ([#2286](AcademySoftwareFoundation/openvdb#2286)), the synchronous resource adapters ([#2272](AcademySoftwareFoundation/openvdb#2272)), and the `MeshToGrid` `CALL_CUBS` `#undef` fix ([#2284](AcademySoftwareFoundation/openvdb#2284)). 2. **`fvdb::TorchResource`.** A ~40-line stateless resource ([`src/fvdb/TorchResource.h`](src/fvdb/TorchResource.h)) modeling nanoVDB's stream-ordered `AsyncResource` concept over `c10::cuda::CUDACachingAllocator::raw_alloc_with_stream` / `raw_delete` — the dispatchers to Torch's currently active CUDA allocator (see Summary). Passed as the `ResourceT` template parameter at all 13 upstream builder call sites — `voxelsToGrid`, `DilateGrid`, `MergeGrids`, `PruneGrid`, `RefineGrid`, `CoarsenGrid` — always via the `fvdb::BuilderResource` alias ([`src/fvdb/BuilderResource.h`](src/fvdb/BuilderResource.h)), never named directly, so the allocator policy lives in a single line (a non-torch build, e.g. the ONNX Runtime EP planned in #579, retargets the alias there instead of touching every op). Being stateless, it binds through each builder's defaulted constructor argument, so no instance is plumbed through. Retains #655's `FVDB_NANOVDB_TRACE_ALLOCS` tracing (`=1` traces ≥ 256 KiB, a value starting with `2` traces everything). 3. **`PadGrid` gains a `ResourceT` seam.** The conv builders used `DilateGrid<..., TorchResource>` for odd kernels and `PadGrid` — on the rival pool — for even ones: same loop, same grid, a different allocator depending on kernel parity. fvdb's own `morphology::PadGrid` drives nanoVDB's `TopologyBuilder` (internal mask buffers, `countNodes` CUB scratch, `TempPool`) but hardcoded `DeviceResource`. It now takes a `ResourceT` parameter mirroring the upstream `DilateGrid` signature and forwards it, with `BuilderResource` passed at all 7 call sites. The default keeps it source-compatible. 4. **CUB scratch in `BuildFineGridFromCoarse`.** `cub::DeviceSegmentedReduce` temp storage used a bare `cudaMallocAsync`; it now routes through `BuilderResource`. Both `cub` calls are also now `C10_CUDA_CHECK`-wrapped — previously unchecked, as was the allocation. 5. **The `SaveNanoVDB` CUDA path.** The save path allocated its largest device buffers from nanoVDB's default pool: the per-batch `(N+1)`-element value staging buffer, the `indexToGrid` output grid handle, and the defensive host-upload buffer. All three now use `TorchDeviceBuffer`, and `indexToGrid`'s internal scratch routes through `TorchResource` via the #2286 seam. Stream-ordering is preserved: the replaced stream-ordered `DeviceBuffer` constructors become `raw_alloc` on the same current stream the copies and kernels are queued on. The host path (`indexToGridHost`) and the `HostBuffer` file-staging buffers are unchanged. ## Not routed (no upstream seam yet; all off the hot paths) - `DistributedPointsToGrid` multi-GPU scratch (deferred upstream behind AcademySoftwareFoundation/openvdb#2248) — the most valuable remaining seam - `VoxelBlockManager` / `buildVoxelBlockManager` scratch in `ReinitializeSdf.cu` - the builders' small dual-space `mProcessedRoot` / `mData` buffers (upstream roadmap Step 3) `MeshToGrid` is the one merged seam fvdb does not use: `BuildGridFromMesh.cu` does its own parametric surface sampling and goes through `_createNanoGridFromIJK`, so there is nothing to route. ## Test plan - [x] `./build.sh install` succeeds on a clean tree against the new pin (full CUDA build, `-Werror`). - [x] Injection verified live via `FVDB_NANOVDB_TRACE_ALLOCS`: a 500k-point `Grid.from_points` + `dilated_grid(2)` prints 42 `TorchResource` traces with correct results (484,631 → 8,461,871 voxels); `from_nearest_voxels_to_points` at 2M points shows `PadGrid` scratch routed. - [x] 584 tests passed — conv semantics + integration (203), conv/conv-transpose default + prune + empty grids (103), basic ops (276, 1 skipped), sliced batch (2, covering the `BuildFineGridFromCoarse` CUB path). - [x] `test_io.py` — 622 passed against the `7946f17e` pin; a traced `save_nanovdb` (`FVDB_NANOVDB_TRACE_ALLOCS=2`) shows the `indexToGrid` scratch flowing through `TorchResource`. ## Followups - Rebase the TSDF / ESDF / Occupancy stack (#656) onto this branch in place of #655, threading `TorchResource` through the new ops it adds. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Jonathan Swartz <jonathan@jswartz.info> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 025aa3e commit 419fcb6

18 files changed

Lines changed: 275 additions & 74 deletions

src/cmake/get_nanovdb.cmake

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
CPMAddPackage(
55
NAME nanovdb
66
GITHUB_REPOSITORY AcademySoftwareFoundation/openvdb
7-
GIT_TAG e538a0646b14125a043f623f205fcf218c5070a0
7+
GIT_TAG 7946f17edb443fe46076a22ea933e52a23453c24
88
SOURCE_SUBDIR nanovdb/nanovdb
99
DOWNLOAD_ONLY YES
1010
)

src/fvdb/BuilderResource.h

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
// Copyright Contributors to the OpenVDB Project
2+
// SPDX-License-Identifier: Apache-2.0
3+
//
4+
#ifndef FVDB_BUILDERRESOURCE_H
5+
#define FVDB_BUILDERRESOURCE_H
6+
7+
#include <fvdb/TorchResource.h>
8+
9+
namespace fvdb {
10+
11+
/// @brief The memory resource fvdb's ops bind as the ResourceT template
12+
/// parameter of nanoVDB's CUDA builders (and of fvdb's own PadGrid),
13+
/// routing their internal device scratch.
14+
///
15+
/// This alias is the single seam choosing that policy: call sites name
16+
/// BuilderResource, never a concrete resource type. Today it is
17+
/// TorchResource, which allocates from PyTorch's currently active CUDA
18+
/// allocator (see TorchResource.h). A build that must run these
19+
/// builders without torch (e.g. an ONNX Runtime execution provider,
20+
/// where c10 is unavailable) retargets the alias here — behind a
21+
/// build-time switch guarding the TorchResource include — instead of
22+
/// touching every op.
23+
///
24+
/// The alias covers the builders' scratch only. Buffer allocations that
25+
/// are torch tensors by design (TorchDeviceBuffer, the SaveNanoVDB
26+
/// staging buffers) name their types directly.
27+
///
28+
/// Note the seam is compile-time and relies on the resource being
29+
/// stateless: builders bind the shared instance from
30+
/// nanovdb::cuda::default_resource<BuilderResource>() through their
31+
/// defaulted constructor arguments. A stateful resource (e.g. one
32+
/// holding a per-session allocator handle) additionally needs an
33+
/// instance plumbed through the ops' call sites.
34+
using BuilderResource = TorchResource;
35+
36+
} // namespace fvdb
37+
38+
#endif // FVDB_BUILDERRESOURCE_H

src/fvdb/TorchResource.h

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
// Copyright Contributors to the OpenVDB Project
2+
// SPDX-License-Identifier: Apache-2.0
3+
//
4+
#ifndef FVDB_TORCHRESOURCE_H
5+
#define FVDB_TORCHRESOURCE_H
6+
7+
#include <nanovdb/cuda/DeviceResource.h>
8+
9+
#include <c10/cuda/CUDACachingAllocator.h>
10+
11+
#include <cstdio>
12+
#include <cstdlib>
13+
#include <stdexcept>
14+
15+
namespace fvdb {
16+
17+
/// @brief NanoVDB stream-ordered memory resource backed by PyTorch's currently
18+
/// active CUDA allocator.
19+
///
20+
/// c10::cuda::CUDACachingAllocator is a namespace, not a concrete
21+
/// allocator: its free functions raw_alloc_with_stream / raw_delete
22+
/// dispatch through CUDACachingAllocator::get(), the runtime-swappable
23+
/// c10::cuda::CUDAAllocator* Torch itself allocates tensors from. This
24+
/// resource therefore follows whatever allocator the user has installed —
25+
/// the native caching allocator (including PYTORCH_CUDA_ALLOC_CONF knobs),
26+
/// the cudaMallocAsync backend (PYTORCH_CUDA_ALLOC_CONF=backend:cudaMallocAsync),
27+
/// or a user-provided allocator installed via
28+
/// torch.cuda.memory.change_current_allocator(CUDAPluggableAllocator(...)).
29+
///
30+
/// Passed as the ResourceT template parameter of NanoVDB's CUDA builders
31+
/// (PointsToGrid / DilateGrid / MergeGrids / PruneGrid / RefineGrid /
32+
/// CoarsenGrid) — always via the fvdb::BuilderResource alias
33+
/// (BuilderResource.h), never named directly at call sites — it routes
34+
/// their internal device scratch — O(N-points) sort
35+
/// keys, CUB temp storage, topology mask buffers — through the same pool
36+
/// that fvdb / PyTorch tensors use. Without this, nanoVDB's default
37+
/// DeviceResource allocates from a second cudaMallocAsync pool that
38+
/// partitions VRAM against torch's pool, and large workloads (e.g.
39+
/// multi-frame TSDF integration) OOM even when the GPU has free memory in
40+
/// aggregate.
41+
///
42+
/// The resource is stateless, so builders can bind the shared instance
43+
/// returned by nanovdb::cuda::default_resource<TorchResource>() — naming
44+
/// the template parameter at a call site is sufficient, no instance needs
45+
/// to be threaded through.
46+
///
47+
/// Set FVDB_NANOVDB_TRACE_ALLOCS=1 in the environment to trace allocations
48+
/// of 256 KiB and larger to stderr (a value starting with '2' traces every
49+
/// allocation). Useful for diagnosing topology-op memory blowup on large
50+
/// scenes.
51+
struct TorchResource : nanovdb::cuda::SyncFromAsync<TorchResource> {
52+
/// Alignment guaranteed by every allocation. Torch's native caching
53+
/// allocator returns blocks aligned to at least 512 bytes and the
54+
/// cudaMallocAsync backend to at least 256, so advertising nanoVDB's
55+
/// conventional 256 (matching cuda::DeviceResource) is satisfied and the
56+
/// alignment parameter below can be ignored. A pluggable allocator wrapping
57+
/// any cudaMalloc-family call satisfies 256 as well.
58+
static constexpr size_t DEFAULT_ALIGNMENT = 256;
59+
60+
/// @brief Stream-ordered allocation from torch's active CUDA allocator.
61+
/// @note raw_alloc_with_stream records @p stream against the block so torch
62+
/// defers reuse until work on it completes, matching the stream-ordered
63+
/// semantics of the cudaMallocAsync call it replaces. Allocation
64+
/// happens on the current device, like cudaMallocAsync. The call
65+
/// dispatches to CUDACachingAllocator::get(), so a swapped-in backend
66+
/// or pluggable allocator is honored.
67+
void *
68+
allocate_async(size_t bytes, size_t /*alignment*/, cudaStream_t stream) {
69+
if (const char *env = std::getenv("FVDB_NANOVDB_TRACE_ALLOCS")) {
70+
const size_t cutoff =
71+
(env[0] == '2') ? 0 : (1ull << 18); // '2' = trace all, else >= 256 KiB
72+
if (bytes >= cutoff) {
73+
std::fprintf(stderr,
74+
"[fvdb/nanovdb] TorchResource alloc %12zu bytes (%.3f MB)\n",
75+
bytes,
76+
double(bytes) / 1e6);
77+
}
78+
}
79+
void *p = c10::cuda::CUDACachingAllocator::raw_alloc_with_stream(bytes, stream);
80+
if (!p) {
81+
throw std::runtime_error("fvdb: TorchResource::allocate_async failed");
82+
}
83+
return p;
84+
}
85+
86+
/// @brief Free through torch's active CUDA allocator.
87+
/// @note The stream argument is deliberately ignored: raw_delete relies on
88+
/// the stream recorded at allocation time — the native backend's
89+
/// per-stream event tracking, or the alloc-time stream Torch hands a
90+
/// pluggable allocator's free function — so the free is safe without
91+
/// ordering on the caller's stream. This is the same contract Torch's
92+
/// own tensor frees rely on.
93+
void
94+
deallocate_async(void *p, size_t /*bytes*/, size_t /*alignment*/, cudaStream_t /*stream*/) {
95+
if (p == nullptr) {
96+
return;
97+
}
98+
c10::cuda::CUDACachingAllocator::raw_delete(p);
99+
}
100+
};
101+
102+
static_assert(nanovdb::cuda::is_async_resource<TorchResource>::value,
103+
"TorchResource must model nanoVDB's stream-ordered AsyncResource concept");
104+
105+
} // namespace fvdb
106+
107+
#endif // FVDB_TORCHRESOURCE_H

src/fvdb/detail/io/SaveNanoVDB.cu

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
// Copyright Contributors to the OpenVDB Project
22
// SPDX-License-Identifier: Apache-2.0
33
//
4+
#include <fvdb/BuilderResource.h>
5+
#include <fvdb/TorchDeviceBuffer.h>
46
#include <fvdb/detail/io/SaveNanoVDB.h>
57
#include <fvdb/detail/utils/Utils.h>
68

@@ -618,7 +620,7 @@ fvdbToNanovdbGridWithValues(const GridBatchData &gridBatchData,
618620
}
619621

620622
using HostGridHandle = nanovdb::GridHandle<nanovdb::HostBuffer>;
621-
using DeviceGridHandle = nanovdb::GridHandle<nanovdb::cuda::DeviceBuffer>;
623+
using DeviceGridHandle = nanovdb::GridHandle<TorchDeviceBuffer>;
622624
using ValueT = typename nanovdb::BuildToValueMap<OutBuildT>::type;
623625

624626
// Hoist tensor shape info out of the per-batch loop. The data tensor has shape
@@ -647,7 +649,7 @@ fvdbToNanovdbGridWithValues(const GridBatchData &gridBatchData,
647649
// Determine the device pointer to the source index grid buffer. CPU-resident grids normally
648650
// return through the host path above; the upload branch is kept as a defensive fallback if
649651
// this helper is reused without that dispatch.
650-
nanovdb::cuda::DeviceBuffer tmpDevBuf; // empty unless we need to upload
652+
TorchDeviceBuffer tmpDevBuf; // empty unless we need to upload
651653
const torch::Device gridDevice = gridBatchData.device();
652654
const torch::Device cudaDevice = gridDevice.is_cuda()
653655
? gridDevice
@@ -662,7 +664,7 @@ fvdbToNanovdbGridWithValues(const GridBatchData &gridBatchData,
662664
const uint64_t srcBufferSize = gridBatchData.nanoGridHandle().buffer().size();
663665
const uint8_t *srcHostData =
664666
static_cast<const uint8_t *>(gridBatchData.nanoGridHandle().buffer().data());
665-
tmpDevBuf = nanovdb::cuda::DeviceBuffer(srcBufferSize, cudaDevice.index(), stream.stream());
667+
tmpDevBuf = TorchDeviceBuffer(srcBufferSize, cudaDevice);
666668
cudaCheck(cudaMemcpyAsync(tmpDevBuf.deviceData(),
667669
srcHostData,
668670
srcBufferSize,
@@ -685,7 +687,7 @@ fvdbToNanovdbGridWithValues(const GridBatchData &gridBatchData,
685687
// on the same stream as the indexToGrid kernels so the GPU can run them back-to-back.
686688

687689
std::vector<DeviceGridHandle> deviceHandles;
688-
std::vector<nanovdb::cuda::DeviceBuffer> perBatchValueBufs;
690+
std::vector<TorchDeviceBuffer> perBatchValueBufs;
689691
std::vector<nanovdb::HostBuffer> hostBuffers;
690692
std::vector<uint64_t> origGridBytesPerBi;
691693
deviceHandles.reserve(gridBatchData.batchSize());
@@ -708,9 +710,8 @@ fvdbToNanovdbGridWithValues(const GridBatchData &gridBatchData,
708710
dSrcBufferStart + gridBatchData.cumBytesAt(bi));
709711

710712
const uint64_t valueBufElems = static_cast<uint64_t>(numVoxelsBi) + 1u;
711-
nanovdb::cuda::DeviceBuffer valueBuf(
712-
valueBufElems * sizeof(ValueT), cudaDevice.index(), stream.stream());
713-
ValueT *dValuesBufBase = static_cast<ValueT *>(valueBuf.deviceData());
713+
TorchDeviceBuffer valueBuf(valueBufElems * sizeof(ValueT), cudaDevice);
714+
ValueT *dValuesBufBase = reinterpret_cast<ValueT *>(valueBuf.deviceData());
714715
cudaCheck(cudaMemsetAsync(dValuesBufBase, 0, sizeof(ValueT), stream.stream()));
715716
if (numVoxelsBi > 0) {
716717
cudaCheck(cudaMemcpyAsync(dValuesBufBase + 1,
@@ -720,8 +721,11 @@ fvdbToNanovdbGridWithValues(const GridBatchData &gridBatchData,
720721
stream.stream()));
721722
}
722723

723-
DeviceGridHandle dh = nanovdb::tools::cuda::indexToGrid<OutBuildT>(
724-
dSrcGrid, dValuesBufBase, nanovdb::cuda::DeviceBuffer(), stream.stream());
724+
// The guide buffer only communicates the target device; the output grid buffer and the
725+
// builder's internal scratch both come from torch's caching allocator.
726+
DeviceGridHandle dh = nanovdb::tools::cuda::
727+
indexToGrid<OutBuildT, nanovdb::ValueOnIndex, TorchDeviceBuffer, BuilderResource>(
728+
dSrcGrid, dValuesBufBase, TorchDeviceBuffer(0, cudaDevice), stream.stream());
725729

726730
const uint64_t origGridBytes = dh.buffer().size();
727731
const uint64_t totalBytes = origGridBytes + blindOverhead;

src/fvdb/detail/ops/BuildCoarseGridFromFine.cu

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
// Copyright Contributors to the OpenVDB Project
22
// SPDX-License-Identifier: Apache-2.0
33
//
4+
#include <fvdb/BuilderResource.h>
45
#include <fvdb/GridBatchData.h>
56
#include <fvdb/detail/GridBatchDataFactory.h>
67
#include <fvdb/detail/ops/BuildCoarseGridFromFine.h>
@@ -84,7 +85,8 @@ coarseGridHandleFromFineCUDA(const GridBatchData &fineGridBatch,
8485
TORCH_CHECK(grid, "Grid is null");
8586
nanovdb::GridHandle<TorchDeviceBuffer> handle;
8687
for (int p = 0; p < nPasses; p += 1) {
87-
nanovdb::tools::cuda::CoarsenGrid<nanovdb::ValueOnIndex> op(grid, stream.stream());
88+
nanovdb::tools::cuda::CoarsenGrid<nanovdb::ValueOnIndex, BuilderResource> op(
89+
grid, stream.stream());
8890
op.setChecksum(nanovdb::CheckMode::Default);
8991
op.setVerbose(0);
9092
handle = op.getHandle(guide);

src/fvdb/detail/ops/BuildDenseGrid.cu

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
// Copyright Contributors to the OpenVDB Project
22
// SPDX-License-Identifier: Apache-2.0
33
//
4+
#include <fvdb/BuilderResource.h>
45
#include <fvdb/GridBatchData.h>
56
#include <fvdb/detail/GridBatchDataFactory.h>
67
#include <fvdb/detail/ops/BuildDenseGrid.h>
@@ -143,8 +144,9 @@ dispatchCreateNanoGridFromDense<torch::kCUDA>(int64_t batchSize,
143144
handles.push_back(createEmptyGridHandle(guide.device()));
144145
} else if (i == 0) {
145146
handles.push_back(
146-
nanovdb::tools::cuda::voxelsToGrid<GridT, nanovdb::Coord *, TorchDeviceBuffer>(
147-
(nanovdb::Coord *)ijkData.data_ptr(), nVoxels, 1.0, guide));
147+
nanovdb::tools::cuda::
148+
voxelsToGrid<GridT, nanovdb::Coord *, TorchDeviceBuffer, BuilderResource>(
149+
(nanovdb::Coord *)ijkData.data_ptr(), nVoxels, 1.0, guide));
148150
C10_CUDA_KERNEL_LAUNCH_CHECK();
149151
} else {
150152
handles.push_back(handles[0].copy(guide));

src/fvdb/detail/ops/BuildDilatedGrid.cu

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
// Copyright Contributors to the OpenVDB Project
22
// SPDX-License-Identifier: Apache-2.0
33
//
4+
#include <fvdb/BuilderResource.h>
45
#include <fvdb/TorchDeviceBuffer.h>
56
#include <fvdb/detail/GridBatchDataFactory.h>
67
#include <fvdb/detail/ops/BuildDilatedGrid.h>
@@ -54,7 +55,8 @@ dispatchDilateGrid<torch::kCUDA>(const GridBatchData &gridBatch,
5455
TORCH_CHECK(grid, "Grid is null");
5556

5657
for (auto j = 0; j < dilationAmount[i]; j += 1) {
57-
nanovdb::tools::cuda::DilateGrid<nanovdb::ValueOnIndex> dilateOp(grid, stream);
58+
nanovdb::tools::cuda::DilateGrid<nanovdb::ValueOnIndex, BuilderResource> dilateOp(
59+
grid, stream);
5860
dilateOp.setOperation(nanovdb::tools::morphology::NN_FACE_EDGE_VERTEX);
5961
dilateOp.setChecksum(nanovdb::CheckMode::Default);
6062
dilateOp.setVerbose(0);

src/fvdb/detail/ops/BuildFineGridFromCoarse.cu

Lines changed: 32 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
// Copyright Contributors to the OpenVDB Project
22
// SPDX-License-Identifier: Apache-2.0
33
//
4+
#include <fvdb/BuilderResource.h>
45
#include <fvdb/GridBatchData.h>
56
#include <fvdb/detail/GridBatchDataFactory.h>
67
#include <fvdb/detail/ops/BuildFineGridFromCoarse.h>
@@ -305,24 +306,35 @@ dispatchFineIJKForCoarseGrid<torch::kPrivateUse1>(const GridBatchData &batchHdl,
305306

306307
void *dTempStorage = nullptr;
307308
size_t tempStorageBytes = 0;
308-
cub::DeviceSegmentedReduce::Sum(dTempStorage,
309-
tempStorageBytes,
310-
mask.value().jdata().const_data_ptr<bool>(),
311-
maskCounts,
312-
deviceNumSegments,
313-
beginOffsets,
314-
endOffsets,
315-
stream);
316-
cudaMallocAsync(&dTempStorage, tempStorageBytes, stream);
317-
cub::DeviceSegmentedReduce::Sum(dTempStorage,
318-
tempStorageBytes,
319-
mask.value().jdata().const_data_ptr<bool>(),
320-
maskCounts,
321-
deviceNumSegments,
322-
beginOffsets,
323-
endOffsets,
324-
stream);
325-
cudaFreeAsync(dTempStorage, stream);
309+
C10_CUDA_CHECK(
310+
cub::DeviceSegmentedReduce::Sum(dTempStorage,
311+
tempStorageBytes,
312+
mask.value().jdata().const_data_ptr<bool>(),
313+
maskCounts,
314+
deviceNumSegments,
315+
beginOffsets,
316+
endOffsets,
317+
stream));
318+
319+
// Route the CUB scratch through the builder resource rather than bare
320+
// cudaMallocAsync, so it shares torch's pool instead of partitioning VRAM against
321+
// it (same rationale as the nanoVDB builders -- see fvdb/BuilderResource.h).
322+
auto &resource = nanovdb::cuda::default_resource<BuilderResource>();
323+
dTempStorage = resource.allocate_async(
324+
tempStorageBytes, BuilderResource::DEFAULT_ALIGNMENT, stream);
325+
326+
C10_CUDA_CHECK(
327+
cub::DeviceSegmentedReduce::Sum(dTempStorage,
328+
tempStorageBytes,
329+
mask.value().jdata().const_data_ptr<bool>(),
330+
maskCounts,
331+
deviceNumSegments,
332+
beginOffsets,
333+
endOffsets,
334+
stream));
335+
336+
resource.deallocate_async(
337+
dTempStorage, tempStorageBytes, BuilderResource::DEFAULT_ALIGNMENT, stream);
326338
}
327339

328340
for (const auto deviceId: c10::irange(c10::cuda::device_count())) {
@@ -424,7 +436,8 @@ fineGridHandleFromCoarseCUDA(const GridBatchData &coarseBatchHdl,
424436
TORCH_CHECK(grid, "Grid is null");
425437
nanovdb::GridHandle<TorchDeviceBuffer> handle;
426438
for (int p = 0; p < nPasses; p += 1) {
427-
nanovdb::tools::cuda::RefineGrid<nanovdb::ValueOnIndex> op(grid, stream.stream());
439+
nanovdb::tools::cuda::RefineGrid<nanovdb::ValueOnIndex, BuilderResource> op(
440+
grid, stream.stream());
428441
op.setChecksum(nanovdb::CheckMode::Default);
429442
op.setVerbose(0);
430443
handle = op.getHandle(guide);

0 commit comments

Comments
 (0)