Add runtime region streaming - #1004
Conversation
FULL_REBUILD is a superset mask, so 'p_flags & FULL_REBUILD' was true for any non-zero flag and every maps_changed signal regenerated and recompiled the whole shader. Add a dedicated SHADER_REBUILD bit and test that instead.
a122ced to
096d2a1
Compare
Loads regions around the clipmap target on background threads and evicts them as the target moves away, so memory scales with the loaded area instead of the whole terrain. Off by default; no effect in the editor or with streaming disabled. While streaming, regions check out stable slots from fixed-capacity texture arrays instead of rebuilding the arrays on every membership change. Inserted regions update collision in place and refresh mesher AABBs. Modified regions save back to the data directory before eviction. Region size is adopted from the first region file on disk. New inspector properties: streaming_enabled, streaming_shape (square or circle), streaming_distance, streaming_slots, streaming_concurrent_loads, streaming_loads_per_frame. Documented in doc_classes. Headless test in project/tests/test_region_streaming.gd.
096d2a1 to
67062f8
Compare
Type the streaming_shape alias with StreamShape via the same file scope using declaration Terrain3DCollision has. Document node streaming members as aliases of Terrain3DStreamer like the collision aliases, with the full descriptions on the streamer class. Drop a leftover include and fix a comment that still referenced the pre-refactor function name.
|
Great, will gladly test on Mac once I have some free time. Although being pretty critic, you should add a full disclosure on how much AI was used and where just |
TokisanGames
added
enhancement
streaming_mode chooses what eviction does with a region's data. Disk (default) saves modified regions and drops them. RAM Resident parks the body in a cache so only the GPU layers follow the loaded area: revisits reinsert without disk IO or decode, and spare load budget prefetches the rest of the world into the cache in the background. Modified cached regions save when leaving RAM mode or disabling streaming. Also extends the test with live enable/disable toggle coverage in both directions and a RAM mode edit round trip (27 asserts).
streaming_slots was allocation-time only: the pooled arrays are built once at capacity, so a live change did nothing and raising the distance past a stale pool starved insertions. Changing it while streaming now rebuilds the terrain data at the new capacity, like toggling streaming does. Shape, distance, mode and the frame budgets already applied live through the state machine; the test now covers all of them switching at runtime, including hysteresis behavior on shrink (42 asserts).
free_editor_textures (on by default) drops the source texture assets at runtime once the texture arrays are generated. Reinitializing the assets on a live change (streaming toggle, slot capacity, data directory) then regenerated the arrays from the empty source list, blanking the terrain to the checkered fallback permanently. Assets now only initialize when the resource is fresh; a live reinitialization keeps the generated arrays.
|
Testing on my macbook air m3 8gb of ram rn in 4.7.1 rc 2 i found some funky behaviour, though the ram savings are so big: ●When you set distance to 15 , in the places the terrain region don't load back up and stay as if there wasn't a region done there. 
●Moreover , as expected since the load /unload is executed syncronomously , there are stutters( specially when the distance is at minimum for obvious reason) ●Adding up to my first statement, when you set the minimum distsnce to 1 and unload regions , sometimes the instances don't disappear along with it's region. 
●Atm from what i can tell this is only working in game only as the editor does not react to changing the streaming values. ● Aside from the distance stuff and the regions sometimes not loading back up, region streaming has a base cost of 10% in my computer, which i suppose it's fine and won't matter much when runtime virtual texture is added , tough i thought it would give more fps since it's unloading far away regions.. Overall i think this is looking pretty good( though i insist to put more effort in reviewing the AI code for safety measures). Also my ram went from like 910 to 220mb when i set distance to 4, which is a great save( though when setting the distance to 1 it has 190mb ) |
Actually, the RVT would make the terrain rendering cheaper, which would in turn make the streaming overhead comparatively larger. I'd be kinda curious to know where the extra frame time is being spent, a visual profiler screenshot would help in that regard. |
A load distance the slot pool cannot hold permanently starved loading: the pool filled, nothing beyond it inserted, nothing evicted, and the world was left with holes (reported on macOS at distance 15 with the default 121 slots, which needs 961). The pool must hold every region within distance + 1 (hysteresis retains the trailing ring): (2 * (distance + 1) + 1)^2 for a square shape, which is exactly why the default is 121 for distance 4. distance is now the primary knob and slots follows it: raising distance grows the pool to fit, capped at the 1024 slot ceiling (a square shape tops out at distance 14). Explicitly lowering slots below what the current distance needs pulls distance back down with a warning. slots also floors at 25, the smallest pool that can hold distance 1, so it can never be set too low to stream. The slots and distance inspector descriptions explain the relationship and what a given distance covers. get_required_slots() and get_max_distance() are exposed for tooling.
Region instance data streams with the region, but the streamer rebuilt every multimesh in the world on each region change: it called the instancer with the rebuild flag, which destroys and recreates all instances everywhere. On an instanced terrain that turned each region load or eviction into a global hitch (reported as multi-second stalls filling the world and stutter while driving; identical in disk and RAM mode because the cost is the instancer, not IO). Insertion now builds only the landed region's instances, and eviction frees the departing region's instances directly (the deferred per-region update path skips a region already gone from the data, which also left evicted instances rendering). Cost scales with one region instead of the whole world. Adds destroy_mmis_by_region() and get_mmi_region_count(), and extends the streaming test with instanced regions.
2ef8e84 to
5c9e729
Compare
What do you mean by that? That rvt wouldn’t save fps with region streaming on? |
I would expect it to save FPS just the same. It's just that if you experience a +10% frame time increase with streaming on a 100% frame time terrain, and RVT lowers it to say 50%, then the 10% streaming overhead would be a larger chunk of the total cost (50% + 10%). At least this if it's a GPU bottleneck. If streaming is CPU bottlenecked then that'd be different. |
Eviction wrote a modified region back to the data directory before dropping it, so a running game silently persisted runtime edits and, against a shared data directory, could overwrite an edit made to the same file by another process. Streaming is now a read-only view of the data on disk: eviction drops a region without writing it back, and the RAM cache flush no longer saves either. The new streaming_persist_edits property (Terrain3DStreamer.persist_edits, off by default) restores the write-back for worlds that want to persist runtime edits; otherwise callers save explicitly with save_directory.
4 participants
Partially addresses #491.
This PR adds optional runtime region streaming. Regions around the clipmap target are loaded on background threads and evicted as the target moves away, so memory scales with the loaded area instead of the whole terrain. Our test world has 700 regions at 512px and costs about 2.3 GB RAM + 2.3 GB VRAM fully resident. With streaming at distance 4 it stays around 170 MB, and in play you can't tell the difference.
How it works
Normally region_id doubles as the texture array layer and the position in _region_locations, so any membership change re-enumerates ids and rebuilds the full texture arrays. That makes add/remove at runtime far too expensive to do per frame. While streaming, a region checks out a stable slot from fixed-capacity pooled arrays instead: the region map stores slot + 1, add/evict touch one texture layer and one region map entry, and the arrays are allocated once. The shader's location table is slot-indexed through get_shader_region_locations().
The runtime logic lives in its own subsystem class, Terrain3DStreamer, owned by Terrain3D the same way Terrain3DCollision and Terrain3DInstancer are, with the streaming_* node properties forwarding as aliases like the collision settings do. Terrain3D itself only creates it, calls step() once per physics frame, and exposes the aliases. The slot pool stays in Terrain3DData since it manipulates the region containers and generated texture arrays directly.
The streamer runs a small state machine each physics frame:
Scope and safety
Off by default, and is_streaming_active() also checks !Engine.is_editor_hint(), so the editor always keeps full residency and behaves exactly as before. With streaming disabled the load path is untouched and the demo runs clean on the new build.
New inspector group Streaming: streaming_enabled, streaming_shape (square or circle loaded area), streaming_mode (Disk, or RAM Resident where region bodies stay cached in memory and only the GPU layers follow the loaded area, with background prefetch of the rest of the world), streaming_distance, streaming_slots, streaming_concurrent_loads, streaming_loads_per_frame. Everything can be toggled at runtime; the test covers live enable/disable in both directions. All documented in doc_classes. There's also get_streaming_stats() for debug overlays.
The first commit is a separate fix that this feature exposed. Terrain3DMaterial::update() tested p_flags & FULL_REBUILD, which matches every non-zero flag because FULL_REBUILD is a superset mask, so every maps_changed regenerated and recompiled the whole shader. You rarely notice it classically since maps_changed only fires on edits, but with per-frame region traffic it turned into a hitch storm. A dedicated SHADER_REBUILD bit fixes it. Worst streaming frame in our churn test went from ~900 ms to ~17 ms.
Testing
Possible improvements
Things I left out to keep this reviewable, can follow up on any of them:
One behavior worth knowing: streamed regions get set_version() on insertion like load_region() does, so files from an older data version are marked modified and get rewritten (upgraded) on eviction. That means the first drive across an old world rewrites its regions one by one. It self-heals after one pass, but if you'd rather not have streaming upgrade files implicitly I can skip the version bump instead.
Disclosure
Parts of this PR were developed with AI assistance. All of it was designed, reviewed and tested by hand: the feature has been soak tested in our game for extended play sessions, and the headless test suite plus the demo regression check above were run against every revision.