Skip to content

Memory Budgeting & Resource Unloading

Jump to bottom
Vitaly Popuzin edited this page Dec 13, 2023 · 6 revisions

Memory Budgeting

Memory budgeting in Unity is essential for maintaining system stability and preventing memory overflow and crashes by keeping memory usage within set limits. It ensures smooth application performance across various platforms with different hardware capacities. This approach also aids in efficient resource management, allowing for strategic asset loading and unloading decisions.

In the project, central component of the memory budgeting system is MemoryBudgetProvider. It provides information about relation of currently used memory to the total system memory. It has following dependecies:

  1. IProfilingProvider provides information about current memory usage by reading it from respective Unity built-in ProfilerRecorder
  2. ISystemMemory provides information about total available system memory. Polymorphism should be used to access data on different devices.
  3. MemoryBudgetProvider implements IConcurrentBudgetProvider and uses data from IProfilingProvider to understand its relation to the total system memory from ISystemMemory and relates it to the respective threshold. It will return false for TrySpendBudget() call if the memory usage is in the Full threshold (defined by settings as a certain percentage of total system memory).

Usage in Loading scenario

Similarly to other budget providers, this MemoryBudgetProvider then is used by different systems that are potentially can lead to memory consumption. Such systems prevent loading flow when the budget cannot be spend. Examples could be found in CreateGltfAssetFromAssetBundleSystem and DeferredLoadingSystem.

---
title: Memory Budgeting for Loading
---
classDiagram

class ClientSystem["System that utilize memory budget"]
class IConcurrentBudgetProvider {+TrySpendBudget() bool }    

ClientSystem --> IConcurrentBudgetProvider : Uses TrySpendBudget() in its own loading logic
IConcurrentBudgetProvider <|-- MemoryBudgetProvider  : implements
MemoryBudgetProvider --> IProfilingProvider : get currently used memory
MemoryBudgetProvider --> ISystemMomery : get total available memory
Loading

Resources Unloading

In scenarios where memory consumption approaches or exceeds critical levels, it becomes imperative to alleviate this pressure by strategically unloading resources that are no longer necessary. The objective of resource unloading is to free up memory space, making it available for new or more relevant resources required for the current user experience. This is achieved by several components:

  1. ReleaseMemorySystem continuously monitors memory usage (provided by MemoryBudgetProvider). When the threshold, defined in the settings, is reached or exceeded, it triggers the resource unloading mechanism. This mechanism should focus on resources that have minimal impact on user experience or those that are easily reloadable when needed again. Currently, it focuses on cleaning most resource expensive caches and pools by calling CacheCleaner class.
  2. CacheCleaner: This class encapsulates the logic for resource clearance. It employs a visitor pattern, wherein various caches and pools are registered. Upon being triggered by the ReleaseMemorySystem, the CacheCleaner executes the unloading of registered pools and caches in a predefined sequence and chunk sizes.
  3. Disposal in Pools and Caches: Each pool and cache is designed with unloading functionalities. Caches follow an LRU-based approach to identify and unload less needed items. PriorityQueue is used in most cases here. Pools, derived from Unity's standard pools, incorporate a unique chunk-based clearance method. All unloading operations are conducted within the constraints of FrameTime budgeting, ensuring that these unloading activities are performed efficiently, maintaining the system's optimal performance. Unloading initiate disposal of respective assets.

Note: Currently, unloading is done in simple and straightforward manner and effective for now, however, there's potential for future enhancements. These improvements could involve more advanced unloading strategies or dynamic adjustments to better suit changing system demands, as well as providing different clearance strategies for different memory usage thresholds.

Below is the simplified class diagram of the Resources Unloading, underlaying main elements of this system. As you can see on it, one of the critical parts of this process is a proper referencing and dereferencing of dependent assets.

---
title: Resources Unloading and Dereferencing
---
classDiagram
    ReleaseMemorySystem --> MemoryBudgetProvider : uses to understand current memory threshold
    MemoryBudgetProvider : +GetMemoryUsageStatus()
    ReleaseMemorySystem --> CacheCleaner : call Unload() when certain memory usage exceeded

    class CacheCleaner {
        +Unload()
        +Register(AssetBundleCache)
        +Register(GltfContainerCache)
        +Register(...)
    }

    CacheCleaner --> TexturesCache : Unload
    CacheCleaner --> WearableAssetsCache : Unload
    CacheCleaner --> WearableCatalog : Unload
    CacheCleaner --> GltfContainerAssetsCache : Unload
    CacheCleaner --> AssetBundleCache : Unload

    %% Cache Classes
    class TexturesCache{ +Unload(amount) }
    class WearableAssetsCache{ +Unload(amount) }
    class WearableCatalog{ +Unload(amount) }
    class GltfContainerAssetsCache { +Unload(amount) }
    class AssetBundleCache{ +Unload(amount) }

    %% Other Associations
    TexturesCache --> Texture2D : Dispose cached assets
    WearableAssetsCache --> CachedWearable : Dispose cached assets
    CachedWearable --> WearableAsset : Dereference
    WearableCatalog --> WearableAsset : Dispose dereferenced
    WearableAsset --> AssetBundleData: Dereference on disposal
    class WearableAsset {
        +Dispose()
        +AddReference()
        +Dereference()
    }
    GltfContainerAssetsCache --> GltfContainerAsset : Dispose cached assets
    GltfContainerAsset : +Dispose()
    GltfContainerAsset --> AssetBundleData : Dereference on disposal
    AssetBundleCache --> AssetBundleData : Dispose dereferenced
    class AssetBundleData { 
        +Dispose()
        +AddReference()
        +Dereference()
    }
Loading

Debug and Profiling

There are several tools, that help to debug and profile this process

Profiling Counters and Markers

  • Mentioned resources and their amount in caches is measured by profiling counters, that can be selected in custom profiling module. Read Viewing the counters in the Profiler Window on the official package page for more details.
  • Update call of each ECS System is covered by the respective profiling marker, so it is easy to find RealseMemorySystem by name in the recorded data of the CPU profiling module.

Debug view

Debug panel is supported by information about currently used memory and 2 thresholds (WARNING and FULL), as well as buttons to simulate respective memory usage.

Tests

For getting more insights on how referencing/dereferencing works in isolated environment, please refre CacheCleanerIntegrationTests.

Further improvements

As it was mentioned, this system is not finished and can be improved in various ways:

  • currently, it doesn't include pools for Transforms, Colliders, Primitives and so.
  • heavy DTO (for Scenes and Wearables) can be considered for clean up
  • information about AssetBundle size can be used for better release memory strategy (requires adjustment on AssetBundleConverter side)
  • more advanced unloading strategies or dynamic adjustments to better suit changing system demands, as well as providing different clearance strategies for different memory usage thresholds.

Clone this wiki locally