Skip to content

Memory planner misses reusable leading arena gaps #10800

Description

@teerthsharma

Problem

find_value_alloc_offset currently considers gaps between coalesced live memory
blocks and otherwise appends after the final block, but it does not consider the
free prefix [0, first_live_block.start). The one-live-block fast path always
appends as well.

That prefix is a valid candidate under the planner's existing lifetime
invariant: if all current live byte ranges start above zero, placing a fitting
value in the prefix cannot overlap any live value.

Minimal example

For values allocated in descending size order:

Value Size Inclusive lifetime Current offset Prefix-aware offset
A 100 [0,1] 0 0
B 80 [1,2] 100 100
C 60 [2,3] 180 0

When C is placed, B is its only live block at [100,180). The current path
appends C and produces a 240-byte arena; considering the leading gap produces a
180-byte arena while preserving all overlap constraints.

Suggested localized change

Keep the current descending-size allocation and block coalescing. For every
non-empty live-block set:

  1. Use the final block end as the append fallback.
  2. Seed the best-fit candidate with offset zero when the leading gap fits.
  3. Let a strictly smaller fitting internal gap replace it.

Using a strict comparison also makes equal-size gaps retain the lower prefix
offset. This does not change the planner's asymptotic complexity or introduce a
new allocation subsystem.

Local validation

Tested locally at b97724553d890bd4dfdc3d56c89094ba1323f4d8 on Windows x64,
MSVC 19.44, Intel Core i7-14700HX:

  • A focused regression test first failed with C at 180 and arena size 240, then
    passed with C at 0 and arena size 180 after the localized change.
  • Final memory-planner tests: 23/23 passed.
  • Relevant planner/subgraph CTest group: 4/4 passed.
  • A public-API resize/reduce runtime witness successfully completed create,
    reshape, setup, and two invokes. Workspace fell from 144 MiB + 32 B to
    112 MiB + 32 B; allocator peak fell by exactly 32 MiB; output hashes matched.
  • Existing subgraph MobileNet benchmarks reported:
    • FP32 MobileNet V1: 23.862980 -> 22.331730 MiB (1.531250 MiB / 6.42% lower).
    • FP16 MobileNet V1: 21.255882 -> 19.916039 MiB (1.339844 MiB / 6.30% lower).
    • The other seven MobileNet cases had no peak-memory change.
  • In 30 randomized baseline/candidate pairs with 50 invokes per process, those
    memory deltas were identical in every pair. Invocation-time bootstrap
    intervals included 1.0, so I am not claiming a speed change.

Would maintainers be open to a small PR implementing this prefix candidate and
the three focused planner tests?

— Teerth Sharma

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions