Skip to content

DAOS: zero-copy data-path counters + cold-buffer warmth guard#5107

Merged
eisenhauer merged 1 commit into
ornladios:masterfrom
eisenhauer:daos-defer-submit-ab
Jul 8, 2026
Merged

DAOS: zero-copy data-path counters + cold-buffer warmth guard#5107
eisenhauer merged 1 commit into
ornladios:masterfrom
eisenhauer:daos-defer-submit-ab

Conversation

@eisenhauer

Copy link
Copy Markdown
Member

Two small additions to DaosWriter, both around the existing MinDeferredSize zero-copy test:

  • Counters — per-rank zero-copy vs copy Put split, emitted as DAOS_ZeroCopy*/DAOS_Copy* timers in profiling.json. Visibility into how much of a workload actually zero-copies.
  • Warmth guard — forces a large buffer that has never recurred to copy rather than zero-copy, since a cold (unregistered) zero-copy loses to a memcpy into the already-registered chunk. Reuse is tracked by address over a short step window. On by default (DAOS_WARMTH_WINDOW=8, 0 disables). Copy to a warm buffer is better than a cold zero-copy put(), so we only try zero-copy Put() if we've seen that user address before.

Add two pieces to DaosWriter, both around the existing MinDeferredSize
zero-copy gate (large deferred array Puts already write zero-copy straight
from the user buffer; small ones copy-and-aggregate):

- Data-path counters: per-rank split of array Puts between the zero-copy
  (External) and copy paths, emitted as DAOS_ZeroCopy*/DAOS_Copy* timers in
  profiling.json (count carried in nCalls, the FreshChunkAllocs idiom). Lets
  us see the actual split on any workload.

- Warmth guard: a large buffer that has never recurred is forced to copy
  instead of zero-copied, since a cold (unregistered) zero-copy loses to a
  memcpy into the already-registered chunk. Reuse is tracked global-by-address
  (matching the VA-keyed MR cache) over a small step window whose depth covers
  multi-buffering. On by default (DAOS_WARMTH_WINDOW=8); env overrides, 0
  disables. Strictly >= prior behavior: catches the never-reused large-buffer
  case, otherwise falls back to today's path; failure mode is a copy. Not yet
  validated at scale.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@eisenhauer eisenhauer merged commit cee34db into ornladios:master Jul 8, 2026
48 checks passed
@eisenhauer eisenhauer deleted the daos-defer-submit-ab branch July 8, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants