Skip to content

[YUNIKORN-3384] Add partition accounting property fuzzer - #1131

Open
tigerquoll wants to merge 1 commit into
apache:masterfrom
tigerquoll:test/core-fuzzer
Open

[YUNIKORN-3384] Add partition accounting property fuzzer#1131
tigerquoll wants to merge 1 commit into
apache:masterfrom
tigerquoll:test/core-fuzzer

Conversation

@tigerquoll

@tigerquoll tigerquoll commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

What this is

A property fuzzer for the core's partition-level accounting. The shim cache property fuzzer (YUNIKORN-3373) guards shim bookkeeping; nothing guarded the core's. This is the structural guard for that class from the core side.

JIRA: YUNIKORN-3384

Design

Seeded, weighted random operation sequences against a real PartitionContext with a small queue hierarchy, driven through the same entry points the RM uses: add/remove node (removal with live allocations included), add/remove app, add ask (regular and required-node), regular and reserved scheduling cycles, and allocation release with each termination type. Placeholders and the replacement flow are out of scope.

After every operation it checks:

  • Q1 each queue's allocated == sum of its children (recursive)
  • Q2 each leaf queue's allocated == sum of its apps' allocated
  • Q3 usage never negative, never over a set maximum
  • Q4 user/group tracked usage == sum of owned apps' allocated
  • Q5 each node's allocated == sum of allocations on it
  • Q6 queue pending == sum of app pending asks
  • R1 every reservation references a live app + ask + node (both ends)

plus a running allocated-minus-released model cross-check. A failure reports the seed, step, and last operations for replay; the run logs its coverage so a fuzzer that stops exercising the interesting paths is visible.

Verification

Green on master across all seeds — this is a guard, not necessarily a finder. Its teeth are proven by mutation testing: five deliberate accounting breaks each fire the expected invariant (e.g. dropping DecAllocatedResource fires Q2/Q3/model; dropping the parent recursion fires Q1; breaking the user tracker fires Q4). All mutations reverted — the committed diff is test-only. Runtime ~4s.

Two real core defects were found while developing this and are filed separately: YUNIKORN-3379 and YUNIKORN-3380. Their trigger operations are deliberately excluded so this fuzzer stays green and mergeable; a follow-up can add them once the bugs are fixed.


Generated by the Author with assistance from Claude Code.

Drive seeded, weighted random sequences of the operations a resource manager
triggers against a real PartitionContext and check the accounting invariants
after every single operation.

Operations: node add and remove (including removal of a node that still holds
allocations), application add and remove, request add (regular and required
node), regular and reserved scheduling cycles, and allocation release. Every
termination type is released except the placeholder replacement: placeholders,
and with them the replacement flow, are out of scope.

Invariants: parent queue usage is the sum of its children (Q1), leaf queue usage
is the sum of its applications (Q2), queue usage is never negative and never
over the maximum (Q3), the user and group trackers match the applications a user
owns (Q4), node usage is the sum of the allocations on it (Q5), queue pending is
the sum of the pending below it (Q6), and every reservation references a live
application, request and node from both ends (R1). A running total of the
allocations placed minus released cross-checks the partition totals.

A failure reports the seed, the step and the last operations so the sequence can
be replayed. The run logs what the sequence covered so a fuzzer that stops
exercising the interesting paths is visible.

Signed-off-by: Tigerquoll <tigerquoll@outlook.com>
@tigerquoll
tigerquoll marked this pull request as ready for review August 30, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant