perf(flowcontrol): index active queues so dispatch scans O(active) flows

[LukeAVanDrie]

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

Every dispatch cycle, the fairness policy's Pick calls PriorityBandAccessor.IterateQueues, which snapshotted every registered queue in the band (empty ones included) into a freshly allocated slice under the registry read lock. Per-dispatch cost was O(registered flows) in CPU and allocation (94-98% of all allocation in the flow-control benchmarks) and throughput decayed superlinearly under flow churn as registered-but-idle flows accumulated between GC cycles. The 1 ms dispatch ticker pays this cost even at zero traffic.

Each priority band now maintains a lock-free index of its non-empty queues, updated on empty<->non-empty transitions inside the queue's existing stats critical section (transitions are serialized per queue by the queue mutex; the index is a sync.Map, so the update never touches the registry mutex). IterateQueues ranges the index directly: no registry lock, no allocation, cost proportional to flows that currently hold items.

This narrows IterateQueues to active (non-empty) queues, matching its documented contract ("each active Flow"). There are three production callers. globalstrict.Pick already skips empty queues and the processor's cleanup/drain sweeps are no-ops on them. The program-aware LAS policy, however, used empty-queue visits to drive attained-service decay, so the second commit makes decay lazy: it is folded in from the decay anchor whenever a program's service is read or accumulated, independent of visits. This changes lasDecayFactor to a per-second rate (it was per-Pick, so its effective half-life varied with dispatch frequency) and the default config now uses a 60s half-life. Flagging that for LAS reviewers specifically.

The view is eventually consistent; callbacks must tolerate a visited queue reporting Len() == 0, which they already did under the snapshot implementation (a snapshotted queue could drain before the callback ran).

Index deactivation uses CompareAndDelete rather than a plain delete: a cleanup-sweep worker can drain a queue through a handle resolved before deleteFlow removed it, and if a successor queue was registered under the same flow ID in the interim, an unconditional delete would hide the live successor from dispatch and future sweeps. A regression test covers this interleaving.

Note for reviewers: iterating under the registry read lock instead was considered and rejected. The cleanup sweep calls registry.ManagedQueue() (which takes the read lock) from inside the iteration callback, so yielding under the lock is a recursive RLock that deadlocks when a writer is queued.

Benchmarks (M4 Pro, -benchtime=3s):

Benchmark	Before	After
PerformanceMatrix L=1/P=1/F=5000/W=5000	30.7k d/s, 42.2KB/op	277k d/s, 1.8KB/op
TopologyChurn	16.7k d/s, 83KB/op	260k d/s, 1.8KB/op
FullPath (flow churn)	2.3k d/s, degrading with run length	118k d/s, flat

Which issue(s) this PR fixes:

Part of #1187

Release note:

Flow control dispatch cost no longer scales with the number of registered flows: fairness policies scan only non-empty queues, removing a per-dispatch allocation proportional to flow count. The program-aware policy's attained-service decay is now wall-clock based: `lasDecayFactor` is a per-second rate (previously per-Pick), `lasHalfLifeSeconds` defaults to 60, and decay no longer pauses while requests are in flight.

[LukeAVanDrie]

/assign @shmuelk

After this, it's probably not worth making any more targeted perf improvements as we are definitely not a bottleneck anymore. My main goal was to reduce unnecessary temp allocs which create GC pressure which can bleed out and affect perf of other parts of EPP.

[LukeAVanDrie]

Found some issues, converting this back to draft while I rework this.

edit: Done and moved back out of DRAFT

[LukeAVanDrie]

cc: @D-Sai-Venkatesh and @praveingk

This perf improvement necessitates changing an aspect of the program aware fairness plugin. Would like your feedback on this to make sure my changes are aligned with your intentions for how decay works.

[D-Sai-Venkatesh]

Hey @LukeAVanDrie, sure will take a look and post our comments here.

[LukeAVanDrie]

Hey @LukeAVanDrie, sure will take a look and post our comments here.

Thanks, no urgency here as this is not a release blocker for v0.10. I just fixed up the go-bench recently, got fresh results, and sent a fix out for the worst cpu and mem bottlenecks.

[shmuelk]

@LukeAVanDrie Please rebase. It will fix the CI failure

[LukeAVanDrie]

cc: @jtechapps

I have my micro-bench framework for the flow control runtime itself, but I was wondering if I am able to use your new perf tooling to investigate overall EPP perf impact before merging (I know it's a nightly).