Skip to content

refactor(flowcontrol): remove the queue plugin registry#2036

Merged
shmuelk merged 3 commits into
llm-d:mainfrom
LukeAVanDrie:flowcontrol/remove-queue-registry
Jul 19, 2026
Merged

refactor(flowcontrol): remove the queue plugin registry#2036
shmuelk merged 3 commits into
llm-d:mainfrom
LukeAVanDrie:flowcontrol/remove-queue-registry

Conversation

@LukeAVanDrie

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

With flow control unified on a single priority queue (#1831), the queue-selection machinery no longer selects anything. This removes it:

  • Delete the string-keyed queue registry/factory (MustRegisterQueue, NewQueueFromName, RegisteredQueueName, the init() self-registration) and construct the priority queue directly via queue.New.
  • Remove the per-band Queue config field, the WithQueue option, and the defaultQueue default. Queues are now a pure internal detail, not user-selectable.
  • Remove the vestigial Name() from the queue contract; its only consumer was a queueType log field that is constant given one implementation.
  • Consolidate the queue tests into priorityqueue_test.go (table-driven ordering, focused TestPriorityQueue_* functions); delete functional_test.go.
  • Drop an unreachable nil-OrderingPolicy guard in buildFlowComponents; the provisioning-failure tests now trigger the real ErrPriorityBandNotFound, which exercises the same lease-rollback and error-scoping paths.

Which issue(s) this PR fixes:

Fixes #2021

Release note:

NONE

The string-keyed queue factory was plugin-registration scaffolding left
over from before ordering/fairness policies moved to the EPP framework
registry. Queues were never promoted to plugins, and with a single queue
type the indirection selects nothing.

- Delete the factory (RegisteredQueueName, QueueConstructor,
  RegisteredQueues, MustRegisterQueue, NewQueueFromName) and the
  init()-time self-registration; add a direct queue.New(policy).
- Remove the per-band Queue config field, the WithQueue option, and the
  defaultQueue default. Queues are an internal detail, not user-selectable.
- buildFlowComponents constructs the queue directly and guards against a
  nil ordering policy (defense in depth: a corrupted config would
  otherwise panic when the queue first compares items).
- Drive the queue conformance suite and benchmarks directly instead of
  ranging over the registry.

Signed-off-by: Luke Van Drie <lukevandrie@google.com>
Follow-on cleanups after removing the queue plugin registry:

- Remove the now-vestigial Name() from the queue contract. Its only
  consumer was a "queueType" log field that is a constant given the
  single queue implementation. Drop it from the interface, the
  implementation, the accessor, the mocks, and the two log sites.
- Consolidate the queue tests into priorityqueue_test.go and delete
  functional_test.go: construct via queue.New directly instead of the
  registry-era constructor indirection, table-drive the ordering cases,
  and split the suite into focused TestPriorityQueue_* functions.
- Drop the nil-OrderingPolicy guard in buildFlowComponents. It was
  unreachable (config validation guarantees a non-nil policy) and only
  existed to give the provisioning-failure tests a trigger. Those tests
  now use ErrPriorityBandNotFound, a real failure mode that exercises
  the same lease-rollback and error-scoping paths.

Signed-off-by: Luke Van Drie <lukevandrie@google.com>
@LukeAVanDrie
LukeAVanDrie requested review from a team and shmuelk as code owners July 15, 2026 19:21
@LukeAVanDrie
LukeAVanDrie requested review from ahg-g and liu-cong July 15, 2026 19:21
@github-actions github-actions Bot added kind/cleanup size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jul 15, 2026
@shmuelk

shmuelk commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Nice straight forward cleanup.

Two nits:

  1. As the PriorityQueue is no longer a plugin, it should probably be moved to be with the rest of the flow control code under pkg/epp/flowcontrol.
  2. The name PriorityQueue is somewhat confusing as flow control has priorityBands. The PriorityQueue queues based on the OrderingPolicy. Perhaps naming it OrderedQueue or RankedQueue (then I would change the OrderingPolicy to RankingPolicy as well) ?

I am happy for these nits to be dealt with in a follow up PR. @LukeAVanDrie your choice.

@LukeAVanDrie

Copy link
Copy Markdown
Contributor Author

Two nits:

  1. As the PriorityQueue is no longer a plugin, it should probably be moved to be with the rest of the flow control code under pkg/epp/flowcontrol.
  2. The name PriorityQueue is somewhat confusing as flow control has priorityBands. The PriorityQueue queues based on the OrderingPolicy. Perhaps naming it OrderedQueue or RankedQueue (then I would change the OrderingPolicy to RankingPolicy as well) ?

I am happy for these nits to be dealt with in a follow up PR. @LukeAVanDrie your choice.

Thanks for the review!

Regarding the first point, I agree that moving the file makes sense, but I will defer that to a follow-up PR to keep this specific cleanup focused as you suggested.

On the second point about naming, I am honestly a bit torn. I definitely understand your point about the potential confusion since flow control already uses the term priorityBands. On the other hand, PriorityQueue is the standard, idiomatic name for this specific data structure, so moving away from it might also cause confusion for those expecting standard nomenclature.

I am not completely sure which tradeoff is better. Maybe we can ask in the Slack to get a few more opinions on whether to keep it as PriorityQueue or change it to OrderedQueue or RankedQueue? Let me know what you think! I would hold on the rename regardless until after the next release though as it is not particularly urgent.

Signed-off-by: Luke Van Drie <lukevandrie@google.com>
@shmuelk
shmuelk merged commit 0511591 into llm-d:main Jul 19, 2026
32 checks passed
ahg-g pushed a commit that referenced this pull request Jul 23, 2026
#2146)

With the queue plugin registry removed (#2036), the priority queue is a
plain internal component of flow control, not a plugin. Move it from
pkg/epp/flowcontrol/framework/plugins/queue to pkg/epp/flowcontrol/queue,
deleting the now-empty pkg/epp/flowcontrol/framework tree so the only
"framework" package left is the plugin SDK at pkg/epp/framework.

Update the flow-control README pillars that still described the deleted
sub-package and the long-removed QueueCapability system.

Signed-off-by: Luke Van Drie <lukevandrie@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/cleanup size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

flowcontrol: remove the vestigial per-band queue selection

2 participants