Merge branch 'main' into ashar/hybrid-concurrency-detector

Author: LukeAVanDrie

pkg/epp/flowcontrol/README.md

@@ -99,13 +99,14 @@ their justifications, please refer to the detailed documentation within the rele
     processing loop. It manages a worker that distributes incoming requests, apply policies, and dispatch
     requests to the backends. Its design focuses on high throughput and backpressure.
 
-2.  **Pluggable `Policy` Framework (`./framework`)**: This defines the core interfaces for all pluggable logic. It
-    features a two-tier policy system for `Fairness` (decisions *between* different flows) and `Ordering`
-    (decisions *within* a single flow) logic, covering both request dispatch and displacement.
-
-3.  **Extensible `SafeQueue` System (`./framework`)**: This defines the `framework.SafeQueue` interface for
-    concurrent-safe request storage. It uses a `QueueCapability` system that allows for diverse and extensible queue
-    implementations (e.g., FIFO, Priority Heap) while maintaining a stable interface.
+2.  **Pluggable `Policy` Contracts (`/pkg/epp/framework/interface/flowcontrol`)**: The plugin SDK defines the core
+    interfaces for all pluggable logic. It features a two-tier policy system for `Fairness` (decisions *between*
+    different flows) and `Ordering` (decisions *within* a single flow) logic, covering both request dispatch and
+    displacement.
+
+3.  **The `SafeQueue` (`./contracts`, `./queue`)**: The `contracts.SafeQueue` interface defines concurrent-safe
+    request storage, and `./queue` provides the priority-queue implementation whose dispatch order is determined by
+    the flow's configured `OrderingPolicy`.
 
 4.  **The `FlowRegistry` (`./registry`, `./contracts`)**: This is the stateful control plane of the system. It manages
     the configuration and lifecycle of all flows, policies, and queues. It presents a view of its state to the

…ramework/plugins/queue/benchmark_test.go …/epp/flowcontrol/queue/benchmark_test.gopkg/epp/flowcontrol/framework/plugins/queue/benchmark_test.go renamed to pkg/epp/flowcontrol/queue/benchmark_test.go pkg/epp/flowcontrol/framework/plugins/queue/benchmark_test.go renamed to pkg/epp/flowcontrol/queue/benchmark_test.go

@@ -28,7 +28,7 @@ import (
 
 	"github.com/llm-d/llm-d-router/pkg/epp/flowcontrol/contracts"
 	"github.com/llm-d/llm-d-router/pkg/epp/flowcontrol/contracts/mocks"
-	"github.com/llm-d/llm-d-router/pkg/epp/flowcontrol/framework/plugins/queue"
+	"github.com/llm-d/llm-d-router/pkg/epp/flowcontrol/queue"
 	"github.com/llm-d/llm-d-router/pkg/epp/framework/interface/flowcontrol"
 	fwkfcmocks "github.com/llm-d/llm-d-router/pkg/epp/framework/interface/flowcontrol/mocks"
 )

…framework/plugins/queue/priorityqueue.go …g/epp/flowcontrol/queue/priorityqueue.gopkg/epp/flowcontrol/framework/plugins/queue/priorityqueue.go renamed to pkg/epp/flowcontrol/queue/priorityqueue.go pkg/epp/flowcontrol/framework/plugins/queue/priorityqueue.go renamed to pkg/epp/flowcontrol/queue/priorityqueue.go

@@ -30,7 +30,7 @@ import (
 
 	"github.com/llm-d/llm-d-router/pkg/common/observability/logging"
 	"github.com/llm-d/llm-d-router/pkg/epp/flowcontrol/contracts"
-	"github.com/llm-d/llm-d-router/pkg/epp/flowcontrol/framework/plugins/queue"
+	"github.com/llm-d/llm-d-router/pkg/epp/flowcontrol/queue"
 	"github.com/llm-d/llm-d-router/pkg/epp/framework/interface/flowcontrol"
 	"github.com/llm-d/llm-d-router/pkg/epp/metrics"
 )

…work/plugins/queue/priorityqueue_test.go …/flowcontrol/queue/priorityqueue_test.gopkg/epp/flowcontrol/framework/plugins/queue/priorityqueue_test.go renamed to pkg/epp/flowcontrol/queue/priorityqueue_test.go pkg/epp/flowcontrol/framework/plugins/queue/priorityqueue_test.go renamed to pkg/epp/flowcontrol/queue/priorityqueue_test.go

@@ -28,7 +28,7 @@ import (
 	testclock "k8s.io/utils/clock/testing"
 
 	"github.com/llm-d/llm-d-router/pkg/epp/flowcontrol/contracts"
-	"github.com/llm-d/llm-d-router/pkg/epp/flowcontrol/framework/plugins/queue"
+	"github.com/llm-d/llm-d-router/pkg/epp/flowcontrol/queue"
 	"github.com/llm-d/llm-d-router/pkg/epp/framework/interface/flowcontrol"
 	"github.com/llm-d/llm-d-router/pkg/epp/framework/interface/flowcontrol/mocks"
 )

pkg/epp/flowcontrol/registry/managedqueue_test.go

@@ -144,9 +144,14 @@ type SaturationDetector interface {
 // Integration:
 // This policy is called during dispatch decision-making, before a request is allowed to proceed. For each
 // priority band, the returned ceiling is compared against current saturation. If saturation exceeds the
-// ceiling for a given priority, requests at that priority are gated (not dispatched).
-//
-// Conformance: Implementations MUST ensure all methods are goroutine-safe.
+// ceiling for a given priority, requests at that priority are gated (not dispatched). The dispatch loop
+// visits bands from highest to lowest priority and stops at the first gated band; lower bands are not
+// considered on that call.
+//
+// Conformance: Implementations MUST ensure all methods are goroutine-safe. Returned ceilings MUST be
+// monotonically non-increasing in the given priority order (highest priority first): because the
+// dispatch loop stops at the first gated band, a lower band whose ceiling exceeds that of a higher band
+// can be marked open on calls where it is unreachable, starving it.
 type UsageLimitPolicy interface {
 	plugin.Plugin