Move SLO routing functionality to a single multi-plugin#1780

Closed

BenjaminBraunDev wants to merge 133 commits intokubernetes-sigs:slo-prediction-experimentalfrom

BenjaminBraunDev:slo-routing-single-plugin

Commits on Oct 27, 2025

Update guide to add steps to deploy healthcheck policy for gke (#1475)

rlakhtakia
authored and
BenjaminBraunDev
committed
fix helm chart support for gke v1alpha2. (#1551)

zetxqx
authored and
BenjaminBraunDev
committed
chore: bump sim model server version (#1555)

nirrozenbaum
authored and
BenjaminBraunDev
committed
remove duplicated section in quickstart guide (#1553)

nirrozenbaum
authored and
BenjaminBraunDev
committed
Merge shuffle score pods logic (#1552)

learner0810
authored and
BenjaminBraunDev
committed
Update docs with 1.0 release (#1557)

rahulgurnani
authored and
BenjaminBraunDev
committed
add-envoyproxy-ai-gateway (#1554)

learner0810
authored and
BenjaminBraunDev
committed
fix-import-groups (#1560)

learner0810
authored and
BenjaminBraunDev
committed
chore(deps): bump golang.org/x/sync from 0.16.0 to 0.17.0 (#1549)

dependabot[bot]
authored and
BenjaminBraunDev
committed
chore(deps): bump sigs.k8s.io/controller-tools from 0.18.0 to 0.19.0 (#1548)

dependabot[bot]
authored and
BenjaminBraunDev
committed
fix flake in weighted random picker (#1561)

nirrozenbaum
authored and
BenjaminBraunDev
committed
Main uniquely name crbac (#1564)

Gregory-Pereira
authored and
BenjaminBraunDev
committed
Update priority in EPP flow control from uint to int (#1518)

rahulgurnani
authored and
BenjaminBraunDev
committed
rename inference_model metrics to inference_objective (#1567)

JeffLuoo
authored and
BenjaminBraunDev
committed
added github action to add a hold label when PRs are pushed to branch other than main (#1570)

nirrozenbaum
authored and
BenjaminBraunDev
committed
Refactor LLMRequest: Structured RequestData for Completions & Chat-Completions (#1446)

vMaroon
authored and
BenjaminBraunDev
committed
epp servicemonitor (#1425)

sallyom
authored and
BenjaminBraunDev
committed
remove scheduler epp flowchart (#1573)

kaushikmitr
authored and
BenjaminBraunDev
committed
Fixes to overview.md and inferencemodel.md (#1281)

DamianSawicki
authored and
BenjaminBraunDev
committed
fix prefix plugin unit-test (#1575)

vMaroon
authored and
BenjaminBraunDev
committed
Update the endpoint picker diagram (#1572)

ahg-g
authored and
BenjaminBraunDev
committed
Docs: Updates Intro Diagram (#1577)

danehans
authored and
BenjaminBraunDev
committed
Conformance: Adds v1.0.0 COnformance Report for Kgateway (#1579)

danehans
authored and
BenjaminBraunDev
committed
Updating the the doc site (#1500)

kfswain
authored and
BenjaminBraunDev
committed
feat: Add top-level Flow Controller (#1525)

LukeAVanDrie
authored and
BenjaminBraunDev
committed
chore: remove unneeded logging before returning err (#1544)

phuhung273
authored and
BenjaminBraunDev
committed
update istio version to v1.28 in guide document (#1582)

flpanbin
authored and
BenjaminBraunDev
committed
docs: dashboards README metrics link fix (#1589)

JaredTan95
authored and
BenjaminBraunDev
committed
fix(deps): update prometheus/common to v0.66.1 and prometheus/client_golang to v1.23.1 (#1580)

JeffLuoo
authored and
BenjaminBraunDev
committed
chore(deps): bump github.com/prometheus/client_golang (#1550)

dependabot[bot]
authored and
BenjaminBraunDev
committed
One liner: add COPY api ./api command to resolve import dependency in /pkg/epp/datalayer/factory.go:28:2: no required module provides package sigs.k8s.io/gateway-api-inference-extension/api/v1 (#1585)

davidbreitgand
authored and
BenjaminBraunDev
committed
Enhance pool namespace resolution: use flag if set, else NAMESPACE en… (#1578)

jyizheng
authored and
BenjaminBraunDev
committed
Scrubbing out more InferenceModel references (#1592)

kfswain
authored and
BenjaminBraunDev
committed
cleanup of quickstart and readme (#1588)

nirrozenbaum
authored and
BenjaminBraunDev
committed
Conformance: Adds Report for Kgateway with Agentgateway (#1587)

danehans
authored and
BenjaminBraunDev
committed
refactor: flow control config (#1581)

LukeAVanDrie
authored and
BenjaminBraunDev
committed
chore(deps): bump google.golang.org/grpc from 1.75.0 to 1.75.1 (#1596)

dependabot[bot]
authored and
BenjaminBraunDev
committed
chore(deps): bump github.com/onsi/ginkgo/v2 from 2.25.1 to 2.25.3 (#1597)

dependabot[bot]
authored and
BenjaminBraunDev
committed
Deprecate inferencepool-resources.yaml (#1586)

rahulgurnani
authored and
BenjaminBraunDev
committed
chore(deps): bump the kubernetes group with 6 updates (#1594)

dependabot[bot]
authored and
BenjaminBraunDev
committed
chore(deps): bump google.golang.org/protobuf from 1.36.8 to 1.36.9 (#1595)

dependabot[bot]
authored and
BenjaminBraunDev
committed
Add makefile entries for api-lint (#1384)

rikatz
authored and
BenjaminBraunDev
committed
test: Refactor end to end test code to enable reuse downstream (#1515)

shmuelk
authored and
BenjaminBraunDev
committed
Add BBR user guide, yaml for model-aware routing (#1498)

srampal
authored and
BenjaminBraunDev
committed
fix(conformance): Use pinned version of EPP for conformance test instead of main. (#1262)

zetxqx
authored and
BenjaminBraunDev
committed
Updates gateway manifests for v1 InferencePool (#1603)

danehans
authored and
BenjaminBraunDev
committed
duplicate fix (#1605)

sagiahrac
authored and
BenjaminBraunDev
committed
Docs: Updates kgateway and agentgateway Implementations for v1.0.0 (#1607)

danehans
authored and
BenjaminBraunDev
committed
Fix some markdown formatting errors (#1609)

authored and
BenjaminBraunDev
committed
helm: add NAMESPACE env var to EPP deployment for pool namespace (#1610)

jyizheng
authored and
BenjaminBraunDev
committed
chore: bump sim version (#1612)

nirrozenbaum
authored and
BenjaminBraunDev
committed
add gke monitoring helm support. (#1600)

zetxqx
authored and
BenjaminBraunDev
committed
Rename prefix scorer HashBlockSize to BlockSize (#1613)

Frapschen
authored and
BenjaminBraunDev
committed
Consolidate ha config into a single enableLeaderElection, also fix rolling update stuck bug (#1620)

liu-cong
authored and
BenjaminBraunDev
committed
improve the readable of test-unit make target (#1614)

Frapschen
authored and
BenjaminBraunDev
committed
feat: Adapt flow control to per-request saturation (#1622)

LukeAVanDrie
authored and
BenjaminBraunDev
committed
fix missing GCPBakendPolicy. (#1623)

zetxqx
authored and
BenjaminBraunDev
committed
use replicas field in helm to decide if EPP should run in HA mode (#1628)

nirrozenbaum
authored and
BenjaminBraunDev
committed
Explicitly set pool-namespace flag for wider compatibility (#1633)

liu-cong
authored and
BenjaminBraunDev
committed
Bug fix - error calling gt: incompatible types for comparison: float64 and int (#1630)

liu-cong
authored and
BenjaminBraunDev
committed
enable istio as a provider + configuring destinationRule (#1381)

Gregory-Pereira
authored and
BenjaminBraunDev
committed
Update to v1.0.1-rc1 helm chart which fixed many bugs (#1641)

liu-cong
authored and
BenjaminBraunDev
committed
fix: Correct indentation for MkDocs Admonitions display issue (#1643)

yankay
authored and
BenjaminBraunDev
committed
Automatically update chart versions in guides (#1645)

learner0810
authored and
BenjaminBraunDev
committed
[chore] Fix serve-multiple-genai-models.md mistake (#1647)

Frapschen
authored and
BenjaminBraunDev
committed
remove istio destinationrule creation from quickstart (#1648)

nirrozenbaum
authored and
BenjaminBraunDev
committed
Adds WeightedRandomPicker plugin description (#1657)

learner0810
authored and
BenjaminBraunDev
committed
chore(deps): bump github.com/prometheus/prometheus (#1634)

dependabot[bot]
authored and
BenjaminBraunDev
committed
Loosen validation for inference pool crd to support up to 8 ports per inference pool (#1653)

syw14
authored and
BenjaminBraunDev
committed
Adding simple helm CI (#1635)

kfswain
authored and
BenjaminBraunDev
committed
Updating proposal process to reflect how we are currently operating (#1660)

kfswain
authored and
BenjaminBraunDev
committed
chore: update helm to use latest release instead of rc (#1662)

nirrozenbaum
authored and
BenjaminBraunDev
committed
support vLLM cache salting in prefix aware scorer (#1646)

authored and
BenjaminBraunDev
committed
Small documentation update to include how to _use_ InferenceObjective in the API summary (#1659)

kfswain
authored and
BenjaminBraunDev
committed
update make target naming in helm chart pushs commands (#1666)

kfswain
authored and
BenjaminBraunDev
committed
Proposal for Multi-Cluster InferencePools (#1374)

authored and
BenjaminBraunDev
committed
Update inference_gateway.json (#1676)

mnmehta
authored and
BenjaminBraunDev
committed
Adding Kubvernor to the list of implementors (#1313)

authored and
BenjaminBraunDev
committed
Multi-Cluster: Adds v1alpha1 API Types and Docs (#1658)

danehans
authored and
BenjaminBraunDev
committed
chore(deps): bump google.golang.org/protobuf from 1.36.9 to 1.36.10 (#1684)

dependabot[bot]
authored and
BenjaminBraunDev
committed
chore(deps): bump github.com/envoyproxy/go-control-plane/envoy (#1686)

dependabot[bot]
authored and
BenjaminBraunDev
committed
chore(deps): bump google.golang.org/grpc from 1.75.1 to 1.76.0 (#1685)

dependabot[bot]
authored and
BenjaminBraunDev
committed
Use actual platform architecture when building images (#1681)

shmuelk
authored and
BenjaminBraunDev
committed
chore: bump controller-tools version in Makefile (#1694)

nirrozenbaum
authored and
BenjaminBraunDev
committed
Dep: Bumps Gateway API to v1.4.0 (#1691)

danehans
authored and
BenjaminBraunDev
committed
updated link to the updated recordings youtube channel (#1692)

nirrozenbaum
authored and
BenjaminBraunDev
committed
docs: add epp version history (#1360)

zhengkezhou1
authored and
BenjaminBraunDev
committed
[traces] init the trace sdk (#1638)

Frapschen
authored and
BenjaminBraunDev
committed
feat: Remove deleted pod from prefix-cache scorer (#1376)

kfirtoledo
authored and
BenjaminBraunDev
committed
Conformance: Adds Weight-Based Traffic Splitting Test (#1669)

danehans
authored and
BenjaminBraunDev
committed
feat(fc): Initial wiring of the flow control layer (#1701)

LukeAVanDrie
authored and
BenjaminBraunDev
committed
chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc (#1709)

dependabot[bot]
authored and
BenjaminBraunDev
committed
chore(deps): bump sigs.k8s.io/controller-runtime from 0.22.1 to 0.22.3 (#1711)

dependabot[bot]
authored and
BenjaminBraunDev
committed
chore(deps): bump github.com/onsi/ginkgo/v2 from 2.25.3 to 2.26.0 (#1712)

dependabot[bot]
authored and
BenjaminBraunDev
committed
chore(deps): bump github.com/prometheus/common from 0.66.1 to 0.67.1 (#1710)

dependabot[bot]
authored and
BenjaminBraunDev
committed
rename conformance infra namespace (#1667)

davidjumani
authored and
BenjaminBraunDev
committed
Fix LabelSelector validation markers for map field (#1679)

KillianGolds
authored and
BenjaminBraunDev
committed
feat: Flow Control context refactor (#1702)

LukeAVanDrie
authored and
BenjaminBraunDev
committed
feat: Add initial Flow Control metrics (#1714)

LukeAVanDrie
authored and
BenjaminBraunDev
committed
Docs: added migration guide (#1558)

capri-xiyue
authored and
BenjaminBraunDev
committed
Fix prefix-cache-scorer benchmark panic (#1664)

Frapschen
authored and
BenjaminBraunDev
committed
Docs: Versions the quickstart guide (#1604)

danehans
authored and
BenjaminBraunDev
committed
Auto-configure prefix-cache-scorer parameters from engine metrics (#1629)

learner0810
authored and
BenjaminBraunDev
committed
Adds PATCH Variable to Release Script (#1723)

danehans
authored and
BenjaminBraunDev
committed
chore: Improve Flow Control docs and logging (#1726)

LukeAVanDrie
authored and
BenjaminBraunDev
committed
Add manifest outputs that split v1 and experimental (#1644)

learner0810
authored and
BenjaminBraunDev
committed
fix-multiple-scheduling-profiles (#1698)

learner0810
authored and
BenjaminBraunDev
committed
Add missing yq dependency to ensure make artifacts (#1728)

yankay
authored and
BenjaminBraunDev
committed
Fix(test): resolve data race in StreamedRequest (#1727)

LukeAVanDrie
authored and
BenjaminBraunDev
committed
Docs: Adds k8s supported verions (#1731)

danehans
authored and
BenjaminBraunDev
committed
Break PostResponse `requestcontrol` plugin into 3 separate plugins to add streamed request functionality (#1661)

BenjaminBraunDev
and
nirrozenbaum
committed
Conformance:use gateway api utility functions from v1.4.0 and minor fix typos. (#1704)

zetxqx
authored and
BenjaminBraunDev
committed
Adding a flag to control whether auth is added to the EPP metrics server (#1639)

authored and
BenjaminBraunDev
committed
Support for vLLM Data parallel (#1663)

shmuelk
authored and
BenjaminBraunDev
committed
Docs: Bumps Quickstart to v1.0.2 Release (#1745)

danehans
authored and
BenjaminBraunDev
committed
Docs: Updates Kgateway in Quickstart (#1740)

danehans
authored and
BenjaminBraunDev
committed
align with PR #1580 (#1749)

elevran
authored and
BenjaminBraunDev
committed
Add Install Gateway section in Getting Started guide (#1673)

dharaneeshvrd
authored and
BenjaminBraunDev
committed
chore(deps): bump github.com/prometheus/prometheus (#1753)

dependabot[bot]
authored and
BenjaminBraunDev
committed
support multi modal inputs (#1617)

learner0810
authored and
BenjaminBraunDev
committed
Add links in openai request format (#1757)

learner0810
authored and
BenjaminBraunDev
committed
Fix incorrect `stream_options` value in Observability example (#1758)

aman4433
authored and
BenjaminBraunDev
committed
Docs: Bumps Quickstart to Use Kgateway v2.2.0-main (#1761)

danehans
authored and
BenjaminBraunDev
committed
Docs: Updates Latest/Main Quickstart (#1747)

danehans
authored and
BenjaminBraunDev
committed
Docs: Versioned Quickstart Install All CRDs (#1762)

danehans
authored and
BenjaminBraunDev
committed
fixed meeting link (#1734)

nirrozenbaum
authored and
BenjaminBraunDev
committed
Experimental SLO-Aware Routing and Latency Prediction (#1568)

BenjaminBraunDev
and
kaushikmitr
committed
Fix syntax and compiler errors after rebase with main
BenjaminBraunDev
committed
Refactor SLO aware routing to not use podmetrics or datalayer
BenjaminBraunDev
committed
Move slo scoring and tracking logic to one central routing plugin
BenjaminBraunDev
committed
Remove merge residue from latencypredictor_helper
BenjaminBraunDev
committed
Use correct datastore interface in director_test and remove unused constants from director
BenjaminBraunDev
committed

Commits on Oct 28, 2025

Added changes from removed slo tracking file
BenjaminBraunDev
committed