Open
Description
Changes done on upstream-sync and should move to upstream
-
Pod Role: [UPSTREAM-SYNC] Define and set Pod Role for prefill/decode filtering #127 (replaced by alternative: use upstream Pod labels in filter) -
Mutate headers: [UPSTREAM-SYNC] Added support for mutating headers #129 (needed for P/D) (replaced by changes in upstream to move headers to Req.Headers) - Post response handling: TBD (needed for SessionAffinity, PrefixAware and KVCacheAware)
- Set request id end to end: TBD (needed to correlate across calls for plugins that implement multiple interfaces)
-
Allow schedulers to be called with an existing Scheduling context: [UPSTREAM] Break up scheduling to allow calling with an existing context #134 (no longer needed, mutated headers moved to request) - Enable OpenAI ChatCompletions API: TBD
Do NOT close this issue - it is used to collect pending upstream work
Upstream changes needed (e.g., v1/models
scraping) are NOT collected here
Metadata
Metadata
Assignees
Labels
No labels