Skip to content

Commit 614329d

Browse files
committed
address review: move config into chart configmap
Signed-off-by: Alok Behera <alokbeherak061@gmail.com>
1 parent bf16d39 commit 614329d

2 files changed

Lines changed: 29 additions & 33 deletions

File tree

config/charts/routerlib/templates/_config.yaml

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,35 @@ data:
8080
parser:
8181
pluginRef: {{ $parserType }}
8282
{{- end }}
83+
passthrough-plugins.yaml: |
84+
# Request-format agnostic routing. Use this when the request format is
85+
# unknown or unsupported, so the EPP cannot rely on scorers that read the
86+
# request body (e.g. the prefix-cache scorer). The passthrough parser does
87+
# not parse the body, and routing falls back to backend-state scorers:
88+
# - active-request-scorer : prefers the least busy endpoint
89+
# - session-affinity-scorer: keeps a session pinned to the same endpoint
90+
# Select it by setting router.epp.pluginsConfigFile: passthrough-plugins.yaml
91+
apiVersion: llm-d.ai/v1alpha1
92+
kind: EndpointPickerConfig
93+
plugins:
94+
- type: passthrough-parser
95+
- type: active-request-scorer
96+
- type: session-affinity-scorer
97+
- type: decode-filter
98+
- type: max-score-picker
99+
- type: single-profile-handler
100+
requestHandler:
101+
parsers:
102+
- pluginRef: passthrough-parser
103+
schedulingProfiles:
104+
- name: default
105+
plugins:
106+
- pluginRef: decode-filter
107+
- pluginRef: max-score-picker
108+
- pluginRef: active-request-scorer
109+
weight: 1
110+
- pluginRef: session-affinity-scorer
111+
weight: 1
83112
{{- if .Values.router.epp.pluginsCustomConfig }}
84113
{{- .Values.router.epp.pluginsCustomConfig | toYaml | nindent 2 }}
85114
{{- end }}

deploy/config/passthrough-epp-config.yaml

Lines changed: 0 additions & 33 deletions
This file was deleted.

0 commit comments

Comments
 (0)