Problem
Users cannot add custom init containers or sidecars to InferenceService pods created by the predictor topology. This limits several advanced deployment patterns:
- Custom model download: A team has a proprietary model distribution system (e.g., their own CDN, Artifactory, P2P download like Dragonfly) and needs a custom init container to fetch the model before KServe's serving runtime starts.
- Model validation: Run a checksum / signature validation init container before serving.
- Certificate injection: A sidecar to inject or rotate mTLS certs.
- Service mesh proxy: While Istio sidecar injection is usually automatic, some teams need to explicitly add a proxy sidecar or customize the sidecar container spec.
- Observability agent: A sidecar that collects custom metrics/logs (e.g., Fluent Bit, Vector, custom Prometheus exporter).
KServe supports init containers and sidecars in the InferenceService predictor spec via containers.
Suggested approach
Expose initContainers and sidecars on ModelServerCustomSpec (predictor) and the equivalent on the LLM topology parameters. Validate that sidecars don't conflict with the main serving container.
Acceptance criteria
Problem
Users cannot add custom init containers or sidecars to InferenceService pods created by the predictor topology. This limits several advanced deployment patterns:
KServe supports init containers and sidecars in the
InferenceServicepredictor spec viacontainers.Suggested approach
Expose
initContainersandsidecarsonModelServerCustomSpec(predictor) and the equivalent on the LLM topology parameters. Validate that sidecars don't conflict with the main serving container.Acceptance criteria
initContainersandcontainers(sidecars) fields.