Initial Inference Extension Plugin and Deployer

[danehans]

Description

Adds initial support for an inference extension endpoint picker deployer and plugin.

Plugin:

• Contributes an Upstream based on an InferencePool.
• Creates the endpoints collection based on namespace-local Pod IPs from the InferencePool selector.
• Creates an Envoy ORIGINAL_DST from the InferencePool-generated.

Deployer:

• Adds initial support for an InferencePool controller and deployer. This controller and deployer are independent of a gateway deployer. For now, both share the GatewayConfig type since refactoring this type would further increase the size of this PR.

API changes

N/A

Code changes

Plugin:

• Adds inferenceextension/endpointpicker pkg
• Updates krtcollections pkg to transform InferencePool endpoints into Upstream endpoints.
• Bumps go deps for inference extension pkg
• Runs code generators due to go dep bump.

Deployer:

• Controller, deployer, and helm pkgs.

CI changes

N/A

Docs changes

Godocs added throughout. User docs will be added in a future PR.

Context

Supports #10411

Interesting decisions

Plugin:

• To keep the PR small, this is the first of multiple PRs to implement the endpoint picker plugin. This PR _does not create the ext-prc cluster nor does it add the ext-proc filter to the listeners filter chain.

Deployer:

• This PR Implements a separate deployer since an InferencePool (and the supporting infra resources) do not depend on a Gateway. Instead, a deployed Gateway will use config from an InferencePool to learn how to connect to it, what failure mode to use, etc.

Testing steps

Unit tests were added. e2e tests are still required and not included here due to the size of the PR.

Notes for reviewers

• Refer to the upstream docs for add'l context.

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works

[danehans]

Note: To pass CI, this PR includes a few identical commits as #10676 for go mod bump, code generations, etc.

[yuval-k]

where is 40000 coming from?

[danehans]

From https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/test/testdata/envoy.yaml which is based on https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/config/manifests/gateway/extension_policy.yaml.

[danehans]

@yuval-k 4fd238f (WIP) adds support for creating the ext_proc cluster and route filters, PTAL. This commit does not include your review feedback. I'll work on that today.