Skip to content

Commit 535a818

Browse files
nirrozenbaumnirrozenbaum
authored
feat: move payload processing from a separate repo into a dir (#589)
<!--- Provide a general summary of your changes in the Title above --> ## Description initial PR for moving code from https://github.com/opendatahub-io/ai-gateway-payload-processing to this repo (into a separate dir). this will be done in a series of PR to maintain code contributions. ## How Has This Been Tested? <!--- Please describe in detail how you tested your changes. --> <!--- Include details of your testing environment, and the tests you ran to --> <!--- see how your change affects other areas of the code, etc. --> ## Merge criteria: <!--- This PR will be merged by any repository approver when it meets all the points in the checklist --> <!--- Go over all the following points, and put an `x` in all the boxes that apply. --> - [ ] The commits are squashed in a cohesive manner and have meaningful messages. - [ ] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious). - [ ] The developer has manually tested the changes and verified that the changes work <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Introduced payload processing module with extensible plugin framework enabling custom request/response handling and mutations * Added Docker containerization for production deployment * **Chores** * Implemented comprehensive build, test, and deployment automation infrastructure * Updated project ownership configuration * **Documentation** * Added payload processing module documentation <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Nir Rozenbaum <nrozenba@redhat.com>
1 parent 2547fa1 commit 535a818

12 files changed

Lines changed: 820 additions & 0 deletions

File tree

‎OWNERS‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ approvers:
44
- chaitanya1731
55
- nerdalert
66
- jland-redhat
7+
- nirrozenbaum
78
- dmytro-zaharnytskyi
89
- SB159
910
- noyitz
@@ -21,6 +22,7 @@ reviewers:
2122
- chaitanya1731
2223
- nerdalert
2324
- jland-redhat
25+
- nirrozenbaum
2426
- dmytro-zaharnytskyi
2527
- SB159
2628
- noyitz

‎payload-processing/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Binaries for programs and plugins
2+
bin/*

‎payload-processing/Dockerfile‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Dockerfile has specific requirement to put this ARG at the beginning:
2+
# https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
3+
ARG BUILDER_IMAGE=golang:1.25
4+
ARG BASE_IMAGE=gcr.io/distroless/static:nonroot
5+
6+
## Multistage build
7+
FROM ${BUILDER_IMAGE} AS builder
8+
ENV CGO_ENABLED=0
9+
ENV GOOS=linux
10+
ENV GOARCH=amd64
11+
ARG COMMIT_SHA=unknown
12+
ARG BUILD_REF
13+
14+
# Dependencies
15+
WORKDIR /src
16+
COPY go.mod go.sum ./
17+
RUN go mod download
18+
19+
# Sources
20+
COPY cmd/ cmd/
21+
COPY pkg/ pkg/
22+
23+
# -X needs the exact import path of the dependency's version package (matches go.mod / module graph).
24+
RUN VERSION_PKG="$(go list -f '{{.ImportPath}}' sigs.k8s.io/gateway-api-inference-extension/version)" && \
25+
go build -ldflags="-X ${VERSION_PKG}.CommitSHA=${COMMIT_SHA} -X ${VERSION_PKG}.BuildRef=${BUILD_REF}" -o /bbr ./cmd
26+
27+
# Multistage deploy
28+
FROM ${BASE_IMAGE}
29+
30+
WORKDIR /
31+
COPY --from=builder /bbr /bbr
32+
33+
ENTRYPOINT ["/bbr"]

‎payload-processing/Makefile‎

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
# ENVTEST_K8S_VERSION refers to the version of kubebuilder assets to be downloaded by envtest binary.
2+
ENVTEST_K8S_VERSION = 1.31.0
3+
# Set COVERAGE=true or COVERAGE=1 on make test-unit to print per-func coverage (cover.out is removed after).
4+
COVERAGE ?=
5+
6+
# Setting SHELL to bash allows bash commands to be executed by recipes.
7+
# Options are set to exit when a recipe line exits non-zero or a piped command fails.
8+
SHELL = /usr/bin/env bash -o pipefail
9+
.SHELLFLAGS = -ec
10+
11+
GIT_COMMIT_SHA ?= "$(shell git rev-parse HEAD 2>/dev/null)"
12+
GIT_TAG ?= $(shell git describe --tags --dirty --always)
13+
TARGETARCH ?= $(shell go env GOARCH)
14+
PLATFORMS ?= linux/$(TARGETARCH)
15+
DOCKER_BUILDX_CMD ?= docker buildx
16+
IMAGE_BUILD_CMD ?= $(DOCKER_BUILDX_CMD) build
17+
IMAGE_BUILD_EXTRA_OPTS ?=
18+
19+
IMAGE_REGISTRY ?= quay.io/opendatahub-io
20+
IMAGE_NAME := ai-gateway-payload-processing
21+
IMAGE_REPO ?= $(IMAGE_REGISTRY)/$(IMAGE_NAME)
22+
IMAGE_TAG ?= $(IMAGE_REPO):$(GIT_TAG)
23+
24+
BASE_IMAGE ?= gcr.io/distroless/static:nonroot
25+
BUILDER_IMAGE ?= golang:1.25
26+
ifdef GO_VERSION
27+
BUILDER_IMAGE = golang:$(GO_VERSION)
28+
endif
29+
30+
BUILD_REF ?= $(shell git describe --abbrev=0 2>/dev/null)
31+
ifdef EXTRA_TAG
32+
IMAGE_EXTRA_TAG ?= $(IMAGE_REPO):$(EXTRA_TAG)
33+
BUILD_REF = $(EXTRA_TAG)
34+
endif
35+
ifdef IMAGE_EXTRA_TAG
36+
IMAGE_BUILD_EXTRA_OPTS += -t $(IMAGE_EXTRA_TAG)
37+
endif
38+
39+
# The name of the kind cluster to use for the "kind-load" target.
40+
KIND_CLUSTER ?= kind
41+
42+
##@ General
43+
44+
# The help target prints out all targets with their descriptions organized
45+
# beneath their categories. The categories are represented by '##@' and the
46+
# target descriptions by '##'. The awk command is responsible for reading the
47+
# entire set of makefiles included in this invocation, looking for lines of the
48+
# file as xyz: ## something, and then pretty-format the target and help. Then,
49+
# if there's a line with ##@ something, that gets pretty-printed as a category.
50+
# More info on the usage of ANSI control characters for terminal formatting:
51+
# https://en.wikipedia.org/wiki/ANSI_escape_code#SGR_parameters
52+
# More info on the awk command:
53+
# http://linuxcommand.org/lc3_adv_awk.php
54+
55+
.PHONY: help
56+
help: ## Display this help.
57+
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_0-9-]+:.*?##/ { printf " \033[36m%-15s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
58+
59+
##@ Development
60+
61+
.PHONY: fmt
62+
fmt: ## Run go fmt against code.
63+
go fmt ./...
64+
65+
.PHONY: vet
66+
vet: ## Run go vet against code.
67+
go vet ./...
68+
69+
70+
.PHONY: lint
71+
lint: golangci-lint ## Run golangci-lint linter
72+
$(GOLANGCI_LINT) run --timeout 5m
73+
74+
.PHONY: lint-fix
75+
lint-fix: golangci-lint ## Run golangci-lint linter and perform fixes
76+
$(GOLANGCI_LINT) run --fix
77+
78+
.PHONY: tidy
79+
tidy: ## Run go work sync (if go.work exists) and go mod tidy per module.
80+
@if [ -f go.work ]; then go work sync; fi
81+
find . -name go.mod -execdir sh -c 'go mod tidy' \;
82+
83+
.PHONY: verify
84+
verify: tidy vet fmt lint ## Verify the codebase (tidy, vet, fmt, lint).
85+
86+
.PHONY: test-unit
87+
test-unit: envtest ## Run unit tests. Optional: COVERAGE=true (or 1) for go tool cover summary.
88+
@set -e; \
89+
kubebuilder_assets_path="$$($(ENVTEST) use $(ENVTEST_K8S_VERSION) --bin-dir $(LOCALBIN) -p path)"; \
90+
if [ "$(COVERAGE)" = "true" ] || [ "$(COVERAGE)" = "1" ]; then \
91+
CGO_ENABLED=1 KUBEBUILDER_ASSETS="$$kubebuilder_assets_path" go test ./pkg/... -race -count=1 -coverprofile=cover.out; \
92+
go tool cover -func=cover.out; \
93+
rm -f cover.out; \
94+
else \
95+
CGO_ENABLED=1 KUBEBUILDER_ASSETS="$$kubebuilder_assets_path" go test ./pkg/... -race -count=1; \
96+
fi
97+
98+
.PHONY: test
99+
test: test-unit ## Run unit tests (alias for test-unit).
100+
101+
##@ Build
102+
103+
# Build the container image
104+
.PHONY: image-local-build
105+
image-local-build: ## Build the image using Docker Buildx for local development.
106+
set -e; \
107+
builder=$$($(DOCKER_BUILDX_CMD) create --use); \
108+
trap '$(DOCKER_BUILDX_CMD) rm -f "$$builder"' EXIT; \
109+
$(MAKE) image-build PUSH="$(PUSH)" LOAD="$(LOAD)"
110+
111+
.PHONY: image-local-push
112+
image-local-push: PUSH=--push ## Build the image for local development and push it to $IMAGE_REPO.
113+
image-local-push: image-local-build
114+
115+
.PHONY: image-local-load
116+
image-local-load: LOAD=--load ## Build the image for local development and load it in the local Docker registry.
117+
image-local-load: image-local-build
118+
119+
.PHONY: image-build
120+
image-build: ## Build the image using Docker Buildx.
121+
$(IMAGE_BUILD_CMD) -t $(IMAGE_TAG) \
122+
--platform=$(PLATFORMS) \
123+
--build-arg BASE_IMAGE=$(BASE_IMAGE) \
124+
--build-arg BUILDER_IMAGE=$(BUILDER_IMAGE) \
125+
--build-arg COMMIT_SHA=${GIT_COMMIT_SHA} \
126+
--build-arg BUILD_REF=${BUILD_REF} \
127+
$(PUSH) \
128+
$(LOAD) \
129+
$(IMAGE_BUILD_EXTRA_OPTS) ./
130+
131+
.PHONY: image-push
132+
image-push: PUSH=--push ## Build the image and push it to $IMAGE_REPO.
133+
image-push: image-build
134+
135+
.PHONY: image-load
136+
image-load: LOAD=--load ## Build the image and load it in the local Docker registry.
137+
image-load: image-build
138+
139+
.PHONY: image-kind
140+
image-kind: image-build ## Build the image and load it to kind cluster $KIND_CLUSTER ("kind" by default).
141+
kind load docker-image $(IMAGE_TAG) --name $(KIND_CLUSTER)
142+
143+
##@ Dependencies
144+
145+
## Location to install dependencies to
146+
LOCALBIN ?= $(shell pwd)/bin
147+
$(LOCALBIN):
148+
[ -d $@ ] || mkdir -p $@
149+
150+
ENVTEST ?= $(LOCALBIN)/setup-envtest
151+
GOLANGCI_LINT = $(LOCALBIN)/golangci-lint
152+
153+
ENVTEST_VERSION ?= release-0.19
154+
GOLANGCI_LINT_VERSION ?= v2.9.0
155+
156+
.PHONY: envtest
157+
envtest: $(ENVTEST) ## Download setup-envtest locally if necessary.
158+
$(ENVTEST): $(LOCALBIN)
159+
$(call go-install-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest,$(ENVTEST_VERSION))
160+
161+
.PHONY: golangci-lint
162+
golangci-lint: $(GOLANGCI_LINT) ## Download golangci-lint locally if necessary.
163+
$(GOLANGCI_LINT): $(LOCALBIN)
164+
$(call go-install-tool,$(GOLANGCI_LINT),github.com/golangci/golangci-lint/v2/cmd/golangci-lint,$(GOLANGCI_LINT_VERSION))
165+
166+
# go-install-tool will 'go install' any package with custom target and name of binary, if it doesn't exist
167+
# $1 - target path with name of binary
168+
# $2 - package url which can be installed
169+
# $3 - specific version of package
170+
define go-install-tool
171+
@[ -f "$(1)-$(3)" ] || { \
172+
set -e; \
173+
package=$(2)@$(3) ;\
174+
echo "Downloading $${package}" ;\
175+
rm -f $(1) || true ;\
176+
GOBIN=$(LOCALBIN) go install $${package} ;\
177+
mv $(1) $(1)-$(3) ;\
178+
} ;\
179+
ln -sf $(1)-$(3) $(1)
180+
endef

‎payload-processing/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# AI Gateway Payload Processing
2+
3+
This directory contains Payload Processing plugins that will be connected to an AI Gateway via a pluggable BBR (Body Based Routing) framework developed as part of the [Kubernetes Inference Gateway](https://github.com/kubernetes-sigs/gateway-api-inference-extension).
4+
5+
BBR plugins enable custom request/response mutations of both headers and body, allowing advanced capabilities such as promoting the model from a field in the body to a header and routing to a selected endpoint accordingly.

‎payload-processing/cmd/main.go‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
/*
2+
Copyright 2025.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
/**
18+
* This file is adapted from Gateway API Inference Extension
19+
* Original source: https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/cmd/bbr/main.go
20+
* Licensed under the Apache License, Version 2.0
21+
*/
22+
23+
package main
24+
25+
import (
26+
"os"
27+
28+
ctrl "sigs.k8s.io/controller-runtime"
29+
"sigs.k8s.io/gateway-api-inference-extension/cmd/bbr/runner"
30+
)
31+
32+
func main() {
33+
// Register ai-gateway payload processing plugins with pluggable bbr
34+
registerPlugins()
35+
36+
if err := runner.NewRunner().
37+
// WithCustomCollectors(...). // THIS should be used for custom metrics exposed by our plugins
38+
Run(ctrl.SetupSignalHandler()); err != nil {
39+
os.Exit(1)
40+
}
41+
}
42+
43+
func registerPlugins() {
44+
// TODO uncomment after all code moves
45+
// framework.Register(provider_resolver.ModelProviderResolverPluginType, provider_resolver.ModelProviderResolverFactory)
46+
// framework.Register(api_translation.APITranslationPluginType, api_translation.APITranslationFactory)
47+
// framework.Register(apikey_injection.APIKeyInjectionPluginType, apikey_injection.APIKeyInjectionFactory)
48+
}

‎payload-processing/go.mod‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
module github.com/opendatahub-io/ai-gateway-payload-processing
2+
3+
go 1.25.0
4+
5+
require (
6+
github.com/stretchr/testify v1.11.1
7+
k8s.io/api v0.35.2
8+
k8s.io/apimachinery v0.35.2
9+
sigs.k8s.io/controller-runtime v0.23.3
10+
sigs.k8s.io/gateway-api-inference-extension v0.0.0-20260318135032-876ac9d909d0
11+
)
12+
13+
require (
14+
cel.dev/expr v0.25.1 // indirect
15+
github.com/antlr4-go/antlr/v4 v4.13.0 // indirect
16+
github.com/beorn7/perks v1.0.1 // indirect
17+
github.com/blang/semver/v4 v4.0.0 // indirect
18+
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
19+
github.com/cespare/xxhash/v2 v2.3.0 // indirect
20+
github.com/cncf/xds/go v0.0.0-20251210132809-ee656c7534f5 // indirect
21+
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
22+
github.com/emicklei/go-restful/v3 v3.13.0 // indirect
23+
github.com/envoyproxy/go-control-plane/envoy v1.37.0 // indirect
24+
github.com/envoyproxy/protoc-gen-validate v1.3.0 // indirect
25+
github.com/evanphx/json-patch/v5 v5.9.11 // indirect
26+
github.com/felixge/httpsnoop v1.0.4 // indirect
27+
github.com/fsnotify/fsnotify v1.9.0 // indirect
28+
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
29+
github.com/go-logr/logr v1.4.3 // indirect
30+
github.com/go-logr/stdr v1.2.2 // indirect
31+
github.com/go-logr/zapr v1.3.0 // indirect
32+
github.com/go-openapi/jsonpointer v0.22.4 // indirect
33+
github.com/go-openapi/jsonreference v0.21.4 // indirect
34+
github.com/go-openapi/swag v0.25.4 // indirect
35+
github.com/go-openapi/swag/cmdutils v0.25.4 // indirect
36+
github.com/go-openapi/swag/conv v0.25.4 // indirect
37+
github.com/go-openapi/swag/fileutils v0.25.4 // indirect
38+
github.com/go-openapi/swag/jsonname v0.25.4 // indirect
39+
github.com/go-openapi/swag/jsonutils v0.25.4 // indirect
40+
github.com/go-openapi/swag/loading v0.25.4 // indirect
41+
github.com/go-openapi/swag/mangling v0.25.4 // indirect
42+
github.com/go-openapi/swag/netutils v0.25.4 // indirect
43+
github.com/go-openapi/swag/stringutils v0.25.4 // indirect
44+
github.com/go-openapi/swag/typeutils v0.25.4 // indirect
45+
github.com/go-openapi/swag/yamlutils v0.25.4 // indirect
46+
github.com/google/btree v1.1.3 // indirect
47+
github.com/google/cel-go v0.26.0 // indirect
48+
github.com/google/gnostic-models v0.7.0 // indirect
49+
github.com/google/go-cmp v0.7.0 // indirect
50+
github.com/google/uuid v1.6.0 // indirect
51+
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect
52+
github.com/inconshreveable/mousetrap v1.1.0 // indirect
53+
github.com/json-iterator/go v1.1.12 // indirect
54+
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
55+
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
56+
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
57+
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
58+
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
59+
github.com/prometheus/client_golang v1.23.2 // indirect
60+
github.com/prometheus/client_model v0.6.2 // indirect
61+
github.com/prometheus/common v0.67.5 // indirect
62+
github.com/prometheus/procfs v0.17.0 // indirect
63+
github.com/spf13/cobra v1.10.2 // indirect
64+
github.com/spf13/pflag v1.0.10 // indirect
65+
github.com/stoewer/go-strcase v1.3.1 // indirect
66+
github.com/x448/float16 v0.8.4 // indirect
67+
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
68+
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 // indirect
69+
go.opentelemetry.io/otel v1.42.0 // indirect
70+
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.41.0 // indirect
71+
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.41.0 // indirect
72+
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.42.0 // indirect
73+
go.opentelemetry.io/otel/metric v1.42.0 // indirect
74+
go.opentelemetry.io/otel/sdk v1.42.0 // indirect
75+
go.opentelemetry.io/otel/trace v1.42.0 // indirect
76+
go.opentelemetry.io/proto/otlp v1.9.0 // indirect
77+
go.uber.org/multierr v1.11.0 // indirect
78+
go.uber.org/zap v1.27.1 // indirect
79+
go.yaml.in/yaml/v2 v2.4.3 // indirect
80+
go.yaml.in/yaml/v3 v3.0.4 // indirect
81+
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect
82+
golang.org/x/net v0.50.0 // indirect
83+
golang.org/x/oauth2 v0.35.0 // indirect
84+
golang.org/x/sync v0.19.0 // indirect
85+
golang.org/x/sys v0.41.0 // indirect
86+
golang.org/x/term v0.40.0 // indirect
87+
golang.org/x/text v0.34.0 // indirect
88+
golang.org/x/time v0.14.0 // indirect
89+
gomodules.xyz/jsonpatch/v2 v2.5.0 // indirect
90+
google.golang.org/genproto/googleapis/api v0.0.0-20260209200024-4cfbd4190f57 // indirect
91+
google.golang.org/genproto/googleapis/rpc v0.0.0-20260209200024-4cfbd4190f57 // indirect
92+
google.golang.org/grpc v1.79.2 // indirect
93+
google.golang.org/protobuf v1.36.11 // indirect
94+
gopkg.in/evanphx/json-patch.v4 v4.13.0 // indirect
95+
gopkg.in/inf.v0 v0.9.1 // indirect
96+
gopkg.in/yaml.v3 v3.0.1 // indirect
97+
k8s.io/apiextensions-apiserver v0.35.2 // indirect
98+
k8s.io/apiserver v0.35.2 // indirect
99+
k8s.io/client-go v0.35.2 // indirect
100+
k8s.io/component-base v0.35.2 // indirect
101+
k8s.io/klog/v2 v2.130.1 // indirect
102+
k8s.io/kube-openapi v0.0.0-20260127142750-a19766b6e2d4 // indirect
103+
k8s.io/utils v0.0.0-20260108192941-914a6e750570 // indirect
104+
sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.31.2 // indirect
105+
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 // indirect
106+
sigs.k8s.io/randfill v1.0.0 // indirect
107+
sigs.k8s.io/structured-merge-diff/v6 v6.3.2 // indirect
108+
sigs.k8s.io/yaml v1.6.0 // indirect
109+
)

0 commit comments

Comments
 (0)