RenderChatRequest in pkg/tokenization/types/types.go uses []interface{} for Tools and map[string]interface{} for ChatTemplateKWArgs. This means we have no compile-time guarantee that what we parse and forward matches the actual OpenAI API shape for tools (ref: https://platform.openai.com/docs/api-reference/chat/create#chat-create-tools) or that it aligns with the duplicated definitions in the inference scheduler (ref: https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/pkg/epp/framework/interface/scheduling/types.go#L163-L176).
Right now we runtime-cast these fields in uds_tokenizer.go when building the gRPC proto, which means a malformed tool silently gets dropped. We should define typed structs for ChatCompletionTool, FunctionDefinition, and ChatTemplateKWArgs here and align with what GIE uses. Ideally GIE also moves to strong typing so there are no surprises at the boundary — worth raising upstream too.
RenderChatRequestinpkg/tokenization/types/types.gouses[]interface{}forToolsandmap[string]interface{}forChatTemplateKWArgs. This means we have no compile-time guarantee that what we parse and forward matches the actual OpenAI API shape for tools (ref: https://platform.openai.com/docs/api-reference/chat/create#chat-create-tools) or that it aligns with the duplicated definitions in the inference scheduler (ref: https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/pkg/epp/framework/interface/scheduling/types.go#L163-L176).Right now we runtime-cast these fields in
uds_tokenizer.gowhen building the gRPC proto, which means a malformed tool silently gets dropped. We should define typed structs forChatCompletionTool,FunctionDefinition, andChatTemplateKWArgshere and align with what GIE uses. Ideally GIE also moves to strong typing so there are no surprises at the boundary — worth raising upstream too.