feat: openAI Files API Implementation

[sivanantha321]

Description

This PR implements the OpenAI Files API (https://platform.openai.com/docs/api-reference/files) in the Envoy AI Gateway, adding support for four file operations. The Files API is a prerequisite for the Batch Processing API (proposal #7: https://github.com/envoyproxy/ai-gateway/blob/main/docs/proposals/007-batch-processing/proposal.md), as batch jobs reference files uploaded via this API.

Implemented Endpoints

Endpoint	Method	Path	Description
Upload file	POST	/v1/files	Upload a file via multipart/form-data with purpose, expiration, and a model extra field for routing
Retrieve file	GET	/v1/files/{file_id}	Fetch file metadata by ID
Retrieve file content	GET	/v1/files/{file_id}/content	Download raw file content
Delete file	DELETE	/v1/files/{file_id}	Delete a file by ID

---

File ID Encoding & Multi-Backend Routing

A core challenge of the Files API in a gateway context is routing stickiness: once a file is uploaded to a specific backend (e.g., OpenAI, Azure), all subsequent operations on that file (retrieve, retrieve content, delete) must be routed to the same backend. Otherwise, the backend will return a "file not found" error.

To solve this without requiring the client to pass extra routing headers on every request, the gateway encodes the model/backend information directly into the file ID returned to the client. This is inspired by LiteLLM's approach [1].

Encoding Format

Original ID:  file-abc123
Encoded ID:   file-<base64url(id:file-abc123;model:gpt-4o-mini)>

Concretely:   file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk
              └─┬─┘└──────────────────┬──────────────────────────┘
             prefix   base64url(id:file-abc123;model:gpt-4o-mini)

Key properties:

• Preserves OpenAI-compatible prefixes (file-, batch_) so the ID looks valid to clients and SDKs
• Transparent to clients — clients use the encoded ID as-is; they don't need to know it contains routing info
• Enables automatic routing — the gateway decodes the file ID on subsequent requests to extract the model name and route to the correct backend

Routing Flow — Example API Usage

Step 1: Upload a file — Client includes model as an extra multipart field:

curl -X POST https://gateway.example.com/v1/files \
  -H "Authorization: Bearer $API_KEY" \
  -F "purpose=fine-tune" \
  -F "file=@training_data.jsonl" \
  -F "model=gpt-4o-mini"

The gateway routes to the backend mapped to gpt-4o-mini, and on the response, encodes the file ID:

{
  "id": "file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk",
  "object": "file",
  "purpose": "fine-tune",
  "filename": "training_data.jsonl"
}

Step 2: Retrieve the file — Client uses the encoded ID as-is:

curl https://gateway.example.com/v1/files/file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk \
  -H "Authorization: Bearer $API_KEY"

The gateway:

1. Decodes the file ID → extracts model: gpt-4o-mini and original ID: file-abc123
2. Routes to the correct backend using the model name
3. Sends GET /v1/files/file-abc123 to the upstream
4. Re-encodes the file ID in the response before returning to the client

Step 3: Retrieve file content:

curl https://gateway.example.com/v1/files/file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk/content \
  -H "Authorization: Bearer $API_KEY"

Step 4: Delete the file:

curl -X DELETE https://gateway.example.com/v1/files/file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk \
  -H "Authorization: Bearer $API_KEY"

The same decode → route → re-encode pattern applies for all operations.

Implementation Details

The encoding/decoding is implemented via two functions in internal/translator/util.go:

• EncodeIDWithModel(id, modelName, idType) — used in ResponseBody of CreateFile translator to encode the file ID returned by the upstream
• DecodeFileID(encodedID) — used in the processor/server layer to extract the model name and original ID from incoming requests

Internal headers (OriginalFileIDHeaderKey, DecodedFileIDHeaderKey) carry the original and decoded file IDs through the request processing pipeline so that translators for Retrieve, RetrieveContent, and Delete can:

• Set the correct upstream path using the decoded (original) file ID
• Restore the encoded file ID in the response body for the client

---

Tracing: Non-Standard OpenInference Approach

The OpenInference specification [2] does not define semantic conventions for file operations. OpenInference has well-defined span types for LLM/chat completions [3], embeddings [4], etc., but file upload/retrieve/delete are not part of the specification.

This PR implements tracing for file operations using a non-standard adaptation of the OpenInference conventions:

Recorder	Span Name	Request Type	Response Type	Notes
CreateFileRecorder	CreateFile	openai.FileNewParams	openai.FileObject	Records output.file_id on success
RetrieveFileRecorder	RetrieveFile	struct{} (no body)	openai.FileObject	Records output.file_id on success
RetrieveFileContentRecorder	RetrieveFileContent	struct{} (no body)	struct{} (raw bytes)	No output attributes (binary content)
DeleteFileRecorder	DeleteFile	struct{} (no body)	openai.FileDeleted	Records output.file_id on success

What's non-standard:

• All four recorders set openinference.span.kind = "LLM" and llm.system = "openai" — borrowing from existing OpenInference conventions even though file operations are not LLM inference calls. This is because OpenInference has no dedicated span kind for file/storage operations.
• Span kind is set to trace.SpanKindInternal (consistent with other OpenInference spans in the codebase).
• Request attributes are minimal — only openinference.span.kind and llm.system are set, since file operations don't have model/prompt/token semantics.
• Response attributes use custom keys like output.file_id and output.mime_type which are not part of the OpenInference spec.
• RetrieveFileContent records no output attributes at all, since the response is raw binary file data.
• All recorders embed NoopChunkRecorder since file operations are never streamed.

---

Key Changes

API Schema Types (internal/apischema/openai/openai.go):

• New types: FileNewParams, FileObject, FileDeleted, FilePurpose, FileObjectPurpose, FileObjectStatus, FileNewParamsExpiresAfter
• UnmarshalMultipart / MarshalMultipart methods on FileNewParams for handling multipart/form-data encoding

Endpoint Specs (internal/endpointspec/endpointspec.go):

• New endpoint specs: CreateFileEndpointSpec, RetrieveFileEndpointSpec, RetrieveFileContentEndpointSpec, DeleteFileEndpointSpec
• ParseBody signature extended with requestHeaders map[string]string parameter to support Content-Type parsing for multipart requests
• CreateFileEndpointSpec.ParseBody validates multipart/form-data Content-Type and boundary, then parses the body via UnmarshalMultipart
• Retrieve/Delete specs return empty struct{} since these are body-less operations (GET/DELETE)

Server Routing (internal/extproc/server.go):

• Refactored from exact string matching to regex + HTTP method-based routing to support parameterized paths (e.g., /v1/files/{file_id}) and method differentiation (GET vs DELETE on the same path)

Processor (internal/extproc/processor_impl.go):

• Added ProcessRequestHeaders support for body-less requests (GET/DELETE/HEAD) so routing can be initialized without waiting for a request body
• New initRequest extracted as a shared initialization path for both header-phase and body-phase processing

Translators (internal/translator/openai_file.go):

• New OpenAI-to-OpenAI passthrough translators for all four file endpoints
• EncodeIDWithModel / DecodeFileID in internal/translator/util.go for multi-backend routing via encoded file IDs

Metrics (internal/metrics/noop_metrics.go):

• New NoopMetrics / NoopMetricsFactory for endpoints that don't yet have dedicated metrics

Tests:

• End-to-end data-plane tests covering upload, retrieve, retrieve content, and delete
• Unit tests for EncodeIDWithModel / DecodeFileID round-trip encoding
• Updated existing endpoint spec and server tests to match new ParseBody signature and regex-based routing

Related Issues/PRs (if applicable)

Related: Proposal #7 — Batch Processing API Support [5]

Special notes for reviewers (if applicable)

• The ParseBody interface change (added requestHeaders parameter) touches all existing endpoint spec implementations — each gains an unused _ map[string]string parameter to satisfy the interface.
• The server routing refactor from exact-match map[string]ProcessorFactory to a []Route with regex + http method matching is a significant architectural change that affects all existing route registrations.
• Metrics for file API endpoints currently use NoopMetricsFactory — a TODO is noted for adding dedicated metrics support.
• The OpenInference tracing approach for file operations is non-standard and may need to be revisited if/when the OpenInference spec adds file operation conventions.
• The model is required as an extra multipart field during file upload — this is a gateway-specific requirement not present in the standard OpenAI Files API.

1: https://github.com/envoyproxy/ai-gateway/blob/main/docs/proposals/007-batch-processing/proposal.md#introduce-fileid-and-batchid-encoding
2: https://github.com/Arize-ai/openinference/tree/main/spec
3: https://github.com/Arize-ai/openinference/blob/main/spec/semantic_conventions.md
4: https://github.com/Arize-ai/openinference/blob/main/spec/embedding_spans.md
5: https://github.com/envoyproxy/ai-gateway/blob/main/docs/proposals/007-batch-processing/proposal.md

[codecov-commenter]

Codecov Report

❌ Patch coverage is 88.59813% with 61 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.48%. Comparing base (7610c0e) to head (017fa28).

Files with missing lines	Patch %	Lines
internal/translator/openai_file.go	75.70%	13 Missing and 13 partials ⚠️
internal/apischema/openai/openai.go	73.68%	10 Missing and 10 partials ⚠️
internal/tracing/tracing.go	77.77%	8 Missing ⚠️
internal/extproc/processor_impl.go	95.31%	2 Missing and 1 partial ⚠️
internal/internalapi/user_facing_errors.go	0.00%	1 Missing and 1 partial ⚠️
internal/translator/util.go	91.66%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1982      +/-   ##
==========================================
+ Coverage   84.38%   84.48%   +0.10%     
==========================================
  Files         130      133       +3     
  Lines       17985    18510     +525     
==========================================
+ Hits        15176    15638     +462     
- Misses       1867     1903      +36     
- Partials      942      969      +27     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[sivanantha321]

/retest

[dosubot]

Related Documentation

3 document(s) may need updating based on files changed in this PR:

Envoy's Space

openai /ai-gateway/blob/main/site/docs/getting-started/connect-providers/openai.md

View Suggested Changes

@@ -93,6 +93,45 @@
   }' \
   $GATEWAY_URL/v1/completions
 ```
+
+#### Files API
+
+The gateway supports the OpenAI Files API, which allows you to upload, retrieve, and manage files. This is particularly useful for batch processing and other features that require file storage.
+
+**Upload a file:**
+
+```shell
+curl -X POST $GATEWAY_URL/v1/files \
+  -H "Content-Type: multipart/form-data" \
+  -F "file=@/path/to/file.jsonl" \
+  -F "purpose=batch" \
+  -F "model_name=gpt-4o-mini"
+```
+
+:::info Model Name Required
+When uploading files, you must include the `model_name` parameter for routing purposes. The returned file ID will be encoded with the model name (e.g., `file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk`).
+:::
+
+**Retrieve file metadata:**
+
+```shell
+curl $GATEWAY_URL/v1/files/{file_id}
+```
+
+**Download file content:**
+
+```shell
+curl $GATEWAY_URL/v1/files/{file_id}/content \
+  -o downloaded-file.jsonl
+```
+
+**Delete a file:**
+
+```shell
+curl -X DELETE $GATEWAY_URL/v1/files/{file_id}
+```
+
+The Files API supports multiple file purposes including `assistants`, `batch`, `fine-tune`, `vision`, `user_data`, and `evals`. For more details on all supported Files API operations, see the [Supported Endpoints](../capabilities/llm-integrations/supported-endpoints.md) documentation.
 
 ## Troubleshooting
 

[Accept] [Decline]

supported-endpoints /ai-gateway/blob/main/site/docs/capabilities/llm-integrations/supported-endpoints.md

View Suggested Changes

@@ -190,6 +190,69 @@
     "n": 1
   }' \
   $GATEWAY_URL/v1/images/generations
+```
+
+### Files
+
+**Endpoints:**
+
+- `POST /v1/files` - Upload a file
+- `GET /v1/files/{file_id}` - Retrieve file metadata
+- `GET /v1/files/{file_id}/content` - Download file content
+- `DELETE /v1/files/{file_id}` - Delete a file
+
+**Status:** ✅ Fully Supported
+
+**Description:** The Files API allows you to upload, retrieve, and manage files that can be used with other OpenAI features like the Batch API and Assistants API.
+
+**Features:**
+
+- ✅ Multipart upload with purpose types (`assistants`, `batch`, `fine-tune`, `vision`, `user_data`, `evals`)
+- ✅ Optional expiration policies with anchor timestamps and duration
+- ✅ File metadata retrieval
+- ✅ File content download
+- ✅ File deletion
+- ✅ File ID encoding for multi-backend routing
+- ✅ Provider fallback and load balancing
+
+**Supported Providers:**
+
+- OpenAI
+
+**Routing Requirement:**
+
+The `model_name` parameter must be included as an extra field during file upload for routing purposes. File IDs are encoded with the model name (e.g., `file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk`) to enable proper routing of subsequent operations to the correct backend.
+
+**Example: Upload a File**
+
+```bash
+curl -X POST $GATEWAY_URL/v1/files \
+  -H "Authorization: Bearer $API_KEY" \
+  -F "file=@/path/to/file.jsonl" \
+  -F "purpose=batch" \
+  -F "model_name=gpt-4o-mini"
+```
+
+**Example: Retrieve File Metadata**
+
+```bash
+curl $GATEWAY_URL/v1/files/file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk \
+  -H "Authorization: Bearer $API_KEY"
+```
+
+**Example: Download File Content**
+
+```bash
+curl $GATEWAY_URL/v1/files/file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk/content \
+  -H "Authorization: Bearer $API_KEY" \
+  -o downloaded-file.jsonl
+```
+
+**Example: Delete a File**
+
+```bash
+curl -X DELETE $GATEWAY_URL/v1/files/file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk \
+  -H "Authorization: Bearer $API_KEY"
 ```
 
 ### Responses

[Accept] [Decline]

supported-endpoints /ai-gateway/blob/main/site/versioned_docs/version-0.5/capabilities/llm-integrations/supported-endpoints.md

View Suggested Changes

@@ -190,6 +190,76 @@
   }' \
   $GATEWAY_URL/v1/images/generations
 ```
+
+### Files
+
+**Description:** The Files API allows you to upload, retrieve, and manage files that can be used with other OpenAI features like the Batch API and Assistants API.
+
+**Supported Operations:**
+
+| Operation | Endpoint | Method | Description |
+|-----------|----------|--------|-------------|
+| Upload File | `/v1/files` | POST | Upload a file with multipart/form-data |
+| Retrieve File | `/v1/files/{file_id}` | GET | Get metadata about a specific file |
+| Download File Content | `/v1/files/{file_id}/content` | GET | Download the content of a file |
+| Delete File | `/v1/files/{file_id}` | DELETE | Delete a file |
+
+**Status:** ✅ Fully Supported
+
+**Supported Providers:**
+
+- OpenAI
+
+**Key Features:**
+
+- ✅ Multipart upload with `multipart/form-data` format
+- ✅ Purpose types: `assistants`, `batch`, `fine-tune`, `vision`, `user_data`, `evals`
+- ✅ Expiration policies with anchor timestamps and duration
+- ✅ File ID encoding with model name for multi-backend routing
+- ✅ Model selection required via `model_name` parameter
+
+**File ID Encoding:**
+
+The gateway encodes file IDs with model routing information to enable correct routing across multiple backends. When you upload a file, the returned file ID includes the model name. Use this encoded ID for all subsequent operations (retrieve, download, delete).
+
+**Example: Upload a File**
+
+```bash
+curl -X POST $GATEWAY_URL/v1/files \
+  -H "Authorization: Bearer $OPENAI_API_KEY" \
+  -F "file=@/path/to/file.jsonl" \
+  -F "purpose=batch" \
+  -F "model_name=gpt-4"
+```
+
+**Example: Retrieve File Metadata**
+
+```bash
+curl $GATEWAY_URL/v1/files/file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk \
+  -H "Authorization: Bearer $OPENAI_API_KEY"
+```
+
+**Example: Download File Content**
+
+```bash
+curl $GATEWAY_URL/v1/files/file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk/content \
+  -H "Authorization: Bearer $OPENAI_API_KEY" \
+  -o downloaded-file.jsonl
+```
+
+**Example: Delete a File**
+
+```bash
+curl -X DELETE $GATEWAY_URL/v1/files/file-aWQ6ZmlsZS1hYmMxMjM7bW9kZWw6Z3B0LTRvLW1pbmk \
+  -H "Authorization: Bearer $OPENAI_API_KEY"
+```
+
+**Notes:**
+
+- The `model_name` parameter is required during file upload for routing purposes (gateway-specific requirement)
+- File IDs returned from upload operations are encoded with the model name
+- Always use the full encoded file ID for subsequent operations
+- Files API is a prerequisite for using the Batch Processing API
 
 ### Responses
 

[Accept] [Decline]

Note: You must be authenticated to accept/decline updates.

^{How did I do? Any feedback?}  

[sivanantha321]

/retest

[sivanantha321]

/retest

[sivanantha321]

/retest

[xiaolin593]

Thank you so much for working on this! Overall looks good to me. left some comments. @envoyproxy/ai-gateway-maintainers please have a look.

[xiaolin593]

do we want annoymous_file name? should we reject when filename is not provided by user?

[sivanantha321]

This behaviour is taken from official OpenAI SDK. https://github.com/openai/openai-go/blob/cbf83a69541f646ec84071a0040f3ca524c2238f/internal/apiform/encoder.go#L366

[xiaolin593]

may need a upper bound(30 days) check for expiresAfter.Seconds per

The number of seconds after the anchor time that the file will expire. Must be between 3600 (1 hour) and 2592000 (30 days).

minimum3600
maximum2592000

[sivanantha321]

The limit may change for different providers. So I haven't added the check here.

[xiaolin593]

status is deprecated, maybe we don't add it here at all?

[sivanantha321]

I have added it for compatibility reasons. I can remove it if you want.

[xiaolin593]

any particular reason to set capacity 20?

[sivanantha321]

no particular reason. It is to avoid repeated reallocation as the default capacity is usually low.

[xiaolin593]

do we need to decode id for subsequent request routing to right backend when retrieve file?

[sivanantha321]

Yes

[sivanantha321]

/retest