feat: add Requesty as an OpenAI-compatible provider - #63
Open
Thibaultjaigu wants to merge 2 commits into
Open
Conversation
Contributor
Reviewer's GuideAdds a new OpenAI-compatible Requesty provider modeled on the existing OpenRouter provider, wiring it into the provider registry/config and updating docs. Sequence diagram for using RequestyProvider via ProviderRegistrysequenceDiagram
participant App
participant ProviderRegistry
participant RequestyProvider
participant RequestyAPI
App->>ProviderRegistry: NewProviderRegistry("requesty")
ProviderRegistry-->>App: *ProviderRegistry
App->>ProviderRegistry: (create provider) NewRequestyProvider(apiKey, model, extraHeaders)
ProviderRegistry-->>App: RequestyProvider
App->>RequestyProvider: SetDefaultOptions(config)
App->>RequestyProvider: PrepareRequest(prompt, options)
RequestyProvider-->>App: requestBody
App->>RequestyAPI: POST Endpoint() with Headers() and requestBody
RequestyAPI-->>App: responseBody
App->>RequestyProvider: ParseResponse(responseBody)
RequestyProvider-->>App: completionText
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've found 4 issues, and left some high level feedback:
- There’s a fair amount of duplication between
PrepareRequest,PrepareCompletionRequest,PrepareRequestWithSchema, andPrepareRequestWithMessages(e.g., model/options/fallback_models/provider_preferences handling); consider extracting a shared helper to reduce divergence and make future changes less error-prone. - Several places assume concrete types for options (e.g.,
fallback_modelsas[]string,provider_preferencesasmap[string]interface{},imagesas[]types.ContentPart); if callers pass these via generic configuration maps you may want to normalize/convert from[]interface{}/other shapes to avoid panics or silent no-ops.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- There’s a fair amount of duplication between `PrepareRequest`, `PrepareCompletionRequest`, `PrepareRequestWithSchema`, and `PrepareRequestWithMessages` (e.g., model/options/fallback_models/provider_preferences handling); consider extracting a shared helper to reduce divergence and make future changes less error-prone.
- Several places assume concrete types for options (e.g., `fallback_models` as `[]string`, `provider_preferences` as `map[string]interface{}`, `images` as `[]types.ContentPart`); if callers pass these via generic configuration maps you may want to normalize/convert from `[]interface{}`/other shapes to avoid panics or silent no-ops.
## Individual Comments
### Comment 1
<location path="providers/requesty.go" line_range="71-72" />
<code_context>
+
+// GenerationEndpoint returns the Requesty API endpoint for retrieving generation details.
+// This can be used to query stats like cost and token usage after a request.
+func (p *RequestyProvider) GenerationEndpoint(generationID string) string {
+ return fmt.Sprintf("https://router.requesty.ai/v1/generation?id=%s", generationID)
+}
+
</code_context>
<issue_to_address>
**issue (bug_risk):** URL encode the generation ID when building the GenerationEndpoint URL
`generationID` is inserted into the query string without encoding, so characters like `+`, `?`, `&`, or spaces could produce an invalid or ambiguous URL. Please apply `url.QueryEscape` to `generationID` (and import `net/url`) before formatting the URL to ensure it’s always safe.
</issue_to_address>
### Comment 2
<location path="providers/requesty.go" line_range="483-484" />
<code_context>
+
+// ParseStreamResponse processes a chunk from a streaming Requesty response.
+func (p *RequestyProvider) ParseStreamResponse(chunk []byte) (string, error) {
+ // Skip empty chunks and "[DONE]" markers
+ if len(chunk) == 0 || string(chunk) == "[DONE]" {
+ return "", nil
+ }
</code_context>
<issue_to_address>
**suggestion:** Handle common streaming framing (e.g., prefixed "data:" and whitespace) when skipping DONE markers
This currently only skips chunks equal to `"[DONE]"`. In many SSE implementations, lines look like `"data: [DONE]\n\n"`, with a `data:` prefix and trailing whitespace. If that framing isn’t stripped earlier, `ParseStreamResponse` will try to JSON-decode these control lines and fail. Please trim whitespace and optionally remove a leading `"data:"` before comparing against the `[DONE]` sentinel.
Suggested implementation:
```golang
// ParseStreamResponse processes a chunk from a streaming Requesty response.
func (p *RequestyProvider) ParseStreamResponse(chunk []byte) (string, error) {
// Treat empty chunks as no-op
if len(chunk) == 0 {
return "", nil
}
// Normalize common SSE framing: trim whitespace and optional "data:" prefix
line := strings.TrimSpace(string(chunk))
if line == "" {
return "", nil
}
if strings.HasPrefix(line, "data:") {
line = strings.TrimSpace(line[len("data:"):])
}
// Skip "[DONE]" control messages
if line == "[DONE]" {
return "", nil
}
// NOTE: If higher-level code expects parsed JSON or a specific payload shape,
// this is where that parsing should occur. For now, we return the normalized line.
return line, nil
}
```
1. Ensure `providers/requesty.go` imports the `strings` package. For example, in the import block add:
`import "strings"` (or include it in the existing grouped import).
2. If other parts of the code expect `ParseStreamResponse` to JSON-decode the chunk into a specific structure, move that JSON parsing to happen after the `[DONE]` filtering using `line` rather than the raw `chunk`.
</issue_to_address>
### Comment 3
<location path="providers/requesty.go" line_range="107-113" />
<code_context>
+
+// Headers returns the HTTP headers required for Requesty API requests.
+func (p *RequestyProvider) Headers() map[string]string {
+ headers := map[string]string{
+ "Content-Type": "application/json",
+ "Authorization": "Bearer " + p.apiKey,
+ "HTTP-Referer": "https://github.com/teilomillet/gollm", // Identify the app to Requesty
+ }
+
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Use a standard Referer header name instead of HTTP-Referer unless the API requires it
`HTTP-Referer` is non-standard; the usual header is `Referer`. Unless Requesty’s docs specify `HTTP-Referer`, prefer `Referer` (or send both) for better interoperability with typical HTTP tooling and providers.
```suggestion
// Headers returns the HTTP headers required for Requesty API requests.
func (p *RequestyProvider) Headers() map[string]string {
headers := map[string]string{
"Content-Type": "application/json",
"Authorization": "Bearer " + p.apiKey,
// Use the standard Referer header for interoperability, and also send HTTP-Referer
// in case the Requesty API expects that non-standard header.
"Referer": "https://github.com/teilomillet/gollm", // Identify the app to Requesty
"HTTP-Referer": "https://github.com/teilomillet/gollm",
}
```
</issue_to_address>
### Comment 4
<location path="providers/requesty.go" line_range="125" />
<code_context>
+ return headers
+}
+
+// PrepareRequest creates a chat completion request for the Requesty API.
+func (p *RequestyProvider) PrepareRequest(prompt string, options map[string]interface{}) ([]byte, error) {
+ // Start with the passed options
</code_context>
<issue_to_address>
**issue (complexity):** Consider extracting shared request-building, message-construction, and response-struct logic into private helpers to eliminate duplication and keep each public method focused on its specific behavior.
You can significantly cut duplication and tighten behavior by centralizing the common request/response plumbing into small helpers without changing functionality.
### 1. Factor out common request-building
`PrepareRequest`, `PrepareCompletionRequest`, `PrepareRequestWithSchema`, and `PrepareRequestWithMessages` all manually:
- copy `options`
- set `model`
- merge `p.options`
- handle `fallback_models` → `models`
- handle `provider_preferences` → `provider`
- sometimes handle `stream`, `tools`, `tool_choice`, `enable_prompt_caching`
This can be centralized in a private helper and reused.
```go
// inside RequestyProvider
func (p *RequestyProvider) baseRequest(options map[string]interface{}) map[string]interface{} {
req := make(map[string]interface{}, len(options)+len(p.options)+4)
for k, v := range options {
req[k] = v
}
// model
req["model"] = p.model
// merge provider options (do not override call-site options)
for k, v := range p.options {
if _, exists := req[k]; !exists {
req[k] = v
}
}
// fallback_models -> models
if fallbackModels, ok := req["fallback_models"].([]string); ok {
req["models"] = append([]string{p.model}, fallbackModels...)
delete(req, "fallback_models")
}
// provider_preferences -> provider
if providerPrefs, ok := req["provider_preferences"].(map[string]interface{}); ok {
req["provider"] = providerPrefs
delete(req, "provider_preferences")
}
return req
}
```
Then each public method only focuses on its specifics:
```go
func (p *RequestyProvider) PrepareRequest(prompt string, options map[string]interface{}) ([]byte, error) {
req := p.baseRequest(options)
messages := p.buildMessagesForPrompt(prompt, req)
req["messages"] = messages
// tools / tool_choice
if tools, ok := req["tools"].([]interface{}); ok && len(tools) > 0 {
req["tools"] = tools
}
if toolChoice, ok := req["tool_choice"]; ok {
req["tool_choice"] = toolChoice
}
// stream
if stream, ok := req["stream"].(bool); ok && stream {
req["stream"] = true
}
// prompt caching flag is purely advisory for Requesty
if caching, ok := req["enable_prompt_caching"].(bool); ok && caching {
delete(req, "enable_prompt_caching")
}
return json.Marshal(req)
}
```
```go
func (p *RequestyProvider) PrepareCompletionRequest(prompt string, options map[string]interface{}) ([]byte, error) {
req := p.baseRequest(options)
req["prompt"] = prompt
if stream, ok := req["stream"].(bool); ok && stream {
req["stream"] = true
}
return json.Marshal(req)
}
```
```go
func (p *RequestyProvider) PrepareRequestWithSchema(prompt string, options map[string]interface{}, schema interface{}) ([]byte, error) {
optsCopy := make(map[string]interface{}, len(options)+1)
for k, v := range options {
optsCopy[k] = v
}
optsCopy["response_format"] = map[string]interface{}{
"type": "json_object",
"schema": schema,
}
// reuse normal chat request path
return p.PrepareRequest(prompt, optsCopy)
}
```
```go
func (p *RequestyProvider) PrepareRequestWithMessages(messages []types.MemoryMessage, options map[string]interface{}) ([]byte, error) {
req := p.baseRequest(options)
req["messages"] = p.buildMessagesFromMemory(messages, req)
// tools / tool_choice, stream as above
if tools, ok := req["tools"].([]interface{}); ok && len(tools) > 0 {
req["tools"] = tools
}
if toolChoice, ok := req["tool_choice"]; ok {
req["tool_choice"] = toolChoice
}
if stream, ok := req["stream"].(bool); ok && stream {
req["stream"] = true
}
// prompt caching handled in buildMessagesFromMemory; drop flag
delete(req, "enable_prompt_caching")
return json.Marshal(req)
}
```
`PrepareStreamRequest` can also reuse `baseRequest` or just wrap `PrepareRequest` as you already do.
### 2. Factor message-building (single prompt vs memory)
You can reuse the same multimodal/image handling and system-message behavior between prompt-based and memory-based calls.
```go
func (p *RequestyProvider) buildMessagesForPrompt(prompt string, req map[string]interface{}) []map[string]interface{} {
var messages []map[string]interface{}
if sysMsg, ok := req["system_message"].(string); ok {
messages = append(messages, map[string]interface{}{
"role": "system",
"content": sysMsg,
})
delete(req, "system_message")
}
// multimodal user message
if images, ok := req["images"].([]types.ContentPart); ok && len(images) > 0 {
contentArray := []map[string]interface{}{
{"type": "text", "text": prompt},
}
contentArray = append(contentArray, ConvertImagesToOpenAIContent(images)...)
delete(req, "images")
messages = append(messages, map[string]interface{}{
"role": "user",
"content": contentArray,
})
} else {
messages = append(messages, map[string]interface{}{
"role": "user",
"content": prompt,
})
}
return messages
}
```
```go
func (p *RequestyProvider) buildMessagesFromMemory(messages []types.MemoryMessage, req map[string]interface{}) []map[string]interface{} {
formatted := make([]map[string]interface{}, 0, len(messages))
for _, msg := range messages {
formattedMsg := map[string]interface{}{"role": msg.Role}
if msg.HasMultiContent() {
formattedMsg["content"] = BuildOpenAIContentFromParts(msg.MultiContent)
} else {
formattedMsg["content"] = msg.Content
formattedMsg["content"] = p.applyPromptCachingIfNeeded(msg, req, formattedMsg["content"])
}
formatted = append(formatted, formattedMsg)
}
// last user message + images (reusing existing helper)
if images, ok := req["images"].([]types.ContentPart); ok && len(images) > 0 {
for i := len(formatted) - 1; i >= 0; i-- {
if formatted[i]["role"] == "user" {
contentArray := NormalizeContentArray(formatted[i]["content"])
contentArray = append(contentArray, ConvertImagesToOpenAIContent(images)...)
formatted[i]["content"] = contentArray
break
}
}
delete(req, "images")
}
return formatted
}
```
And isolate the Anthropic prompt-caching mutation:
```go
func (p *RequestyProvider) applyPromptCachingIfNeeded(msg types.MemoryMessage, req map[string]interface{}, content interface{}) interface{} {
caching, ok := req["enable_prompt_caching"].(bool)
if !ok || !caching || msg.Role != "user" {
return content
}
text, ok := msg.Content.(string)
if !ok || len(text) <= 1000 {
return content
}
if !strings.HasPrefix(p.model, "anthropic/") {
return content
}
return []map[string]interface{}{
{
"type": "text",
"text": text,
"cache_control": map[string]string{
"type": "ephemeral",
},
},
}
}
```
This keeps the main message loops straightforward and moves provider-specific heuristics into a single, testable function.
### 3. Share response types between parsing functions
`ParseResponse`, `HandleFunctionCalls`, and `ParseStreamResponse` each define similar anonymous structs. These can be lifted to private types at the top of the file:
```go
type requestyChatResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
FunctionCall *json.RawMessage `json:"function_call"`
ToolCalls []struct {
ID string `json:"id"`
Type string `json:"type"`
Function struct {
Name string `json:"name"`
Arguments json.RawMessage `json:"arguments"`
} `json:"function"`
} `json:"tool_calls"`
} `json:"message"`
FinishReason string `json:"finish_reason"`
NativeFinishReason string `json:"native_finish_reason"`
} `json:"choices"`
Error struct {
Message string `json:"message"`
} `json:"error"`
ID string `json:"id"`
Model string `json:"model"`
}
type requestyTextResponse struct {
Choices []struct {
Text string `json:"text"`
} `json:"choices"`
Error struct {
Message string `json:"message"`
} `json:"error"`
ID string `json:"id"`
Model string `json:"model"`
}
type requestyStreamChunk struct {
Choices []struct {
Delta struct {
Content string `json:"content"`
ToolCalls []struct {
ID string `json:"id"`
Type string `json:"type"`
Function struct {
Name string `json:"name"`
Arguments json.RawMessage `json:"arguments"`
} `json:"function"`
} `json:"tool_calls"`
} `json:"delta"`
FinishReason string `json:"finish_reason"`
} `json:"choices"`
Error struct {
Message string `json:"message"`
} `json:"error"`
ID string `json:"id"`
Model string `json:"model"`
Usage *struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
} `json:"usage"`
}
```
Usage:
```go
func (p *RequestyProvider) ParseResponse(body []byte) (string, error) {
var chatResp requestyChatResponse
chatErr := json.Unmarshal(body, &chatResp)
// ... unchanged logic ...
var textResp requestyTextResponse
// ...
}
```
```go
func (p *RequestyProvider) HandleFunctionCalls(body []byte) ([]byte, error) {
var resp requestyChatResponse
// ...
}
```
```go
func (p *RequestyProvider) ParseStreamResponse(chunk []byte) (string, error) {
var resp requestyStreamChunk
// ...
}
```
This reduces repetition and centralizes any future changes to the response shape.
All of these changes keep the existing behavior but substantially trim the file, reduce duplication, and make future modifications (especially around options handling and response formats) safer and more consistent.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Comment on lines
+483
to
+484
| // Skip empty chunks and "[DONE]" markers | ||
| if len(chunk) == 0 || string(chunk) == "[DONE]" { |
Contributor
There was a problem hiding this comment.
suggestion: Handle common streaming framing (e.g., prefixed "data:" and whitespace) when skipping DONE markers
This currently only skips chunks equal to "[DONE]". In many SSE implementations, lines look like "data: [DONE]\n\n", with a data: prefix and trailing whitespace. If that framing isn’t stripped earlier, ParseStreamResponse will try to JSON-decode these control lines and fail. Please trim whitespace and optionally remove a leading "data:" before comparing against the [DONE] sentinel.
Suggested implementation:
// ParseStreamResponse processes a chunk from a streaming Requesty response.
func (p *RequestyProvider) ParseStreamResponse(chunk []byte) (string, error) {
// Treat empty chunks as no-op
if len(chunk) == 0 {
return "", nil
}
// Normalize common SSE framing: trim whitespace and optional "data:" prefix
line := strings.TrimSpace(string(chunk))
if line == "" {
return "", nil
}
if strings.HasPrefix(line, "data:") {
line = strings.TrimSpace(line[len("data:"):])
}
// Skip "[DONE]" control messages
if line == "[DONE]" {
return "", nil
}
// NOTE: If higher-level code expects parsed JSON or a specific payload shape,
// this is where that parsing should occur. For now, we return the normalized line.
return line, nil
}- Ensure
providers/requesty.goimports thestringspackage. For example, in the import block add:
import "strings"(or include it in the existing grouped import). - If other parts of the code expect
ParseStreamResponseto JSON-decode the chunk into a specific structure, move that JSON parsing to happen after the[DONE]filtering usinglinerather than the rawchunk.
Comment on lines
+107
to
+113
| // Headers returns the HTTP headers required for Requesty API requests. | ||
| func (p *RequestyProvider) Headers() map[string]string { | ||
| headers := map[string]string{ | ||
| "Content-Type": "application/json", | ||
| "Authorization": "Bearer " + p.apiKey, | ||
| "HTTP-Referer": "https://github.com/teilomillet/gollm", // Identify the app to Requesty | ||
| } |
Contributor
There was a problem hiding this comment.
suggestion (bug_risk): Use a standard Referer header name instead of HTTP-Referer unless the API requires it
HTTP-Referer is non-standard; the usual header is Referer. Unless Requesty’s docs specify HTTP-Referer, prefer Referer (or send both) for better interoperability with typical HTTP tooling and providers.
Suggested change
| // Headers returns the HTTP headers required for Requesty API requests. | |
| func (p *RequestyProvider) Headers() map[string]string { | |
| headers := map[string]string{ | |
| "Content-Type": "application/json", | |
| "Authorization": "Bearer " + p.apiKey, | |
| "HTTP-Referer": "https://github.com/teilomillet/gollm", // Identify the app to Requesty | |
| } | |
| // Headers returns the HTTP headers required for Requesty API requests. | |
| func (p *RequestyProvider) Headers() map[string]string { | |
| headers := map[string]string{ | |
| "Content-Type": "application/json", | |
| "Authorization": "Bearer " + p.apiKey, | |
| // Use the standard Referer header for interoperability, and also send HTTP-Referer | |
| // in case the Requesty API expects that non-standard header. | |
| "Referer": "https://github.com/teilomillet/gollm", // Identify the app to Requesty | |
| "HTTP-Referer": "https://github.com/teilomillet/gollm", | |
| } |
| return headers | ||
| } | ||
|
|
||
| // PrepareRequest creates a chat completion request for the Requesty API. |
Contributor
There was a problem hiding this comment.
issue (complexity): Consider extracting shared request-building, message-construction, and response-struct logic into private helpers to eliminate duplication and keep each public method focused on its specific behavior.
You can significantly cut duplication and tighten behavior by centralizing the common request/response plumbing into small helpers without changing functionality.
1. Factor out common request-building
PrepareRequest, PrepareCompletionRequest, PrepareRequestWithSchema, and PrepareRequestWithMessages all manually:
- copy
options - set
model - merge
p.options - handle
fallback_models→models - handle
provider_preferences→provider - sometimes handle
stream,tools,tool_choice,enable_prompt_caching
This can be centralized in a private helper and reused.
// inside RequestyProvider
func (p *RequestyProvider) baseRequest(options map[string]interface{}) map[string]interface{} {
req := make(map[string]interface{}, len(options)+len(p.options)+4)
for k, v := range options {
req[k] = v
}
// model
req["model"] = p.model
// merge provider options (do not override call-site options)
for k, v := range p.options {
if _, exists := req[k]; !exists {
req[k] = v
}
}
// fallback_models -> models
if fallbackModels, ok := req["fallback_models"].([]string); ok {
req["models"] = append([]string{p.model}, fallbackModels...)
delete(req, "fallback_models")
}
// provider_preferences -> provider
if providerPrefs, ok := req["provider_preferences"].(map[string]interface{}); ok {
req["provider"] = providerPrefs
delete(req, "provider_preferences")
}
return req
}Then each public method only focuses on its specifics:
func (p *RequestyProvider) PrepareRequest(prompt string, options map[string]interface{}) ([]byte, error) {
req := p.baseRequest(options)
messages := p.buildMessagesForPrompt(prompt, req)
req["messages"] = messages
// tools / tool_choice
if tools, ok := req["tools"].([]interface{}); ok && len(tools) > 0 {
req["tools"] = tools
}
if toolChoice, ok := req["tool_choice"]; ok {
req["tool_choice"] = toolChoice
}
// stream
if stream, ok := req["stream"].(bool); ok && stream {
req["stream"] = true
}
// prompt caching flag is purely advisory for Requesty
if caching, ok := req["enable_prompt_caching"].(bool); ok && caching {
delete(req, "enable_prompt_caching")
}
return json.Marshal(req)
}func (p *RequestyProvider) PrepareCompletionRequest(prompt string, options map[string]interface{}) ([]byte, error) {
req := p.baseRequest(options)
req["prompt"] = prompt
if stream, ok := req["stream"].(bool); ok && stream {
req["stream"] = true
}
return json.Marshal(req)
}func (p *RequestyProvider) PrepareRequestWithSchema(prompt string, options map[string]interface{}, schema interface{}) ([]byte, error) {
optsCopy := make(map[string]interface{}, len(options)+1)
for k, v := range options {
optsCopy[k] = v
}
optsCopy["response_format"] = map[string]interface{}{
"type": "json_object",
"schema": schema,
}
// reuse normal chat request path
return p.PrepareRequest(prompt, optsCopy)
}func (p *RequestyProvider) PrepareRequestWithMessages(messages []types.MemoryMessage, options map[string]interface{}) ([]byte, error) {
req := p.baseRequest(options)
req["messages"] = p.buildMessagesFromMemory(messages, req)
// tools / tool_choice, stream as above
if tools, ok := req["tools"].([]interface{}); ok && len(tools) > 0 {
req["tools"] = tools
}
if toolChoice, ok := req["tool_choice"]; ok {
req["tool_choice"] = toolChoice
}
if stream, ok := req["stream"].(bool); ok && stream {
req["stream"] = true
}
// prompt caching handled in buildMessagesFromMemory; drop flag
delete(req, "enable_prompt_caching")
return json.Marshal(req)
}PrepareStreamRequest can also reuse baseRequest or just wrap PrepareRequest as you already do.
2. Factor message-building (single prompt vs memory)
You can reuse the same multimodal/image handling and system-message behavior between prompt-based and memory-based calls.
func (p *RequestyProvider) buildMessagesForPrompt(prompt string, req map[string]interface{}) []map[string]interface{} {
var messages []map[string]interface{}
if sysMsg, ok := req["system_message"].(string); ok {
messages = append(messages, map[string]interface{}{
"role": "system",
"content": sysMsg,
})
delete(req, "system_message")
}
// multimodal user message
if images, ok := req["images"].([]types.ContentPart); ok && len(images) > 0 {
contentArray := []map[string]interface{}{
{"type": "text", "text": prompt},
}
contentArray = append(contentArray, ConvertImagesToOpenAIContent(images)...)
delete(req, "images")
messages = append(messages, map[string]interface{}{
"role": "user",
"content": contentArray,
})
} else {
messages = append(messages, map[string]interface{}{
"role": "user",
"content": prompt,
})
}
return messages
}func (p *RequestyProvider) buildMessagesFromMemory(messages []types.MemoryMessage, req map[string]interface{}) []map[string]interface{} {
formatted := make([]map[string]interface{}, 0, len(messages))
for _, msg := range messages {
formattedMsg := map[string]interface{}{"role": msg.Role}
if msg.HasMultiContent() {
formattedMsg["content"] = BuildOpenAIContentFromParts(msg.MultiContent)
} else {
formattedMsg["content"] = msg.Content
formattedMsg["content"] = p.applyPromptCachingIfNeeded(msg, req, formattedMsg["content"])
}
formatted = append(formatted, formattedMsg)
}
// last user message + images (reusing existing helper)
if images, ok := req["images"].([]types.ContentPart); ok && len(images) > 0 {
for i := len(formatted) - 1; i >= 0; i-- {
if formatted[i]["role"] == "user" {
contentArray := NormalizeContentArray(formatted[i]["content"])
contentArray = append(contentArray, ConvertImagesToOpenAIContent(images)...)
formatted[i]["content"] = contentArray
break
}
}
delete(req, "images")
}
return formatted
}And isolate the Anthropic prompt-caching mutation:
func (p *RequestyProvider) applyPromptCachingIfNeeded(msg types.MemoryMessage, req map[string]interface{}, content interface{}) interface{} {
caching, ok := req["enable_prompt_caching"].(bool)
if !ok || !caching || msg.Role != "user" {
return content
}
text, ok := msg.Content.(string)
if !ok || len(text) <= 1000 {
return content
}
if !strings.HasPrefix(p.model, "anthropic/") {
return content
}
return []map[string]interface{}{
{
"type": "text",
"text": text,
"cache_control": map[string]string{
"type": "ephemeral",
},
},
}
}This keeps the main message loops straightforward and moves provider-specific heuristics into a single, testable function.
3. Share response types between parsing functions
ParseResponse, HandleFunctionCalls, and ParseStreamResponse each define similar anonymous structs. These can be lifted to private types at the top of the file:
type requestyChatResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
FunctionCall *json.RawMessage `json:"function_call"`
ToolCalls []struct {
ID string `json:"id"`
Type string `json:"type"`
Function struct {
Name string `json:"name"`
Arguments json.RawMessage `json:"arguments"`
} `json:"function"`
} `json:"tool_calls"`
} `json:"message"`
FinishReason string `json:"finish_reason"`
NativeFinishReason string `json:"native_finish_reason"`
} `json:"choices"`
Error struct {
Message string `json:"message"`
} `json:"error"`
ID string `json:"id"`
Model string `json:"model"`
}
type requestyTextResponse struct {
Choices []struct {
Text string `json:"text"`
} `json:"choices"`
Error struct {
Message string `json:"message"`
} `json:"error"`
ID string `json:"id"`
Model string `json:"model"`
}
type requestyStreamChunk struct {
Choices []struct {
Delta struct {
Content string `json:"content"`
ToolCalls []struct {
ID string `json:"id"`
Type string `json:"type"`
Function struct {
Name string `json:"name"`
Arguments json.RawMessage `json:"arguments"`
} `json:"function"`
} `json:"tool_calls"`
} `json:"delta"`
FinishReason string `json:"finish_reason"`
} `json:"choices"`
Error struct {
Message string `json:"message"`
} `json:"error"`
ID string `json:"id"`
Model string `json:"model"`
Usage *struct {
PromptTokens int `json:"prompt_tokens"`
CompletionTokens int `json:"completion_tokens"`
TotalTokens int `json:"total_tokens"`
} `json:"usage"`
}Usage:
func (p *RequestyProvider) ParseResponse(body []byte) (string, error) {
var chatResp requestyChatResponse
chatErr := json.Unmarshal(body, &chatResp)
// ... unchanged logic ...
var textResp requestyTextResponse
// ...
}func (p *RequestyProvider) HandleFunctionCalls(body []byte) ([]byte, error) {
var resp requestyChatResponse
// ...
}func (p *RequestyProvider) ParseStreamResponse(chunk []byte) (string, error) {
var resp requestyStreamChunk
// ...
}This reduces repetition and centralizes any future changes to the response shape.
All of these changes keep the existing behavior but substantially trim the file, reduce duplication, and make future modifications (especially around options handling and response formats) safer and more consistent.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a dedicated
requestyprovider, mirroring the existing OpenRouter provider as closely as possible.Requesty is an OpenAI-compatible LLM gateway that exposes many models through a single API using
provider/modelnaming (for exampleopenai/gpt-4o,anthropic/claude-sonnet-4-5), the same convention as OpenRouter, so the provider mirrors the OpenRouter implementation.Changes:
providers/requesty.go: new provider implementing theProviderinterface, modeled onopenrouter.go. Uses base URLhttps://router.requesty.ai/v1, Bearer auth, and the OpenAI-compatible chat/completions, streaming, tool-calling and JSON-schema paths. The OpenRouter-onlyopenrouter/autoauto-route sentinel was dropped since it is not applicable;fallback_modelsand provider routing preferences are kept.providers/provider.go: registerrequestyin the constructor map and the standard provider config map (TypeOpenAI endpoint).README.md: add Requesty to the providers list.Verification: I tested this against the live endpoint before opening the PR. Building a
requestyprovider through the registry and calling the API returned HTTP 200, andParseResponsereturned the model output.go build ./...,go vet ./providers/, andgo test ./providers/all pass.Docs: https://requesty.ai , https://docs.requesty.ai , https://app.requesty.ai/api-keys
I work at Requesty. This mirrors the existing OpenRouter provider as closely as possible. Happy to adjust or close it if it is not a fit.
Summary by Sourcery
Add a new Requesty LLM provider and register it as an OpenAI-compatible backend.
New Features:
Documentation: