Skip to content

Commit 90ba38b

Browse files
Radheshg04Radheshg04
committed
docs: add docs for image editing and image variations
1 parent fd42bf5 commit 90ba38b

File tree

2 files changed

+122
-23
lines changed

2 files changed

+122
-23
lines changed

‎docs/providers/supported-providers/overview.mdx‎

Lines changed: 23 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -15,27 +15,27 @@ Bifrost can also act as a provider-compatible gateway (for example, <u>[Anthropi
1515

1616
The following table summarizes which operations are supported by each provider via Bifrost’s unified interface.
1717

18-
| Provider | Models | Text | Text (stream) | Chat | Chat (stream) | Responses | Responses (stream) | Images | Images (stream) | Embeddings | TTS | TTS (stream) | STT | STT (stream) | Files | Batch | Count tokens |
19-
|----------|--------|------|----------------|------|---------------|-----------|--------------------|--------|-----------------|------------|-----|-------------|-----|--------------|-------|-------|--------------|
20-
| Anthropic (`anthropic/<model>`) | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ |
21-
| Azure (`azure/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
22-
| Bedrock (`bedrock/<model>`) | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ |
23-
| Cerebras (`cerebras/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
24-
| Cohere (`cohere/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
25-
| Elevenlabs (`elevenlabs/<model>`) | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |
26-
| Gemini (`gemini/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
27-
| Groq (`groq/<model>`) | ✅ | 🟡 | 🟡 | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
28-
| Hugging Face (`huggingface/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅| ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
29-
| Mistral (`mistral/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
30-
| Nebius (`nebius/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
31-
| Ollama (`ollama/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
32-
| OpenAI (`openai/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
33-
| OpenRouter (`openrouter/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
34-
| Parasail (`parasail/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
35-
| Perplexity (`perplexity/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
36-
| SGL (`sgl/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
37-
| Vertex AI (`vertex/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
38-
| xAI (`xai/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
18+
| Provider | Models | Text | Text (stream) | Chat | Chat (stream) | Responses | Responses (stream) | Images | Images (stream) | Image Edit | Image Edit (stream) | Image Variation | Embeddings | TTS | TTS (stream) | STT | STT (stream) | Files | Batch | Count tokens |
19+
|----------|--------|------|----------------|------|---------------|-----------|--------------------|--------|-----------------|------------|---------------------|-----------------|------------|-----|-------------|-----|--------------|-------|-------|--------------|
20+
| Anthropic (`anthropic/<model>`) | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ |
21+
| Azure (`azure/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ |
22+
| Bedrock (`bedrock/<model>`) | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ |
23+
| Cerebras (`cerebras/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
24+
| Cohere (`cohere/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
25+
| Elevenlabs (`elevenlabs/<model>`) | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ |
26+
| Gemini (`gemini/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
27+
| Groq (`groq/<model>`) | ✅ | 🟡 | 🟡 | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
28+
| Hugging Face (`huggingface/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
29+
| Mistral (`mistral/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
30+
| Nebius (`nebius/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
31+
| Ollama (`ollama/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
32+
| OpenAI (`openai/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
33+
| OpenRouter (`openrouter/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
34+
| Parasail (`parasail/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
35+
| Perplexity (`perplexity/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
36+
| SGL (`sgl/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
37+
| Vertex AI (`vertex/<model>`) | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
38+
| xAI (`xai/<model>`) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
3939

4040

4141
- 🟡 Not supported by the downstream provider, but internally implemented by Bifrost as a fallback.
@@ -55,6 +55,8 @@ Notes:
5555
- "Text" refers to the classic text completion interface (`/v1/completions`).
5656
- "Responses" refers to the OpenAI-style Responses API (`/v1/responses`). Non-OpenAI providers map this to their native chat API under the hood.
5757
- "Images" refers to the Image Generation API (`/v1/images/generations`).
58+
- "Image Edit" refers to the Image Edit API (`/v1/images/edits`).
59+
- "Image Variation" refers to the Image Variation API (`/v1/images/variations`).
5860
- TTS corresponds to `/v1/audio/speech` and STT to `/v1/audio/transcriptions`.
5961
- "Files" refers to the Files API operations (`/v1/files`) for uploading, listing, retrieving, and deleting files.
6062
- "Batch" refers to the Batch API operations (`/v1/batches`) for creating, listing, retrieving, canceling, and getting results of batch jobs.

‎docs/providers/supported-providers/replicate.mdx‎

Lines changed: 99 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@ Replicate is architecturally different from other providers in Bifrost. It uses
2323
| Responses API | ✅ | ✅ | `/v1/predictions` |
2424
| Text Completions | ✅ | ✅ | `/v1/predictions` |
2525
| Image Generation | ✅ | ✅ | `/v1/predictions` |
26+
| Image Edit | ✅ | ✅ | `/v1/predictions` |
27+
| Image Variation | ❌ | ❌ | - |
2628
| Files | ✅ | - | `/v1/files` |
2729
| List Models | ✅ | - | `/v1/deployments` |
2830
| Embeddings | ❌ | ❌ | - |
@@ -440,7 +442,102 @@ Image generation streaming provides progressive image updates as data URIs:
440442

441443
---
442444

443-
# 5. Files API
445+
# 5. Image Edit
446+
447+
Image edit runs as a prediction like image generation. You send one or more input images plus a prompt; the model returns edited image(s). The same **input image field mapping** as Image Generation applies (see [Field Mapping by Model](#field-mapping-by-model-1) below).
448+
449+
**Endpoint**: `/v1/images/edits` (Bifrost) → Replicate `/v1/predictions` or deployment predictions.
450+
451+
### Parameter Mapping
452+
453+
| Bifrost / Request | Replicate input |
454+
|-------------------|-----------------|
455+
| `input.images` | Mapped to `image_prompt`, `input_image`, `image`, or `input_images` by model |
456+
| `input.prompt` | `prompt` |
457+
| `params.n` | `number_of_images` |
458+
| `params.output_format` | `output_format` |
459+
| `params.quality` | `quality` |
460+
| `params.background` | `background` |
461+
| `params.seed` | `seed` |
462+
| `params.negative_prompt` | `negative_prompt` |
463+
| `params.num_inference_steps` | `num_inference_steps` |
464+
| `params.extra_params` | Merged into prediction input |
465+
466+
### Field Mapping by Model
467+
468+
Input images are mapped to the same fields as in [Image Generation](#field-mapping-by-model):
469+
470+
| Field | Models |
471+
|-------|--------|
472+
| `image_prompt` | `black-forest-labs/flux-1.1-pro`, `flux-1.1-pro-ultra`, `flux-pro`, `flux-1.1-pro-ultra-finetuned` |
473+
| `input_image` | `black-forest-labs/flux-kontext-pro`, `flux-kontext-max`, `flux-kontext-dev` |
474+
| `image` | `black-forest-labs/flux-dev`, `flux-fill-pro`, `flux-dev-lora`, `flux-krea-dev` |
475+
| `input_images` | All other models (default) |
476+
477+
<Note>
478+
For single-image fields (`image_prompt`, `input_image`, `image`), only the first image from `input.images` is used.
479+
</Note>
480+
481+
### Example
482+
483+
<Tabs>
484+
<Tab title="Gateway">
485+
486+
```bash
487+
curl -X POST http://localhost:8080/v1/images/edits \
488+
-H "Content-Type: application/json" \
489+
-d '{
490+
"model": "replicate/black-forest-labs/flux-fill-pro",
491+
"input": {
492+
"prompt": "Replace the sky with a starry night",
493+
"images": [
494+
{ "image": "<base64-or-data-uri>" }
495+
]
496+
},
497+
"output_format": "webp",
498+
"num_inference_steps": 28
499+
}'
500+
```
501+
502+
</Tab>
503+
<Tab title="Go SDK">
504+
505+
```go
506+
resp, err := client.ImageEditRequest(ctx, &schemas.BifrostImageEditRequest{
507+
Provider: schemas.Replicate,
508+
Model: "black-forest-labs/flux-fill-pro",
509+
Input: &schemas.ImageEditInput{
510+
Prompt: "Replace the sky with a starry night",
511+
Images: []schemas.ImageInput{
512+
{ Image: imageBytes },
513+
},
514+
},
515+
Params: &schemas.ImageEditParameters{
516+
OutputFormat: schemas.Ptr("webp"),
517+
NumInferenceSteps: schemas.Ptr(28),
518+
},
519+
})
520+
```
521+
522+
</Tab>
523+
</Tabs>
524+
525+
### Response
526+
527+
Same as Image Generation: single URL → `data[0].url`, array of URLs → `data[i].url`, or data URIs. Response shape is `BifrostImageGenerationResponse` with `data[].url` or `data[].b64_json`.
528+
529+
### Streaming
530+
531+
Image edit streaming is supported. Events use the same prediction log stream as image generation:
532+
533+
- **Partial chunks**: `type: "image_edit.partial_image"` with `b64_json` (or data URI) until completion.
534+
- **Completed**: `type: "image_edit.completed"` with final image and usage.
535+
536+
Use `Prefer: wait` for sync behavior or rely on polling (async) like other Replicate predictions.
537+
538+
---
539+
540+
# 6. Files API
444541

445542
Replicate's Files API supports uploading, listing, and managing files for use in predictions.
446543

@@ -531,7 +628,7 @@ curl -X POST http://localhost:8080/v1/files/file_abc123/content \
531628

532629
---
533630

534-
# 6. List Models
631+
# 7. List Models
535632

536633
**Endpoint**: `/v1/models`
537634

0 commit comments

Comments
 (0)