[Inference Providers] Add Replicate image-to-video support#2144
Open
lucataco wants to merge 1 commit into
Open
[Inference Providers] Add Replicate image-to-video support#2144lucataco wants to merge 1 commit into
lucataco wants to merge 1 commit into
Conversation
lucataco
requested review from
SBrandeis,
hanouticelina and
julien-c
as code owners
Contributor
Author
|
cc @SBrandeis @hanouticelina, could you take a look when you have a chance? This follows the recent Replicate provider support pattern |
SBrandeis
reviewed
May 7, 2026
Comment on lines
+223
to
+249
| override async getResponse(response: ReplicateOutput): Promise<Blob> { | ||
| if ( | ||
| typeof response === "object" && | ||
| !!response && | ||
| "output" in response && | ||
| Array.isArray(response.output) && | ||
| response.output.length > 0 && | ||
| typeof response.output[0] === "string" && | ||
| isUrl(response.output[0]) | ||
| ) { | ||
| const urlResponse = await fetch(response.output[0]); | ||
| return await urlResponse.blob(); | ||
| } | ||
|
|
||
| if ( | ||
| typeof response === "object" && | ||
| !!response && | ||
| "output" in response && | ||
| typeof response.output === "string" && | ||
| isUrl(response.output) | ||
| ) { | ||
| const urlResponse = await fetch(response.output); | ||
| return await urlResponse.blob(); | ||
| } | ||
|
|
||
| throw new InferenceClientProviderOutputError("Received malformed response from Replicate image-to-video API"); | ||
| } |
Contributor
There was a problem hiding this comment.
From my understanding, the new task "blocks" (waits for the HTTP response) until the model has generated the video, is that correct?
In other words: there is no polling for the generated video
If so, it might be an issue because video models can have generation times larger than 60s which has been proven to be an issue for our proxy layer, due to timeouts
cc @hanouticelina for validation
SBrandeis
reviewed
May 7, 2026
Comment on lines
+200
to
+204
| lora_weights: | ||
| params.mapping?.adapter === "lora" && params.mapping.adapterWeightsPath | ||
| ? `https://huggingface.co/${params.mapping.hfModelId}` | ||
| : undefined, | ||
| }, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
image-to-videosupport in the inference provider helper registry.image,images,input_image,input_images).imageToVideointegration case forWan-AI/Wan2.1-I2V-14B-720Pwith a local hardcoded mapping until the public provider mapping includes the model.Tests
pnpm --filter @huggingface/inference checkpnpm --filter @huggingface/inference format:checkpnpm --filter @huggingface/inference testcc @hanouticelina
Note
Medium Risk
Adds a new Replicate
image-to-videoprovider helper that changes request payload construction (base64 data URLs and multiple image key variants), which could affect compatibility with different Replicate models and payload size/performance.Overview
Adds Replicate support for the
image-to-videotask by registering a newReplicateImageToVideoTaskin the provider helper registry.The new helper base64-encodes
Blobimage inputs into data URLs and sends them under several Replicate-expected keys (image,images,input_image,input_images), then normalizes responses that return either a single URL or a list of URLs into a downloaded videoBlob.Extends integration tests with an
imageToVideocase using a temporaryHARDCODED_MODEL_INFERENCE_MAPPINGentry forWan-AI/Wan2.1-I2V-14B-720P.Reviewed by Cursor Bugbot for commit 9228cd0. Bugbot is set up for automated code reviews on this repo. Configure here.