@remotion/web-renderer: Support more containers and audio-only rendering#6576
@remotion/web-renderer: Support more containers and audio-only rendering#6576samohovets wants to merge 11 commits intomainfrom
@remotion/web-renderer: Support more containers and audio-only rendering#6576Conversation
…feature/web-renderer-more-formats
There was a problem hiding this comment.
Pull request overview
This PR extends @remotion/web-renderer and the Studio web render modal to support additional output containers/codecs and introduces an audio-only export mode that skips all visual rendering/screenshotting.
Changes:
- Add container support for
mkv,wav,mp3,oggand audio codec support formp3,vorbis,pcm-s16, including an MP3 WASM encoder fallback registration. - Implement audio-only rendering by disabling video track creation and skipping per-frame visual rendering when using audio-only containers.
- Update Studio’s Web Render modal to include an “Audio” render mode and mode-dependent container/codec UI behavior.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/web-renderer/src/resolve-audio-codec.ts | Registers MP3 encoder fallback when MP3 is selected or used as a fallback codec. |
| packages/web-renderer/src/render-media-on-web.tsx | Introduces videoEnabled and skips video track + screenshot/frame rendering for audio-only containers. |
| packages/web-renderer/src/register-mp3-encoder.ts | Adds lazy MP3 encoder registration with native support check and WASM fallback import. |
| packages/web-renderer/src/mediabunny-mappings.ts | Expands container/audio codec unions, adds audio-only detection + default codec/container helpers, adds new MIME types. |
| packages/web-renderer/src/index.ts | Re-exports new mapping utilities (isAudioOnlyContainer, getDefaultContainerForCodec). |
| packages/web-renderer/src/can-render-types.ts | Updates resolvedVideoCodec to be nullable to reflect audio-only containers. |
| packages/web-renderer/src/can-render-media-on-web.ts | Skips video validation checks for audio-only containers and resolves video codec to null where applicable. |
| packages/web-renderer/package.json | Adds @mediabunny/mp3-encoder dependency. |
| packages/studio/src/components/RenderQueue/client-side-render-types.ts | Allows videoCodec to be null for audio-only client-side render jobs. |
| packages/studio/src/components/RenderQueue/ClientRenderQueueProcessor.tsx | Passes videoCodec as undefined when null to let the renderer pick defaults (including null). |
| packages/studio/src/components/RenderModal/WebRenderModalBasic.tsx | Adds container lists/labels and hides video codec selection in audio render mode. |
| packages/studio/src/components/RenderModal/WebRenderModalAudio.tsx | Adds labels for new audio codecs and hides mute control in audio-only mode. |
| packages/studio/src/components/RenderModal/WebRenderModal.tsx | Adds “Audio” render mode, container/codec auto-switching, and ensures audio-only exports pass videoCodec: null. |
| bun.lock | Locks the new MP3 encoder dependency. |
| const container = options.container ?? 'mp4'; | ||
| const videoCodec = | ||
| options.videoCodec ?? getDefaultVideoCodecForContainer(container); | ||
| options.videoCodec ?? getDefaultVideoCodecForContainer(container) ?? null; | ||
| const videoEnabled = !isAudioOnlyContainer(container); | ||
| const transparent = options.transparent ?? false; |
There was a problem hiding this comment.
canRenderMediaOnWeb() currently flags WebCodecs as unavailable based on VideoEncoder even for audio-only containers. For audio-only rendering, this should not block rendering; gate the VideoEncoder check behind videoEnabled, and consider checking AudioEncoder availability when !muted instead.
| if (videoEnabled) { | ||
| if (!videoCodec) { | ||
| issues.push({ | ||
| type: 'container-codec-mismatch', | ||
| message: `A video codec is required for container ${container}`, |
There was a problem hiding this comment.
There are existing canRenderMediaOnWeb() tests, but the new audio-only path (videoEnabled false for wav/mp3/ogg) isn’t covered. Add tests asserting that audio-only containers yield resolvedVideoCodec === null and that video-only issues (dimension checks / container-codec mismatch) are not reported.
| issues: CanRenderIssue[]; | ||
| resolvedVideoCodec: WebRendererVideoCodec; | ||
| resolvedVideoCodec: WebRendererVideoCodec | null; | ||
| resolvedAudioCodec: WebRendererAudioCodec | null; | ||
| resolvedOutputTarget: WebRendererOutputTarget; | ||
| }; |
There was a problem hiding this comment.
resolvedVideoCodec is now WebRendererVideoCodec | null to support audio-only containers, but CanRenderMediaOnWebOptions.videoCodec remains non-nullable. For API consistency (and to mirror renderMediaOnWeb()), consider allowing videoCodec?: WebRendererVideoCodec | null in the options type as well.
|
|
||
| export const ensureMp3EncoderRegistered = async (): Promise<void> => { | ||
| if (mp3EncoderRegistered) { | ||
| return; | ||
| } | ||
|
|
||
| const nativeSupport = await canEncodeAudio('mp3'); | ||
| if (nativeSupport) { | ||
| mp3EncoderRegistered = true; | ||
| return; | ||
| } | ||
|
|
||
| const {registerMp3Encoder} = await import('@mediabunny/mp3-encoder'); | ||
| registerMp3Encoder(); | ||
| mp3EncoderRegistered = true; |
There was a problem hiding this comment.
ensureMp3EncoderRegistered() can be called concurrently (e.g. multiple renders / codec probes in parallel). With the current boolean guard, two callers can pass the check and both import/register the encoder. Consider caching an in-flight registration Promise (or setting the flag before awaiting) to make registration idempotent under concurrency.
| export const ensureMp3EncoderRegistered = async (): Promise<void> => { | |
| if (mp3EncoderRegistered) { | |
| return; | |
| } | |
| const nativeSupport = await canEncodeAudio('mp3'); | |
| if (nativeSupport) { | |
| mp3EncoderRegistered = true; | |
| return; | |
| } | |
| const {registerMp3Encoder} = await import('@mediabunny/mp3-encoder'); | |
| registerMp3Encoder(); | |
| mp3EncoderRegistered = true; | |
| let mp3EncoderRegistrationPromise: Promise<void> | null = null; | |
| export const ensureMp3EncoderRegistered = async (): Promise<void> => { | |
| if (mp3EncoderRegistered) { | |
| return; | |
| } | |
| if (mp3EncoderRegistrationPromise) { | |
| return mp3EncoderRegistrationPromise; | |
| } | |
| mp3EncoderRegistrationPromise = (async () => { | |
| try { | |
| const nativeSupport = await canEncodeAudio('mp3'); | |
| if (!nativeSupport) { | |
| const {registerMp3Encoder} = await import('@mediabunny/mp3-encoder'); | |
| registerMp3Encoder(); | |
| } | |
| mp3EncoderRegistered = true; | |
| } finally { | |
| mp3EncoderRegistrationPromise = null; | |
| } | |
| })(); | |
| return mp3EncoderRegistrationPromise; |
| const setContainerFormat = useCallback( | ||
| (newContainer: WebRendererContainer) => { | ||
| setContainer(newContainer); | ||
| setCodec(getDefaultVideoCodecForContainer(newContainer) ?? 'h264'); | ||
| setAudioCodec(getDefaultAudioCodecForContainer(newContainer)); |
There was a problem hiding this comment.
setContainerFormat() always calls setCodec(getDefaultVideoCodecForContainer(newContainer) ?? 'h264'). When switching between audio-only containers (wav/mp3/ogg) this overwrites the user’s previously selected video codec even though video isn’t being rendered. Consider only updating the codec when getDefaultVideoCodecForContainer(newContainer) returns a non-null value (or when renderMode is video).
| const isAudioOnly = renderMode === 'audio'; | ||
| const showAudioSettings = isAudioOnly || !muted; | ||
|
|
There was a problem hiding this comment.
In audio-only mode, codec options are disabled based on encodableCodecs. If MP3/Vorbis are meant to be available via a WASM fallback, encodableCodecs likely won’t include them until the encoder is registered, causing the UI to incorrectly disable those codecs. Consider ensuring the encoder fallback is registered before computing encodableCodecs, or treating these codecs as selectable and letting resolveAudioCodec() decide at render time.
…feature/web-renderer-more-formats
…bleFileStream directly Remove custom WritableStream wrapper in web-fs-target.ts that caused "Cannot write to a closing writable stream" errors. mediabunny's StreamTargetChunk is designed to be directly compatible with FileSystemWritableFileStream.write(), so the stream is now passed through directly. output.finalize() handles stream closing automatically.
…temWritableFileStream directly" This reverts commit e915cb8.
Add QuickTime (.mov) as a video+audio container and FLAC (.flac) as an audio-only container. Also adds flac as a new WebRendererAudioCodec. Updates Studio render modal with the new container/codec options.
Use fastStart: "in-memory" for Mp4OutputFormat and MovOutputFormat so the moov atom is placed at the start of the file. Without this, macOS/QuickTime cannot read duration or dimensions, and browsers cannot stream-play the file without downloading it entirely.
a76cc88 to
97d5e9a
Compare
Summary
@mediabunny/mp3-encoderWASM fallback)Changes
@remotion/web-rendererWebRendererContainerto 6 formats: mp4, webm, mkv, wav, mp3, oggWebRendererAudioCodecto 5 codecs: aac, opus, mp3, vorbis, pcm-s16isAudioOnlyContainer(),getDefaultContainerForCodec()utilities@mediabunny/mp3-encoderdependency for MP3 encoding via WASM when native support is unavailable@remotion/studioTODO
Closes #6045