H26x Codec support updates for MediaRecorder

[handellm]

Chromium's MediaRecorder API now supports HEVC encoding, introducing the hvc1.* codec string, and adds new codecs (hev1.* and avc3.*) supporting variable resolution video in MP4.

Support for HEVC platform encoding was added in WebCodecs in Chromium M130. As a follow-up, support has been added to the MediaRecorder API in Chromium. The API now supports both MP4 and Matroska muxer types with different HEVC and H.264 mime type specifications. HEVC encoding is only supported if the user's device and operating system provide the necessary capabilities.

Request for Mozilla Position on MediaRecorder codec support

• Specification title: H26x Codec support updates for MediaRecorder
• Specification or proposal URL (if available): https://docs.google.com/document/d/1YijhqWaeI118xqZV6Fd-UtqWpGDc-vFfFS6Dg4JEEqc/edit?tab=t.0
• Explainer URL (if available): https://docs.google.com/document/d/1YijhqWaeI118xqZV6Fd-UtqWpGDc-vFfFS6Dg4JEEqc/edit?tab=t.0
• Proposal author(s) (@-mention GitHub accounts):
• MDN URL (optional):
• Caniuse.com URL (optional):
• Bugzilla URL (optional):
• Mozillians who can provide input (optional): @padenot @jan-ivar
• WebKit standards-position:

Other information

Work on the Chromium side is being tracked here.

[tomrittervg]

As a drive-by comment - "HEVC encoding is only supported if the user's device and operating system provide the necessary capabilities." is (possibly?) adding fingerprinting surface. I'm wondering if these device capabilities are already exposed from something else (e.g. decoding support).

[jesup]

HW decoding support != HW encoding support (I'd bet lots more chipsets/etc support decoding than encoding), so it definitely adds a bit-ish. Perhaps many more bits if you try encoding and look at characteristics of the encoder (which are much less constrained than decoders).

[handellm]

In Chrome it's already exposed with WebCodecs. Besides, there's the GPU model name accessible from WebGL which infers support.

Regarding the question regarding how much the bit says: we've found that HEVC HW encode support is be very widespread. It should be in every Intel since gen6. Close to 100% of all modernish Apple devices, and Android phones.

[handellm]

... and then additionally there's MediaCapabilities and presence of profiles for H264. Not sure much new is exposed.

[jan-ivar]

The explainer says: "The MediaRecorder.isTypeSupported() static method is available to check for support for the new codecs."

This seems problematic given the WebRTC WG's decision to deprecate that API a year ago.

• WebKit standards-position:

cc @youennf

[StaZhu]

mediaCapabilities.encodingInfo() with type = record has not been well - supported in Chromium and WebKit yet.

In Chromium’s implementation, it is behind an experimental MediaCapabilitiesEncodingInfoEnabled feature flag.

Both video.canPlayType() and MediaSource.isTypeSupported() have exactly the same problem. It is indeed a pain to implement support for these sync APIs, but I feel we can’t just say we can "deprecate" the “old” APIs directly at present.

Based on this, I think developers should still be recommended to use MediaRecorder.isTypeSupport at present. It's still too early to switch to the mediaCapabilities API.

[jan-ivar]

These are exactly the codecs we had in mind, many in hardware, that we don't want to detect synchronously, when we made the decision. If we don't do it now, we might as well not do it at all.

The WebRTC WG has already decided, we just never got around to writing the PR, which I've now taken the liberty of providing w3c/mediacapture-record#226 (preview).

It recommends developers switch to MediaCapabilities sooner than later. Browsers can in theory throttle that API.

Pasting the below into web console in Canary with experimental features enabled appears to work really well! What's standing in the way of enabling it by default?

await navigator.mediaCapabilities.encodingInfo({
  type: "record",
  video: {
    contentType: "video/webm;codecs=av01",
    width: 1920,
    height: 1080,
    bitrate: 120000,
    framerate: 48,
  }
});

{powerEfficient: false, smooth: false, supported: true}

It appears to work in Safari and Firefox as well (albeit with supported: false of course).

[jan-ivar]

@bholley Do we have a position on the larger issue of encoding here (when hardware is available)?