Make ImageFormat::CUSTOM_CODEC decodable via DecoderFactory (public)

Summary:
Lets VRS natively decode ImageFormat::CUSTOM_CODEC image streams when a decoder is
registered for the stream's codec name, and ships a public example showing how. This
is the generic, codec-agnostic half: no proprietary codec is added here.

Previously the CUSTOM_CODEC read/decode dispatch lived in the internal-only
PixelFrame_fb.cpp behind #if IS_VRS_FB_INTERNAL(), so open-source VRS could not decode
any custom codec. This moves the two generic methods (readCustomCodecFrame /
decodeCustomCodecFrame) into PixelFrame.cpp, always compiled, and removes the gating on
the two switch arms. The methods only dispatch through DecoderFactory keyed on the
codec name -- exactly like the video codecs -- so there is nothing codec-specific in
core. A binary that has registered no decoder for a given codec name gets
makeDecoder() == nullptr and the frame is left undecoded, so stock VRS is unchanged.
areCompatible(CUSTOM_CODEC) stays false (decodability is decided by the registered
decoder, not the static table).

sample_code/SampleCustomCodec.cpp is a minimal, Apache-licensed example: a DecoderI
plus a registerSampleCustomCodecDecoder() that registers it with DecoderFactory. It
documents the extension point for open-source users (the public docs previously said
custom codecs must be decoded "with your own implementation, presumably unknown to
VRS", with no mention of DecoderFactory). The example "codec" is an identity codec --
the wiring is the point, not the algorithm.

Stage-then-commit decode (decode into a local buffer, re-spec the frame only on
success) matches png/jpg/jxl so a failed decode can't leave a reused frame presenting
the previous image as fresh. Block size is bounded against the record before allocating
so a malformed .vrs can't escape the bool API as a bad_alloc.

Both CUSTOM_CODEC entry points (the immediate readCustomCodecFrame and the deferred
decompressImage path) now read the codec spec from an authoritative source and self-guard
it with XR_VERIFY, so a reused frame can't silently mis-dispatch.

Reviewed By: georges-berenger

Differential Revision: D108314140

fbshipit-source-id: 4afda4b7611240901c7e41ed783cc4afbde1f7ae

Author: adambengis

sample_code/CMakeLists.txt

@@ -21,4 +21,5 @@ target_link_libraries(sample_code_app
     vrslib
     vrs_logging
     vrs_os
+    vrs_utils
 )

sample_code/SampleCustomCodec.cpp

@@ -0,0 +1,79 @@
+/*
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <sample_code/SampleCustomCodec.h>
+
+#include <algorithm>
+#include <cstdint>
+
+#include <vrs/ErrorCode.h>
+#include <vrs/RecordFormat.h>
+#include <vrs/utils/DecoderFactory.h>
+
+using namespace vrs;
+using namespace vrs::utils;
+
+// Sample: make ImageFormat::CUSTOM_CODEC streams decodable by VRS tools. VRS bundles no custom
+// codec; you register a DecoderI with DecoderFactory keyed on your codec name, and PixelFrame
+// dispatches to it. This example "codec" is an identity codec whose blob is the raw pixels, to keep
+// the focus on the wiring. Call registerSampleCustomCodecDecoder() once at startup.
+
+namespace vrs_sample_code {
+
+namespace {
+
+constexpr const char* kSampleCodecName = "sample_identity_codec";
+
+class SampleCustomCodecDecoder : public DecoderI {
+ public:
+  int decode(
+      const vector<uint8_t>& encodedFrame,
+      void* outDecodedFrame,
+      const ImageContentBlockSpec& outputImageSpec) override {
+    const size_t rawSize = ImageContentBlockSpec(
+                               outputImageSpec.getPixelFormat(),
+                               outputImageSpec.getWidth(),
+                               outputImageSpec.getHeight())
+                               .getRawImageSize();
+    if (rawSize == ContentBlock::kSizeUnknown || rawSize == 0) {
+      return domainError(DecodeStatus::UnexpectedImageDimensions);
+    }
+    // Identity codec: input is exactly the raw pixels. A real codec decompresses into
+    // outDecodedFrame here; the invariant that carries over is to write at most rawSize bytes.
+    if (encodedFrame.size() != rawSize) {
+      return domainError(DecodeStatus::DecoderError);
+    }
+    std::copy_n(encodedFrame.data(), rawSize, static_cast<uint8_t*>(outDecodedFrame));
+    return 0;
+  }
+};
+
+} // namespace
+
+void registerSampleCustomCodecDecoder() {
+  DecoderFactory::get().registerDecoderMaker(
+      [](const vector<uint8_t>& /*encodedFrame*/,
+         void* /*outDecodedFrame*/,
+         const ImageContentBlockSpec& outputImageSpec,
+         const DecoderOptions& /*options*/) -> std::unique_ptr<DecoderI> {
+        if (outputImageSpec.getCodecName() == kSampleCodecName) {
+          return std::make_unique<SampleCustomCodecDecoder>();
+        }
+        return nullptr;
+      });
+}
+
+} // namespace vrs_sample_code

sample_code/SampleCustomCodec.h

@@ -0,0 +1,24 @@
+/*
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+namespace vrs_sample_code {
+
+/// Register the sample "sample_identity_codec" decoder with DecoderFactory. Call once at startup.
+void registerSampleCustomCodecDecoder();
+
+} // namespace vrs_sample_code

vrs/utils/DecoderFactory.h

@@ -48,17 +48,22 @@ class DecoderI {
   DecoderI& operator=(const DecoderI&) = delete;
   DecoderI(DecoderI&&) = delete;
   DecoderI& operator=(DecoderI&&) = delete;
-  /// Decode compressed image to a frame
+  /// Decode compressed image to a frame. outDecodedFrame is sized to
+  /// outputImageSpec.getRawImageSize(); never write past it. Returns 0 on success, else a
+  /// domainError(DecodeStatus::X) so errorCodeToMessage() can render it.
   virtual int decode(
       const vector<uint8_t>& encodedFrame,
       void* outDecodedFrame,
       const ImageContentBlockSpec& outputImageSpec) = 0;
   /// Flush the decoder's internal state (e.g., decoded picture buffer).
   /// Call this when seeking backward to avoid duplicate POC errors.
-  /// Default implementation does nothing.
+  /// Default implementation does nothing: custom image codecs are stateless and never need it,
+  /// unlike video decoders that carry inter-frame state across seeks.
   virtual void flush() {}
 };
 
+/// Selects a DecoderI for a stream; it must only select, never decode, else the frame decodes
+/// twice.
 using DecoderMaker = std::function<std::unique_ptr<DecoderI>(
     const vector<uint8_t>& encodedFrame,
     void* outDecodedFrame,
@@ -69,6 +74,8 @@ class DecoderFactory {
  public:
   static DecoderFactory& get();
 
+  /// Not thread-safe: register all makers at startup before any file is read. makeDecoder()
+  /// iterates the maker list without a lock.
   void registerDecoderMaker(const DecoderMaker& decoderMaker);
 
   void registerDecoderMaker(

vrs/utils/PixelFrame.cpp

@@ -32,6 +32,7 @@
 #include <vrs/helpers/FileMacros.h>
 #include <vrs/helpers/Throttler.h>
 #include <vrs/utils/BufferRecordReader.hpp>
+#include <vrs/utils/DecoderFactory.h>
 #include <vrs/utils/converters/Grey10PackedConverters.h>
 #include <vrs/utils/converters/Raw10ToGrey10Converter.h>
 #include <utility>
@@ -170,6 +171,8 @@ bool PixelFrame::readFrame(RecordReader* reader, const ContentBlock& cb) {
       return readJpegFrame(reader, cb.getBlockSize());
     case ImageFormat::JXL:
       return readJxlFrame(reader, cb.getBlockSize());
+    case ImageFormat::CUSTOM_CODEC:
+      return readCustomCodecFrame(reader, cb);
     default:
       return false;
   }
@@ -213,12 +216,91 @@ bool PixelFrame::decompressImage(VideoFrameHandler* videoFrameHandler) {
       vector<uint8_t> compressedData(std::move(frameBytes_));
       return readJxlFrame(compressedData);
     }
+    case ImageFormat::CUSTOM_CODEC: {
+      // No ContentBlock here, so imageSpec_ is the only source of the codec name. It is
+      // authoritative only when the frame came from readDiskImageData, which preserves the spec
+      // verbatim.
+      if (!XR_VERIFY(imageSpec_.getImageFormat() == ImageFormat::CUSTOM_CODEC) ||
+          !XR_VERIFY(!imageSpec_.getCodecName().empty())) {
+        return false;
+      }
+      vector<uint8_t> compressedData(std::move(frameBytes_));
+      return decodeCustomCodecFrame(compressedData, imageSpec_);
+    }
     default:
       return false;
   }
   return false;
 }
 
+bool PixelFrame::readCustomCodecFrame(RecordReader* reader, const ContentBlock& cb) {
+  // The codec spec, which carries the codec name, comes from the content block; imageSpec_ is not
+  // authoritative here.
+  const ImageContentBlockSpec& codecSpec = cb.image();
+  if (!XR_VERIFY(codecSpec.getImageFormat() == ImageFormat::CUSTOM_CODEC) ||
+      !XR_VERIFY(!codecSpec.getCodecName().empty())) {
+    return false;
+  }
+  size_t sizeBytes = cb.getBlockSize();
+  if (sizeBytes == 0 || sizeBytes == ContentBlock::kSizeUnknown) {
+    return false;
+  }
+  // Bound against the record before allocating, like readRawFrame, so a bad size can't throw
+  // bad_alloc out of this bool API.
+  if (sizeBytes > reader->getUnreadBytes()) {
+    THROTTLED_LOGE(
+        reader->getRef(),
+        "Custom codec image {} needs {} bytes, only {} available. Recording bug?",
+        codecSpec.asString(),
+        sizeBytes,
+        reader->getUnreadBytes());
+    return false;
+  }
+  vector<uint8_t> compressedData(sizeBytes);
+  if (!VERIFY_SUCCESS(reader->read(compressedData.data(), sizeBytes))) {
+    return false;
+  }
+  return decodeCustomCodecFrame(compressedData, codecSpec);
+}
+
+bool PixelFrame::decodeCustomCodecFrame(
+    const vector<uint8_t>& compressedData,
+    const ImageContentBlockSpec& codecSpec) {
+  if (codecSpec.getImageFormat() != ImageFormat::CUSTOM_CODEC || codecSpec.getCodecName().empty()) {
+    return false;
+  }
+  PixelFormat pixelFormat = codecSpec.getPixelFormat();
+  uint32_t width = codecSpec.getWidth();
+  uint32_t height = codecSpec.getHeight();
+  if (pixelFormat == PixelFormat::UNDEFINED || width == 0 || height == 0) {
+    return false;
+  }
+  // Decode into a local buffer and commit only on success, as png/jpg/jxl do, so a failed decode
+  // can't leave a reused frame presenting the prior image as fresh.
+  ImageContentBlockSpec rawSpec(pixelFormat, width, height);
+  size_t rawSize = rawSpec.getRawImageSize();
+  if (rawSize == ContentBlock::kSizeUnknown || rawSize == 0) {
+    return false;
+  }
+  vector<uint8_t> decoded(rawSize);
+  // Hand the decoder a spec with default strides so getRawImageSize() matches decoded.size();
+  // codecSpec may carry an explicit stride a stride-honoring decoder would overrun.
+  ImageContentBlockSpec decodeSpec(
+      ImageFormat::CUSTOM_CODEC,
+      codecSpec.getCodecName(),
+      ImageContentBlockSpec::kQualityUndefined,
+      pixelFormat,
+      width,
+      height);
+  unique_ptr<DecoderI> decoder =
+      DecoderFactory::get().makeDecoder(compressedData, decoded.data(), decodeSpec);
+  if (!decoder || decoder->decode(compressedData, decoded.data(), decodeSpec) != 0) {
+    return false;
+  }
+  init(rawSpec, std::move(decoded));
+  return true;
+}
+
 bool PixelFrame::readRawFrame(
     RecordReader* reader,
     const ImageContentBlockSpec& inputImageSpec,

vrs/utils/PixelFrame.h

@@ -183,6 +183,10 @@ class PixelFrame {
   /// Decode compressed image data, except for video codec compression.
   bool readCompressedFrame(const vector<uint8_t>& pixels, ImageFormat imageFormat);
 
+  /// Read a CUSTOM_CODEC frame and decode it via a DecoderFactory-registered decoder. Returns
+  /// false if no decoder is registered for the codec name (frame left undecoded).
+  bool readCustomCodecFrame(RecordReader* reader, const ContentBlock& cb);
+
   /// Read a JPEG encoded frame into the internal buffer.
   /// @return True if the frame type is supported & the frame was read.
   bool readJpegFrame(RecordReader* reader, uint32_t sizeBytes);
@@ -336,7 +340,8 @@ class PixelFrame {
   /// Tell if an image format supports a specific pixel format.
   /// Only meaningful for png, jpg, jxl.
   /// Always true for ImageFormat::VIDEO (needs decoders to be sure).
-  /// Always false for ImageFormat::CUSTOM_CODEC (never available).
+  /// Always false for ImageFormat::CUSTOM_CODEC (decodability comes from a registered decoder,
+  /// not this table).
   static bool areCompatible(ImageFormat imageFormat, PixelFormat pixelFormat);
 
   /// Conversion in place from RGBA to RGB (no memory allocation)
@@ -414,7 +419,12 @@ class PixelFrame {
       PixelFormat targetPixelFormat,
       const NormalizeOptions& options) const;
 
- private:
+  /// Decode an in-memory CUSTOM_CODEC payload to RAW pixels via a DecoderFactory-registered
+  /// decoder. codecSpec carries the codec name and decoded format/dimensions.
+  bool decodeCustomCodecFrame(
+      const vector<uint8_t>& compressedData,
+      const ImageContentBlockSpec& codecSpec);
+
   ImageContentBlockSpec imageSpec_;
   vector<uint8_t> frameBytes_;
 };