{CUDA Decoding} Fix SW fallback bugs in xprsDecoderMaker, downgrade CUDA error logs, add XPRS_DISABLE_HW_DECODE

Summary:
First diff in the GPU-accelerated H.265 decoding stack for projectaria-tools (see plan v11 at ~/gdrive/plans/2026-04-03-gpu-accelerated-h265-decoding-pat-v11.md and tech design at https://docs.google.com/document/d/1TMeLy0TqvAdlYo0IY3Qm4BrzE8i035z9MzYvGl_CzeM by Lou Yang). Cleans up two latent bugs in `xprsDecoderMaker` that prevent SW fallback on non-GPU machines, and reduces log spam users see when CUDA is unavailable.

**Known limitation: NVDEC does not support small grayscale H.265 streams.** On Aria Gen2 recordings, the 200x200 eye-tracking and 512x512 SLAM cameras (encoded as grayscale H.265) produce `CUDA_ERROR_NOT_SUPPORTED` from `cuvidCreateDecoder` and fall back to the SW decoder. Only the 2016x1512 RGB camera stream uses the GPU path. The SW fallback machinery in this diff is what makes that graceful — without it, those streams would hard-fail. End-to-end on a 581 MB Aria Gen2 recording (RTX 5080, 580.159.03 driver), GPU vs CPU decode of the RGB stream measured 197.4 FPS vs 16.8 FPS (11.7x speedup, 300 frames). The aria_rerun_viewer load time on the same recording dropped from ~47s (CPU) to ~10s (GPU).

Bug 1 (line 386 of arvr/libraries/vrs/utils/xprs/XprsDecoder.cpp): when an HW decoder's `init()` returns non-OK, the function returned `nullptr` instead of trying the next decoder in the preferred list. This kills the SW fallback path entirely. Fix: log at WARN and `continue` to the next decoder.

Bug 2 (line 363 of the same file): when `xprs::enumDecoders()` returned non-OK, the function bailed out even though the function intentionally collects whatever decoders it managed to enumerate before any throw. On a non-GPU machine the HW decoder loop throws during CUDA init, but SW decoders were already collected. Fix: ignore the result code and only fail if the resulting list is empty.

Log noise: also downgrades all `XR_LOGE` to `XR_LOGW` in `getNvCodecContext()` (cudaContextProvider.cpp) and the catch handler in `enumDecoders`/`enumDecodersByFormat` (xprsDecApi.cpp). On a non-GPU machine these fired every VRS file open even though callers handle the throw and fall back gracefully.

Adds `XPRS_DISABLE_HW_DECODE` env var: setting it to any value forces CPU-only decoding by skipping all HW decoders during enumeration. Useful for deterministic results, working around GPU memory pressure, or comparing HW-vs-SW output. Read once into a static at first call, so the value is fixed for the process lifetime — runtime mutation has no effect.

Same env var skip + same catch-handler downgrade is applied to both `enumDecoders` and `enumDecodersByFormat` (parallel functions with identical structure).

Reviewed By: PiotrBrzyski

Differential Revision: D103253728

Author: YLouWashU

xprs/cudaContextProvider.cpp

@@ -14,10 +14,15 @@
  * limitations under the License.
  */
 
+// DEFAULT_LOG_CHANNEL must be defined before "logging/Log.h": the OSS build's
+// XR_LOGE/XR_LOGW macros are gated on it (the internal logging header defines
+// them unconditionally, which is why this ordering bug is invisible to the Buck
+// build but breaks the OSS CMake build).
+#define DEFAULT_LOG_CHANNEL "XPRS"
+
 #include "cudaContextProvider.h"
 
 #include "logging/Log.h"
-#define DEFAULT_LOG_CHANNEL "XPRS"
 
 namespace xprs {
 
@@ -49,31 +54,35 @@ NvCodecContext NvCodecContextProvider::getNvCodecContext(const int device_num) {
     return nv_codec_context;
   }
 
+  // Note: all failures in this function below log at WARN, not ERROR. The only caller
+  // (xprsDecApi.cpp::enumDecoders) catches the throw and falls back to SW
+  // decoders. On a non-GPU machine this fires every VRS file open, so logging
+  // ERROR-level would spam users who never asked for GPU decoding.
   int ret = cuda_load_functions(&nv_codec_context._cuda_functions, nullptr);
   if (ret < 0) {
     const std::string message = "Loading CUDA functions failed";
-    XR_LOGE("{}", message.c_str());
+    XR_LOGW("{}", message.c_str());
     throw std::runtime_error(message);
   }
   ret = cuvid_load_functions(&nv_codec_context._cuvid_functions, nullptr);
   if (ret < 0) {
     const std::string message = "Loading nvcuvid functions failed";
-    XR_LOGE("{}", message.c_str());
+    XR_LOGW("{}", message.c_str());
     throw std::runtime_error(message);
   }
 
   CUresult cu_result = nv_codec_context._cuda_functions->cuInit(0);
   if (cu_result != CUDA_SUCCESS) {
     const std::string message = "cuInit failed with error code: " + std::to_string(cu_result);
-    XR_LOGE("{}", message.c_str());
+    XR_LOGW("{}", message.c_str());
     throw std::runtime_error(message);
   }
 
   CUdevice cuda_device = 0;
   cu_result = nv_codec_context._cuda_functions->cuDeviceGet(&cuda_device, device_num);
   if (cu_result != CUDA_SUCCESS) {
     const std::string message = "cuDeviceGet failed with error code: " + std::to_string(cu_result);
-    XR_LOGE("{}", message.c_str());
+    XR_LOGW("{}", message.c_str());
     throw std::runtime_error(message);
   }
 
@@ -82,15 +91,15 @@ NvCodecContext NvCodecContextProvider::getNvCodecContext(const int device_num) {
   if (cu_result != CUDA_SUCCESS) {
     const std::string message =
         "cuDeviceGetName failed with error code: " + std::to_string(cu_result);
-    XR_LOGE("{}", message.c_str());
+    XR_LOGW("{}", message.c_str());
     throw std::runtime_error(message);
   }
 
   cu_result = nv_codec_context._cuda_functions->cuCtxCreate(
       &nv_codec_context._cucontext, CU_CTX_SCHED_BLOCKING_SYNC, cuda_device);
   if (cu_result != CUDA_SUCCESS) {
     const std::string message = "cuCtxCreate failed with error code: " + std::to_string(cu_result);
-    XR_LOGE("{}", message.c_str());
+    XR_LOGW("{}", message.c_str());
     throw std::runtime_error(message);
   }
 
@@ -99,7 +108,7 @@ NvCodecContext NvCodecContextProvider::getNvCodecContext(const int device_num) {
   if (cu_result != CUDA_SUCCESS) {
     const std::string message =
         "cuCtxPopCurrent failed with error code: " + std::to_string(cu_result);
-    XR_LOGE("{}", message.c_str());
+    XR_LOGW("{}", message.c_str());
     throw std::runtime_error(message);
   }
 

xprs/nvDecoder.cpp

@@ -14,6 +14,12 @@
  * limitations under the License.
  */
 
+// DEFAULT_LOG_CHANNEL must be defined before <logging/Log.h>: the OSS build's
+// XR_LOGE/XR_LOGW/XR_LOGI macros are gated on it (the internal logging header
+// defines them unconditionally, which is why this ordering bug is invisible to
+// the Buck build but breaks the OSS CMake build).
+#define DEFAULT_LOG_CHANNEL "XPRS"
+
 #include "nvDecoder.h"
 #include "Codecs.h"
 #include "FFmpegUtils.h"
@@ -22,11 +28,10 @@
 
 #include <logging/Log.h>
 #include <cmath>
+#include <memory>
 #include <stdexcept>
 #include <string>
 
-#define DEFAULT_LOG_CHANNEL "XPRS"
-
 namespace xprs {
 
 // Doing a device capability check via NVENC API is more generic and consistent way
@@ -167,6 +172,44 @@ int NvDecoder::HandleVideoSequence(CUVIDEOFORMAT* vdeo_format) {
   _decoded_image_yuv.resize(size_ofdecoded_image_yuv_format_in_bytes);
 
   CUDAContextScope cuda_context_scope(_nvcodec_context);
+
+  // Capability pre-check (NVIDIA-recommended): NVDEC hardware does not support
+  // every codec/chroma/bit-depth/resolution combination. Most notably it cannot
+  // decode monochrome (4:0:0) H.265 — used by Aria's grayscale SLAM and eye
+  // cameras — and returns CUDA_ERROR_NOT_SUPPORTED from cuvidCreateDecoder.
+  // Querying cuvidGetDecoderCaps first lets such streams fall back to SW decode
+  // with a single clear log line, instead of emitting alarming CUDA error logs
+  // for an entirely expected condition. cuvidGetDecoderCaps is loaded optionally
+  // by nv-codec-headers, so guard against a null function pointer.
+  if (_nvcodec_context._cuvid_functions->cuvidGetDecoderCaps != nullptr) {
+    CUVIDDECODECAPS decode_caps = {};
+    decode_caps.eCodecType = video_decode_create_info.CodecType;
+    decode_caps.eChromaFormat = video_decode_create_info.ChromaFormat;
+    decode_caps.nBitDepthMinus8 = video_decode_create_info.bitDepthMinus8;
+    // Only a *successful* caps query that reports the format unsupported (or out
+    // of the supported size range) forces SW fallback. If the query itself fails
+    // (transient driver/context issue), fall through to cuvidCreateDecoder and
+    // let it surface the real error rather than permanently disabling NVDEC.
+    if (_nvcodec_context._cuvid_functions->cuvidGetDecoderCaps(&decode_caps) == CUDA_SUCCESS) {
+      const unsigned long width = video_decode_create_info.ulWidth;
+      const unsigned long height = video_decode_create_info.ulHeight;
+      const bool unsupported = !decode_caps.bIsSupported ||
+          width < static_cast<unsigned long>(decode_caps.nMinWidth) ||
+          height < static_cast<unsigned long>(decode_caps.nMinHeight) ||
+          width > static_cast<unsigned long>(decode_caps.nMaxWidth) ||
+          height > static_cast<unsigned long>(decode_caps.nMaxHeight);
+      if (unsupported) {
+        XR_LOGW(
+            "NVDEC does not support this stream (chroma_format={}, bit_depth={}, {}x{}); falling back to SW decode",
+            static_cast<int>(video_decode_create_info.ChromaFormat),
+            static_cast<int>(video_decode_create_info.bitDepthMinus8) + 8,
+            width,
+            height);
+        throw std::runtime_error("NVDEC unsupported stream format; using SW decode");
+      }
+    }
+  }
+
   CUDA_API_CALL(
       _nvcodec_context._cuvid_functions->cuvidCreateDecoder(&_decoder, &video_decode_create_info),
       _nvcodec_context._cuda_functions,

xprs/xprsDecApi.cpp

@@ -19,6 +19,7 @@
 #include "xprsUtils.h"
 
 #include <algorithm>
+#include <cstdlib>
 #include <string_view>
 
 #ifdef WITH_DAV1D
@@ -85,6 +86,15 @@ bool findDecoderByName(const std::string_view& name, VideoCodec& codec) {
   return false;
 }
 
+// Set XPRS_DISABLE_HW_DECODE to any value to force CPU-only decoding. Useful
+// for deterministic results, working around GPU memory pressure, or comparing
+// HW-vs-SW output. Read once at first call so the value is fixed for the
+// process lifetime — runtime mutation of the env var has no effect.
+bool isHwDecodeDisabled() {
+  static const bool disabled = std::getenv("XPRS_DISABLE_HW_DECODE") != nullptr;
+  return disabled;
+}
+
 } // namespace
 
 ///
@@ -95,10 +105,16 @@ XprsResult enumDecoders(CodecList& codecs, bool hwCapabilityCheck) {
 
   codecs.clear();
   codecs.reserve(std::size(kPreferredDecoderImplementations));
+  const bool hwDisabled = isHwDecodeDisabled();
   try {
     for (const auto& impl : kPreferredDecoderImplementations) {
       VideoCodec codec;
       if (findDecoderByName(impl, codec)) {
+        if (codec.hwAccel && hwDisabled) {
+          XR_LOGI(
+              "Skipping HW decoder {} (XPRS_DISABLE_HW_DECODE is set)", codec.implementationName);
+          continue;
+        }
         if (codec.hwAccel && hwCapabilityCheck) {
 #ifdef WITH_NVCODEC
           const NvCodecContext nvcodecContext = NvCodecContextProvider::getNvCodecContext();
@@ -115,7 +131,10 @@ XprsResult enumDecoders(CodecList& codecs, bool hwCapabilityCheck) {
       }
     }
   } catch (std::exception& e) {
-    XR_LOGE("{}", convertExceptionToError(e, result));
+    // Downgraded from XR_LOGE: on non-GPU machines this fires every time a VRS
+    // file is opened (CUDA init throws). Callers fall back to SW decoders that
+    // were already collected before the throw, so this is expected, not an error.
+    XR_LOGW("HW decoder enumeration skipped: {}", convertExceptionToError(e, result));
   }
 
   // stable_sort so decoders with equal hwAccel keep their
@@ -137,11 +156,17 @@ enumDecodersByFormat(CodecList& codecs, VideoCodecFormat standard, bool hwCapabi
 
   codecs.clear();
   codecs.reserve(std::size(kPreferredDecoderImplementations));
+  const bool hwDisabled = isHwDecodeDisabled();
   try {
     for (const auto& impl : kPreferredDecoderImplementations) {
       VideoCodec codec;
       if (findDecoderByName(impl, codec)) {
         if (codec.format == standard) {
+          if (codec.hwAccel && hwDisabled) {
+            XR_LOGI(
+                "Skipping HW decoder {} (XPRS_DISABLE_HW_DECODE is set)", codec.implementationName);
+            continue;
+          }
           if (codec.hwAccel && hwCapabilityCheck) {
 #ifdef WITH_NVCODEC
             const NvCodecContext nvcodecContext = NvCodecContextProvider::getNvCodecContext();
@@ -159,7 +184,7 @@ enumDecodersByFormat(CodecList& codecs, VideoCodecFormat standard, bool hwCapabi
       }
     }
   } catch (std::exception& e) {
-    XR_LOGE("{}", convertExceptionToError(e, result));
+    XR_LOGW("HW decoder enumeration skipped: {}", convertExceptionToError(e, result));
   }
 
   // stable_sort so decoders with equal hwAccel keep their

xprs/xprsDecoder.cpp

@@ -279,7 +279,11 @@ XprsResult CVideoDecoder::decodeFrame(Frame& frameOut, const Buffer& compressed)
       convertAVFrame(_pix.avFrame(), frameOut);
     }
   } catch (std::exception& e) {
-    XR_LOGE("{}", convertExceptionToError(e, result));
+    // WARNING, not ERROR: a decode exception here is recoverable — the error is
+    // returned via `result` and the caller (xprsDecoderMaker) falls back to the
+    // next decoder. The most common case is NVDEC rejecting an unsupported
+    // stream (e.g. monochrome H.265) during HW->SW probing.
+    XR_LOGW("{}", convertExceptionToError(e, result));
   }
 
   return result;