{CUDA Decoding} Restore SW decoder fallback in xprsDecoderMaker - #262
Open
YLouWashU wants to merge 1 commit into
Open
{CUDA Decoding} Restore SW decoder fallback in xprsDecoderMaker#262YLouWashU wants to merge 1 commit into
YLouWashU wants to merge 1 commit into
Conversation
Contributor
|
@YLouWashU has exported this pull request. If you are a Meta employee, you can view the originating Diff in D114806413. |
YLouWashU
force-pushed
the
export-D114806413
branch
2 times, most recently
from
August 4, 2026 23:56
195900f to
be8d7d9
Compare
Summary:
A wheel built with `ENABLE_NVCODEC=ON` segfaults on any machine without an NVIDIA driver. The GitHub CI "Test wheel" job crashes with exit 139 after:
Cannot load libcuda.so.1
[XPRS][WARNING]: Loading CUDA functions failed
[DecoderFactory][WARNING]: Could not create a decoder for 'H.265'!
With no driver, `getNvCodecContext()` throws while probing the HW entries and `enumDecoders()` catches it, returning a non-OK result. The SW `hevc`/`h264` decoders were already collected before the throw (they precede the HW entries in `kPreferredDecoderImplementations`), so the list is usable. `xprsDecoderMaker` bailed on the non-OK result and discarded that list, so no decoder was created and the null decoder was dereferenced downstream. Note the HW decoders never enter the list at all: the throw happens before `codecs.push_back`.
Three changes:
- Ignore the `enumDecoders()` result and fail only when the decoder list is genuinely empty. This is what fixes the crash.
- On decoder `init()` failure, continue to the next candidate instead of returning, so the HW->SW fallback actually runs. This matters on a machine that does have a GPU, where enumeration succeeds but `init()` fails (NVDEC session limit, VRAM pressure).
- Return `nullptr` rather than `decoder` when the loop is exhausted. Raised by DevMate: with the `continue` above, a candidate that initialized but failed `decode()` could otherwise be returned by a later iteration's fall-through. A non-null return is also load-bearing in `DecoderFactory::makeDecoder`, which stops at the first maker that returns anything, so handing back a known-bad decoder would prevent other registered makers from being tried. This also removes a pre-existing path to the same stale return.
The GPU path is unchanged.
Reviewed By: kongchen1992
Differential Revision: D114806413
YLouWashU
force-pushed
the
export-D114806413
branch
from
August 5, 2026 18:21
be8d7d9 to
4093fa3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
A wheel built with
ENABLE_NVCODEC=ONsegfaults on any machine without an NVIDIA driver. The GitHub CI "Test wheel" job crashes with exit 139 after:With no driver,
getNvCodecContext()throws while probing the HW entries andenumDecoders()catches it, returning a non-OK result. The SWhevc/h264decoders were already collected before the throw (they precede the HW entries inkPreferredDecoderImplementations), so the list is usable.xprsDecoderMakerbailed on the non-OK result and discarded that list, so no decoder was created and the null decoder was dereferenced downstream. Note the HW decoders never enter the list at all: the throw happens beforecodecs.push_back.Three changes:
enumDecoders()result and fail only when the decoder list is genuinely empty. This is what fixes the crash.init()failure, continue to the next candidate instead of returning, so the HW->SW fallback actually runs. This matters on a machine that does have a GPU, where enumeration succeeds butinit()fails (NVDEC session limit, VRAM pressure).nullptrrather thandecoderwhen the loop is exhausted. Raised by DevMate: with thecontinueabove, a candidate that initialized but faileddecode()could otherwise be returned by a later iteration's fall-through. A non-null return is also load-bearing inDecoderFactory::makeDecoder, which stops at the first maker that returns anything, so handing back a known-bad decoder would prevent other registered makers from being tried. This also removes a pre-existing path to the same stale return.The GPU path is unchanged.
Reviewed By: kongchen1992
Differential Revision: D114806413