feat: fall back to system ffmpeg when OpenCV can't decode the video - #460
Open
minh-vt wants to merge 7 commits into
Open
feat: fall back to system ffmpeg when OpenCV can't decode the video#460minh-vt wants to merge 7 commits into
minh-vt wants to merge 7 commits into
Conversation
VSF crops frames to the subtitle region before sending them to OCR. The OCR filter compares detection coordinates (relative to cropped image) against the subtitle area (relative to original full frame), causing all detections to fail intersection checks with 'Out of selection'. Fix: pass filter_sub_area=None when VSF is used, since VSF already filters by cropping. DET and FPS modes continue to use the subtitle area for filtering as before.
Adds FFmpegReader — a cv2.VideoCapture-compatible reader using system ffmpeg. Supports AV1, HEVC, VP9, and any other codec OpenCV's bundled ffmpeg omits. - Detects OpenCV failure via isOpened() - Falls back to FFmpegReader automatically - Sequential pipe reading for the OCR pipeline - Forward-only seeking via frame skipping
ffmpeg's auto-selection tries hardware decode for AV1 on this platform, which fails with 'Failed to get pixel format' and 'Missing Sequence Header'. Force the libdav1d software decoder when codec is AV1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
OpenCV's bundled FFmpeg (
opencv-pythonpip package) doesn't support AV1, HEVC/H.265, or VP9. Opening an AV1-encoded video shows:The app is completely unusable for these videos. The existing "Video Decoder" setting in Settings only affects VideoSubFinder's internal decoder, not OpenCV.
Solution
Adds
backend/tools/ffmpeg_reader.py— a sequential frame reader using system ffmpeg pipe with acv2.VideoCapture-compatible API (isOpened(),get(),set(),read(),release()).Three entry points automatically fall back to ffmpeg when OpenCV fails:
ui/home_interface.py:load_videobackend/main.py:SubtitleExtractor.__init__backend/tools/subtitle_ocr.py:ocr_task_producerEach tries
cv2.VideoCapture(path)first (fast for H.264). IfisOpened()returns False, it falls back toFFmpegReader(path)using system ffmpeg. No settings to toggle — automatic.Files changed
backend/tools/ffmpeg_reader.py(new, ~110 lines)backend/main.py(+6 lines)backend/tools/subtitle_ocr.py(+6 lines)ui/home_interface.py(+5 lines)`