fix(ffmpeg): back off exponentially when camera is unreachable - #1461
Open
fluffyspace wants to merge 1 commit into
Open
fix(ffmpeg): back off exponentially when camera is unreachable#1461fluffyspace wants to merge 1 commit into
fluffyspace wants to merge 1 commit into
Conversation
A dead or unreachable camera makes the frame reader spawn a new decode process every few seconds indefinitely (sleep 5s, connect, 5s timeout, repeat), wasting a full CPU core for a camera that will not come back on its own. Consecutive decode failures now back off exponentially (5s, 10s, 20s, 40s, capped at 60s) instead of retrying at a constant 5s. A successful frame resets the backoff to its base delay. Applied to both the ffmpeg and gstreamer camera components.
✅ Deploy Preview for viseron ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Owner
|
Thank you, would you mind taking a look at the pylint error? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A dead or unreachable camera makes the frame reader spawn a new decode process every few seconds indefinitely (sleep 5s → spawn ffmpeg → 5s connect timeout → repeat). On a host with a camera that is powered off or off the network, this burns a full CPU core of pure wasted work, forever.
Consecutive decode failures now back off exponentially instead of retrying at a constant 5s:
A successful frame resets the backoff to the base 5s delay, so a camera that comes back online is reconnected with no noticeable lag.
Applied to both the ffmpeg and gstreamer camera components (identical logic), with new unit tests covering the exponential sequence and the reset-on-success behavior.
Motivation
On our standby Viseron instance the EZVIZ camera has been offline for two weeks; the container was spending ~30–40% of one core just cycling 'Restarting frame pipe' against the dead RTSP endpoint (~15k restarts over 3 days). This fix drops that to a single attempt every 60s.
Tests
tests/components/ffmpeg/test_camera.py(2 tests) — exponential backoff sequence and reset-on-success.