rsky-relay: clear events before polling & rsky-relay: don't try to reconnect if failed to requestCrawl#106
Merged
rudyfraser merged 5 commits intoMay 15, 2025
Conversation
4af11ce to
96be2e6
Compare
rudyfraser
approved these changes
May 15, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Unlike
miocrate, thepollingcrate doesn't automatically clear the events vec before waiting. This means that we weren't using epoll properly and were always iterating all the connections every time. This also made back-filling unsymmetrical because we were always trying to read from the connections in the same order.Also, stop trying to reconnect crawler if the error occurred during connection phase. This fixes loops for some hosts that seem to be offline (yet the bsky relay listHosts lists them as active). In the future, a more advanced retry logic could be added to handle these cases once again.
Also tweaked some trace levels where appropriate.
Changes
Checklist