Skip to content

fix(coprocessor): host-listener, add timeout to get logs calls#1723

Merged
mergify[bot] merged 1 commit intomainfrom
rudy/fix/host-listener/break-alloy-hidden-infinite-loop-on-bad-message
Jan 30, 2026
Merged

fix(coprocessor): host-listener, add timeout to get logs calls#1723
mergify[bot] merged 1 commit intomainfrom
rudy/fix/host-listener/break-alloy-hidden-infinite-loop-on-bad-message

Conversation

@rudy-6-4
Copy link
Copy Markdown
Contributor

@rudy-6-4 rudy-6-4 commented Jan 6, 2026

From a case seen on testnet where the host-listener was stuck by alloy looping on bad node message. The timeout will cut that loop and force host-listener to retry and reconnect.

@rudy-6-4 rudy-6-4 requested a review from a team as a code owner January 6, 2026 16:33
@cla-bot cla-bot bot added the cla-signed label Jan 6, 2026
@rudy-6-4 rudy-6-4 requested a review from Copilot January 6, 2026 16:33
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds timeout protection to get_logs RPC calls to prevent the host-listener from hanging indefinitely when encountering buggy node messages. The timeout mechanism ensures that the listener can recover and retry connections rather than becoming stuck.

  • Introduces a 120-second timeout constant for websocket get_logs operations
  • Wraps two provider.get_logs() calls with tokio::time::timeout
  • Implements appropriate error handling and retry logic for timeout scenarios

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mergify
Copy link
Copy Markdown

mergify bot commented Jan 6, 2026

🧪 CI Insights

Here's what we observed from your CI run for 1654b94.

🟢 All jobs passed!

But CI Insights is watching 👀

@rudy-6-4 rudy-6-4 force-pushed the rudy/fix/host-listener/break-alloy-hidden-infinite-loop-on-bad-message branch 2 times, most recently from 2f583dd to 14f3a67 Compare January 8, 2026 14:47
@rudy-6-4 rudy-6-4 requested a review from Copilot January 9, 2026 08:56
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@rudy-6-4 rudy-6-4 force-pushed the rudy/fix/host-listener/break-alloy-hidden-infinite-loop-on-bad-message branch from 14f3a67 to d9baaf5 Compare January 19, 2026 14:15
@rudy-6-4 rudy-6-4 requested a review from a team as a code owner January 19, 2026 14:15
@rudy-6-4 rudy-6-4 force-pushed the rudy/fix/host-listener/break-alloy-hidden-infinite-loop-on-bad-message branch 5 times, most recently from 4a5848c to 2fe4586 Compare January 19, 2026 16:03
@rudy-6-4 rudy-6-4 requested a review from Copilot January 20, 2026 08:02
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

From a case seen on testnet where the host-listener was stuck by alloy looping on bad node message.
The timeout will cut that loop and force host-listener to retry and reconnect.
@rudy-6-4 rudy-6-4 force-pushed the rudy/fix/host-listener/break-alloy-hidden-infinite-loop-on-bad-message branch from 2fe4586 to 1654b94 Compare January 29, 2026 13:04
@rudy-6-4
Copy link
Copy Markdown
Contributor Author

@claude /pr-review

@claude
Copy link
Copy Markdown

claude bot commented Jan 29, 2026

Claude finished @rudy-6-4's task in 4m 56s —— View job


PR Review Complete ✅

  • Pre-review validation
  • Gather context (PR summary & guidelines)
  • Run parallel review agents
  • Validate and filter issues
  • Post review

Summary: This PR correctly implements timeout protection for WebSocket RPC calls in the host-listener to prevent infinite loops from buggy node messages. The implementation uses proper tokio::time::timeout() wrappers, follows project conventions, and integrates well with existing retry mechanisms. No blocking issues found.

The review has been posted as a PR comment. See the review comment for full details.

Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[AUTOMATED]

Code Review Summary

Overall Assessment: ✅ LGTM - The implementation is correct and addresses the stated problem effectively.

What This PR Does

Adds timeout protection to WebSocket RPC calls in the host-listener service to prevent the application from hanging indefinitely when the blockchain node sends problematic messages. This is a fix for a production issue observed on testnet.

Review Highlights

Strengths:

  • Correctly uses tokio::time::timeout() to wrap all critical RPC calls
  • Configurable timeout via --timeout-request-websocket CLI argument (default 15s)
  • Proper error handling patterns using anyhow::bail!
  • Existing retry mechanisms accommodate the new timeout error paths seamlessly
  • Good addition of self.stream = None on failure to trigger reconnection
  • Tests updated with the new required argument

Implementation Quality:

  • No race conditions or concurrency issues detected
  • No resource leaks - tokio::time::timeout handles cancellation correctly
  • Error handling is consistent with existing patterns
  • Code follows Rust conventions and should pass clippy

Minor Observations (Non-blocking)

Nit: The functions get_block_by_id() (line 556) and get_block() (lines 592-594) could benefit from a final summary error log when all retries are exhausted, for better observability. Currently, individual retry attempts are logged, but there's no clear "gave up after N attempts" message. However, callers do log these failures, so this is a minor observability gap.

Existing Review Comments Addressed

  • Copilot suggested updating the help text to be more accurate about which RPC calls are covered - I see this has been addressed
  • The timeout was made configurable (as suggested by @obatirou) with a 15s default
  • Error logging patterns are generally consistent

Verified against guidelines: Rust 1.91.0, anyhow for errors, tokio for async, conventional commits format ✅

@mergify
Copy link
Copy Markdown

mergify bot commented Jan 29, 2026

Merge Queue Status

🚫 The pull request has left the queue (rule: main) at 1654b94

This pull request spent 1 hour 3 minutes 17 seconds in the queue, including 2 minutes 1 second running CI.
The checks were run on draft #1859.

Required conditions to merge
  • check-success = run-e2e-tests / fhevm-e2e-test
  • any of [🛡 GitHub branch protection]:
    • check-neutral = common-pull-request/lint (bpr)
    • check-skipped = common-pull-request/lint (bpr)
    • check-success = common-pull-request/lint (bpr)
  • any of [🛡 GitHub branch protection]:
    • check-neutral = coprocessor-cargo-test/cargo-tests (bpr)
    • check-skipped = coprocessor-cargo-test/cargo-tests (bpr)
    • check-success = coprocessor-cargo-test/cargo-tests (bpr)
  • #approved-reviews-by >= 1 [🛡 GitHub branch protection]
  • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
  • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
  • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]
  • any of [🛡 GitHub branch protection]:
    • check-skipped = coprocessor-cargo-listener-tests/cargo-tests (bpr)
    • check-neutral = coprocessor-cargo-listener-tests/cargo-tests (bpr)
    • check-success = coprocessor-cargo-listener-tests/cargo-tests (bpr)
  • any of [🛡 GitHub branch protection]:
    • check-success = coprocessor-dependency-analysis/dependencies-check (bpr)
    • check-neutral = coprocessor-dependency-analysis/dependencies-check (bpr)
    • check-skipped = coprocessor-dependency-analysis/dependencies-check (bpr)
  • any of [🛡 GitHub branch protection]:
    • check-skipped = gateway-contracts-deployment-tests/sc-deploy (bpr)
    • check-neutral = gateway-contracts-deployment-tests/sc-deploy (bpr)
    • check-success = gateway-contracts-deployment-tests/sc-deploy (bpr)
  • any of [🛡 GitHub branch protection]:
    • check-skipped = kms-connector-tests/test-connector (bpr)
    • check-neutral = kms-connector-tests/test-connector (bpr)
    • check-success = kms-connector-tests/test-connector (bpr)

Reason

The merge conditions cannot be satisfied due to failing checks

Failing checks:

Hint

You may have to fix your CI before adding the pull request to the queue again.
If you update this pull request, to fix the CI, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio requeue comment.

mergify bot added a commit that referenced this pull request Jan 29, 2026
@mergify mergify bot added dequeued and removed queued labels Jan 29, 2026
@mergify
Copy link
Copy Markdown

mergify bot commented Jan 30, 2026

queue

❌ The pull request has been removed from the queue main

Details

The merge conditions cannot be satisfied due to failing checks.

You can take a look at Queue: Embarked in merge queue check runs for more details about the failure.

@mergify
Copy link
Copy Markdown

mergify bot commented Jan 30, 2026

Merge Queue Status

🟠 Waiting for queue conditions

Required conditions to enter a queue
  • -closed [📌 queue requirement]
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of [📌 queue -> configuration change requirements]:
    • -mergify-configuration-changed
    • check-success = Configuration changed
  • any of [🔀 queue conditions]:
    • all of [📌 queue conditions of queue main]:
      • #approved-reviews-by >= 1 [🛡 GitHub branch protection]
      • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
      • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
      • base = main
      • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]
      • label!=do-not-merge
      • any of [🛡 GitHub branch protection]:
        • check-success = common-pull-request/lint (bpr)
        • check-neutral = common-pull-request/lint (bpr)
        • check-skipped = common-pull-request/lint (bpr)
      • any of [🛡 GitHub branch protection]:
        • check-skipped = coprocessor-cargo-listener-tests/cargo-tests (bpr)
        • check-neutral = coprocessor-cargo-listener-tests/cargo-tests (bpr)
        • check-success = coprocessor-cargo-listener-tests/cargo-tests (bpr)
      • any of [🛡 GitHub branch protection]:
        • check-success = coprocessor-cargo-test/cargo-tests (bpr)
        • check-neutral = coprocessor-cargo-test/cargo-tests (bpr)
        • check-skipped = coprocessor-cargo-test/cargo-tests (bpr)
      • any of [🛡 GitHub branch protection]:
        • check-success = coprocessor-dependency-analysis/dependencies-check (bpr)
        • check-neutral = coprocessor-dependency-analysis/dependencies-check (bpr)
        • check-skipped = coprocessor-dependency-analysis/dependencies-check (bpr)
      • any of [🛡 GitHub branch protection]:
        • check-skipped = gateway-contracts-deployment-tests/sc-deploy (bpr)
        • check-neutral = gateway-contracts-deployment-tests/sc-deploy (bpr)
        • check-success = gateway-contracts-deployment-tests/sc-deploy (bpr)
      • any of [🛡 GitHub branch protection]:
        • check-skipped = kms-connector-tests/test-connector (bpr)
        • check-neutral = kms-connector-tests/test-connector (bpr)
        • check-success = kms-connector-tests/test-connector (bpr)

@rudy-6-4
Copy link
Copy Markdown
Contributor Author

@Mergifyio queue

@mergify
Copy link
Copy Markdown

mergify bot commented Jan 30, 2026

queue

❌ The pull request has been removed from the queue main

Details

The merge conditions cannot be satisfied due to failing checks.

You can take a look at Queue: Embarked in merge queue check runs for more details about the failure.

@rudy-6-4
Copy link
Copy Markdown
Contributor Author

@Mergifyio queue

@mergify
Copy link
Copy Markdown

mergify bot commented Jan 30, 2026

queue

❌ The pull request has been removed from the queue main

Details

The merge conditions cannot be satisfied due to failing checks.

You can take a look at Queue: Embarked in merge queue check runs for more details about the failure.

@eudelins-zama
Copy link
Copy Markdown
Contributor

@Mergifyio requeue

@mergify
Copy link
Copy Markdown

mergify bot commented Jan 30, 2026

requeue

✅ This pull request will be re-embarked automatically

Details

The followup queue command will be automatically executed to re-embark the pull request.

@mergify
Copy link
Copy Markdown

mergify bot commented Jan 30, 2026

queue

✅ The pull request has been merged automatically

Details

The pull request has been merged automatically at 45525a7

@mergify mergify bot added queued and removed dequeued labels Jan 30, 2026
mergify bot added a commit that referenced this pull request Jan 30, 2026
@mergify mergify bot merged commit 45525a7 into main Jan 30, 2026
126 checks passed
@mergify mergify bot deleted the rudy/fix/host-listener/break-alloy-hidden-infinite-loop-on-bad-message branch January 30, 2026 11:34
@mergify mergify bot removed the queued label Jan 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants