refactor(esplora): clear remaining panic paths #2053

reez · 2025-10-01T13:23:27Z

Description

This PR clears remaining panic paths from bdk_esplora, replacing them with proper error handling.

Attempting to address bitcoindevkit/bdk_wallet#30 for Esplora

Notes to the reviewers

Open to any and all feedback on this.

Other PR's made along these lines:

refactor(electrum): remove unwrap()s and expect()s #1981 (electrum)
esplora: chain_update errors if no point of connection #1971 (esplora)

Changelog notice

Clear remaining panic paths from bdk_esplora.rs.

Checklists

All Submissions:

I followed the contribution guidelines

New Features:

I've added tests for the new feature
I've added docs for the new feature

Bugfixes:

This pull request breaks the existing API
I've added tests to reproduce the issue which are now passing
I'm linking the issue being fixed by this PR

ValuedMammal · 2025-10-01T15:42:58Z

crates/esplora/src/async_ext.rs

+        let last_index =
+            last_index.ok_or_else(|| Box::new(esplora_client::Error::InvalidResponse))?;


One suggestion is to leave last_index as an Option and wrap stop_gap in Some(..) when doing the comparison.

But there's a much better way to handle this, which is to introduce an unused_count variable. Now when processing handles, if txs.is_empty() we increment the unused count, else update the last active index (and reset the unused count). Finally, break once the unused count reaches the gap limit. AFAICT there'd be no reason to keep track of the last_index scanned.

Took your suggestion and worked it in; didn't want to push it to this branch yet if I had the wrong idea, but here is the commit reez@35538e2 on a branch that builds on this branch.

I felt like I had to keep two small guardrails from the old logic:

processed_any (still surface the “no handles ran” error)

gap_limit = stop_gap.max(1)(keeps old “stop_gap==0 still means one empty derivation” behavior)

I don't think it's an error if handles turns up empty - we need a way to break from the loop once all of the keychain-spks are consumed.

Maybe stop_gap = stop_gap.max(parallel_requests), since we check at least 1 spk-index for every request in parallel_requests.

Super helpful thanks mammal, I made the below changes to that my side branch esplora+mammal that I can eventually fold back into this esplora branch if I'm on the right track

I don't think it's an error if handles turns up empty - we need a way to break from the loop once all of the keychain-spks are consumed.

Good call. I made this change 36073d0 based on that

Maybe stop_gap = stop_gap.max(parallel_requests), since we check at least 1 spk-index for every request in parallel_requests.

Good catch, made another change 590edcf based on this

pulled these changes into this pr now

ValuedMammal · 2025-10-06T16:54:47Z

crates/esplora/src/blocking_ext.rs

-            let (index, txs, evicted) = handle.join().expect("thread must not panic")?;
+            let handle_result = handle
+                .join()
+                .map_err(|_| Box::new(esplora_client::Error::InvalidResponse))?;


Because you're dropping the error it won't be very informative to propagate it upward. Instead this module can define its own Error type to handle errors that aren't covered by esplora_client::Error.

Tried to address this with new commit 9ea0dd0, trying to do the most simple thing I could think of that addresses what you mentioned

evanlinjin

Thanks for working on this!

Some great simplifications in logic. Some parts are over-engineered.

Note that panics and expects indicate that "we should never hit this - unless there is a bug!" and I think it's justified to have them (where reasonable).

evanlinjin · 2025-10-17T04:23:29Z

crates/esplora/src/blocking_ext.rs

+#[derive(Debug)]
+pub enum Error {
+    Client(esplora_client::Error),
+    ThreadPanic(Option<String>),


This seems over-engineered. Thread panics are highly unlikely because:

No explicit panics in the thread - all errors are propagated.

We only call esplora_client methods - if the method panics, it's a bug in the esplora_client library and should be addressed in the upstream crate.

ThreadPanic is really only protection against bugs in the dependency chain (should be addressed upstream) or extreme memory exhaustion (which would crash the application anyway).

evanlinjin · 2025-10-17T04:45:02Z

crates/esplora/src/async_ext.rs

+    #[test]
+    fn ensure_last_index_none_returns_error() {
+        let last_index: Option<u32> = None;
+        let err = last_index
+            .ok_or_else(|| Box::new(esplora_client::Error::InvalidResponse))
+            .unwrap_err();
+        assert!(matches!(*err, esplora_client::Error::InvalidResponse));
+    }


I don't think this is needed. If this fails, it's a problem with rust.

evanlinjin · 2025-10-17T04:56:15Z

crates/esplora/src/blocking_ext.rs

        if handles.is_empty() {
+            if !processed_any {
+                return Err(esplora_client::Error::InvalidResponse.into());
+            }


Looks like you are trying to catch a empty-input? You'll need a justification on why an empty input should result in an error. My understanding is that the caller may create a FullScanRequest with an empty keychain and if they do, it's not problematic at all and we should just return nothing (which looks like what the original code is doing).

evanlinjin · 2025-10-17T04:59:05Z

crates/esplora/src/blocking_ext.rs

+    #[test]
+    fn thread_join_panic_maps_to_error() {
+        let handle = std::thread::spawn(|| -> Result<(), Error> {
+            panic!("expected panic for test coverage");
+        });
+
+        let res = (|| -> Result<(), Error> {
+            let handle_result = handle
+                .join()
+                .map_err(Error::from_thread_panic)?;
+            handle_result
+        })();
+
+        assert!(matches!(
+            res.unwrap_err(),
+            Error::ThreadPanic(_)
+        ));
+    }
+
+    #[test]
+    fn ensure_last_index_none_returns_error() {
+        let last_index: Option<u32> = None;
+        let err = last_index
+            .ok_or_else(|| Error::from(esplora_client::Error::InvalidResponse))
+            .unwrap_err();
+        assert!(matches!(err, Error::Client(esplora_client::Error::InvalidResponse)));
+    }
+


No need to test rust for bugs in our repo.

reez · 2025-10-22T18:25:33Z

crates/esplora/src/blocking_ext.rs

    tip = tip
        .extend(conflicts.into_iter().rev().map(|b| (b.height, b.hash)))
-        .expect("evicted are in order");
+        .map_err(|_| Error::from(esplora_client::Error::InvalidResponse))?;


Add error variant, Checkpoint error.

Checkpoint and wraps checkpoint type.

put in lib.rs, have async and blocking use new error

reez · 2025-10-22T18:26:44Z

crates/esplora/src/blocking_ext.rs


        if handles.is_empty() {
+            if !processed_any {
+                return Err(esplora_client::Error::InvalidResponse.into());


remove this

reez · 2025-10-22T18:28:36Z

crates/esplora/src/blocking_ext.rs

+                .join()
+                .map_err(Error::from_thread_panic)?;
+            let (index, txs, evicted) = handle_result?;
+            processed_any = true;


dont need processed_any

reez · 2025-10-22T18:29:09Z

crates/esplora/src/blocking_ext.rs

-            if !txs.is_empty() {
+            let handle_result = handle
+                .join()
+                .map_err(Error::from_thread_panic)?;


dont need this if not worried about panic on thread panic

reez · 2025-10-22T18:31:13Z

crates/esplora/src/blocking_ext.rs

                    client
                        .get_tx_info(&txid)
-                        .map_err(Box::new)
+                        .map_err(Error::from)


map this more explicitly

reez · 2025-10-22T18:31:33Z

crates/esplora/src/blocking_ext.rs

-            let (txid, tx_info) = handle.join().expect("thread must not panic")?;
+            let handle_result = handle
+                .join()
+                .map_err(Error::from_thread_panic)?;


dont worry about panic here

reez · 2025-10-22T18:31:58Z

crates/esplora/src/blocking_ext.rs

                    client
                        .get_output_status(&op.txid, op.vout as _)
-                        .map_err(Box::new)
+                        .map_err(Error::from)


be more explicit here too

reez · 2025-10-22T18:32:09Z

crates/esplora/src/blocking_ext.rs

-            if let Some(op_status) = handle.join().expect("thread must not panic")? {
+            let handle_result = handle
+                .join()
+                .map_err(Error::from_thread_panic)?;


dont worry about panic here

github-project-automation bot added this to BDK Chain Oct 1, 2025

reez force-pushed the esplora branch from 1d05ee7 to 481f92d Compare October 1, 2025 13:45

ValuedMammal reviewed Oct 1, 2025

View reviewed changes

oleonardolima moved this to In Progress in BDK Chain Oct 1, 2025

oleonardolima assigned reez Oct 1, 2025

reez mentioned this pull request Oct 2, 2025

bdk_wallet 2.3 or 3.0 bitcoindevkit/bdk-ffi#869

Open

reez marked this pull request as ready for review October 6, 2025 16:48

ValuedMammal reviewed Oct 6, 2025

View reviewed changes

reez changed the title ~~(draft) refactor(esplora): clear remaining panic paths~~ refactor(esplora): clear remaining panic paths Oct 9, 2025

refactor(esplora): clear remaining panic paths

0cbf254

reez force-pushed the esplora branch from 590edcf to 0cbf254 Compare October 13, 2025 18:35

refactor(esplora): define blocking Error enum

9ea0dd0

evanlinjin requested changes Oct 17, 2025

View reviewed changes

reez commented Oct 22, 2025

View reviewed changes

crates/esplora/src/blocking_ext.rs

client

.get_tx_info(&txid)

.map_err(Box::new)

.map_err(Error::from)

Copy link

Author

reez Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

map this more explicitly

reez commented Oct 22, 2025

View reviewed changes

		let last_index =
		last_index.ok_or_else(\|\| Box::new(esplora_client::Error::InvalidResponse))?;

Uh oh!

refactor(esplora): clear remaining panic paths #2053

Are you sure you want to change the base?

refactor(esplora): clear remaining panic paths #2053

Uh oh!

Conversation

reez commented Oct 1, 2025

Description

Notes to the reviewers

Changelog notice

Checklists

All Submissions:

New Features:

Bugfixes:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evanlinjin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

reez Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

reez Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

reez Oct 22, 2025 •

edited

Loading

reez Oct 22, 2025 •

edited

Loading