lwwallet: report a non-Esplora endpoint instead of parsing its HTML by litbot-9000 · Pull Request #1145 · lightninglabs/wavelength

litbot-9000 · 2026-08-14T02:17:20Z

What this fixes

A mainnet beta user hit this while onboarding:

unable to unlock wallet: start lwwallet: start tip poller:
get initial tip height: parse tip height: strconv.ParseInt:
parsing "<!doctype html>\n<html lang=\"en-US\" dir=\"ltr\">\n\n<head>...

followed by three kilobytes of the mempool.space frontend, inlined into the
gRPC error.

Their wallet.esploraurl was https://mempool.staging.lightningcluster.com,
missing the /api suffix. A mempool.space instance serves its
Esplora-compatible REST API under /api and serves an Angular single-page
app everywhere else. That app answers every unrouted path with index.html
and a 200, so GET /blocks/tip/height returns markup, EsploraClient.get
checks only the status code, and strconv.ParseInt gets the document.

The setting is one line of config, but nothing in the failure says so. The
error names a parser, is raised from inside wallet startup, and buries the
one useful signal (<!doctype html>) in its own noise.

The change

92e56768 — bound the body quoted back in an error. Every Esplora error
path interpolated the entire response body. Collapse it onto one line and cap
it at 256 bytes. The cut lands on a rune boundary; the body is arbitrary bytes
from an endpoint whose shape we have just decided not to trust, so slicing it
blindly would emit a half-encoded rune into the log.

34b5597e — reject an HTML response as a misconfigured endpoint. No
Esplora endpoint answers with HTML, so a response that does did not come from
the API. checkNotHTML runs after the status check on all four request paths
(get, post, TestMempoolAccept, SubmitPackage) and returns a wrapped
ErrNotEsploraAPI naming the URL and the likely fix. Exported so a caller can
tell a misconfiguration apart from a transient failure and decline to retry.

Same daemon, same bad URL, after:

unable to start wallet: start lwwallet: start tip poller:
get initial tip height: get tip height: endpoint is not an Esplora REST API:
https://mempool.staging.lightningcluster.com/blocks/tip/height answered with
an HTML page; a mempool.space or Esplora instance serves this API under /api,
so the URL likely needs that suffix (e.g. https://mempool.space/api)

Why content type and not the body

Body sniffing is the obvious implementation and it is wrong here.
/tx/:txid/raw and /block/:hash/raw return arbitrary binary, which can
legitimately begin with the same < byte an HTML document does — a
version-60 transaction serializes to a leading 0x3c.
TestEsploraAcceptsBinaryBodyStartingWithAngleBracket pins exactly that case
so a later "just check if it starts with <" simplification fails a test
rather than breaking raw block fetches.

The check is also deliberately narrow. It does not validate that the endpoint
is correct, only that it is not a web page. A wrong-but-JSON endpoint still
fails downstream, as it should.

What this does not change

The defaults are all fine already — every constant in lwwallet/defaults.go
carries the suffix, as does chainfees's separate /api/v1/fees/recommended
endpoint. This only affects a hand-set wallet.esploraurl.

Verification

Reproduced and fixed against the real deployment, not just a test server.

Built waved at 2c3af7d6 with -tags "wavewalletrpc swapruntime", ran it
on mainnet against mempool.staging.lightningcluster.com with no /api,
and wavecli create reproduced the reported error byte for byte.
With the branch, the same command produces the message quoted above.
With /api added, a fresh mainnet wallet comes up clean: tip poller synced
at height 962356, wallet_state: WALLET_STATE_READY, server_connected: true, and the operator correctly refuses the unlisted client
(is not whitelisted), which is the next gate rather than this bug.
Probed every endpoint the client uses against that host. All reads work,
including /scripthash/:hash/utxo in the forward byte order the code
documents (the reversed Electrum order returns [], as expected). One
unrelated gap found and reported separately: POST /api/tx 404s there,
while it routes on testnet4 and mempool.space — broadcast would fail on that
host.
TestEsploraRejectsHTMLResponse fails on main with the original
strconv.ParseInt error, confirming it pins the regression.
go test ./lwwallet/, make lint-changed-local, and
make commitmsg-lint range="origin/main..HEAD" all pass.

🤖 Generated with Claude Code

Every Esplora error path interpolated the whole response body. That is fine for the short status line a REST API returns, but a base URL that names a web frontend instead of the API root answers with a full HTML document, and the resulting error carried three kilobytes of markup up through the wallet, the daemon and out to the RPC client, where it is neither readable nor useful. Collapse the body onto one line and bound it, so a single-line log record stays a single line. The cut lands on a rune boundary: the body is arbitrary bytes from an endpoint whose shape we have just decided not to trust, and slicing it blindly would emit a half-encoded rune.

A mempool.space instance serves its Esplora-compatible REST API under /api, and serves a single-page web app everywhere else. That app answers every unrouted path with index.html and a 200, so a base URL missing the /api suffix passes the status check and hands markup to a parser. The operator sees the failure as parse tip height: strconv.ParseInt: parsing "<!doctype html>... raised from inside wallet startup, several layers away from the setting that caused it, with the whole document inlined. No Esplora endpoint answers with HTML, so a response that does did not come from the API. Reject it before the body reaches a parser and name the likely fix in the error. The check keys on the content type alone. Body sniffing would be wrong here: /tx/:txid/raw and /block/:hash/raw return arbitrary binary, which can legitimately begin with the same '<' byte an HTML document does. A test pins that case with a version-60 transaction, whose serialization starts with 0x3c. ErrNotEsploraAPI is exported so a caller can tell a misconfiguration apart from a transient failure and decline to retry it.

litbot-9000 · 2026-08-14T02:29:57Z

📘 Doc drift advisory — `lwwallet`

This PR's Go changes leave lwwallet/CLAUDE.md (and its AGENTS.md mirror) stale: the new exported sentinel ErrNotEsploraAPI and the two behaviours it comes with — the content-type-keyed HTML guard and the bounded error-body quoting — are not described anywhere in the package doc.

diff --git a/lwwallet/AGENTS.md b/lwwallet/AGENTS.md
index 3f4f8b4b..6bd768fd 100644
--- a/lwwallet/AGENTS.md
+++ b/lwwallet/AGENTS.md
@@ -41,7 +41,13 @@ base logic with the neutrino-backed `btcwbackend` sibling via the extracted
   (transactions, blocks, headers) are cached in LRU caches bounded by
   cumulative serialized byte size (see `esplora_cache.go`). Mutable live data
   (tip height, UTXOs, fee estimates) is never cached. Cache integrity: every
-  response is verified to hash to the requested key before insertion.
+  response is verified to hash to the requested key before insertion. Every
+  response also passes an endpoint-shape guard before reaching a parser; a
+  200 carrying an HTML body is rejected with `ErrNotEsploraAPI`.
+- `ErrNotEsploraAPI` — Sentinel error returned when an endpoint answers with
+  an HTML page. It marks a misconfigured base URL rather than a transient
+  failure, so callers can treat it as fatal instead of retrying. Match with
+  `errors.Is`; the message names the likely fix (the `/api` suffix).
 - `EsploraChainService` — `chain.Interface` adapter over `EsploraClient`,
   driven by a shared `TipPoller`. Feeds btcwallet's internal address-credit
   pipeline. Constructor: `NewEsploraChainService(esplora, tipPoller, logger)`.
@@ -89,6 +95,16 @@ base logic with the neutrino-backed `btcwbackend` sibling via the extracted
   (known limitation; acceptable for confirmation-target use cases).
 - LRU caches only hold immutable, hash-addressed data; a verified hash prevents
   a compromised Esplora endpoint from injecting arbitrary cache entries.
+- The HTML guard keys on `Content-Type` alone, never on the body bytes.
+  `/tx/:txid/raw` and `/block/:hash/raw` return arbitrary binary that can
+  legitimately begin with the same `<` byte an HTML document does, so
+  sniffing would reject valid responses. A missing or unparseable content
+  type falls through to the caller's own parser rather than being reported
+  as a misconfiguration.
+- Response bodies quoted back in an error are collapsed onto one line and
+  bounded to 256 bytes, cut on a rune boundary. A base URL naming a web
+  frontend answers with a full HTML page, and the unbounded form carried
+  kilobytes of markup through every layer up to the RPC client.
 - `scriptHashHex` hex-encodes the SHA256 digest in its natural byte order.
   Esplora's REST API differs from the Electrum wire protocol here, which
   reverses it, and a wrong order fails silently because the API answers an
diff --git a/lwwallet/CLAUDE.md b/lwwallet/CLAUDE.md
index 3f4f8b4b..6bd768fd 100644
--- a/lwwallet/CLAUDE.md
+++ b/lwwallet/CLAUDE.md
@@ -41,7 +41,13 @@ base logic with the neutrino-backed `btcwbackend` sibling via the extracted
   (transactions, blocks, headers) are cached in LRU caches bounded by
   cumulative serialized byte size (see `esplora_cache.go`). Mutable live data
   (tip height, UTXOs, fee estimates) is never cached. Cache integrity: every
-  response is verified to hash to the requested key before insertion.
+  response is verified to hash to the requested key before insertion. Every
+  response also passes an endpoint-shape guard before reaching a parser; a
+  200 carrying an HTML body is rejected with `ErrNotEsploraAPI`.
+- `ErrNotEsploraAPI` — Sentinel error returned when an endpoint answers with
+  an HTML page. It marks a misconfigured base URL rather than a transient
+  failure, so callers can treat it as fatal instead of retrying. Match with
+  `errors.Is`; the message names the likely fix (the `/api` suffix).
 - `EsploraChainService` — `chain.Interface` adapter over `EsploraClient`,
   driven by a shared `TipPoller`. Feeds btcwallet's internal address-credit
   pipeline. Constructor: `NewEsploraChainService(esplora, tipPoller, logger)`.
@@ -89,6 +95,16 @@ base logic with the neutrino-backed `btcwbackend` sibling via the extracted
   (known limitation; acceptable for confirmation-target use cases).
 - LRU caches only hold immutable, hash-addressed data; a verified hash prevents
   a compromised Esplora endpoint from injecting arbitrary cache entries.
+- The HTML guard keys on `Content-Type` alone, never on the body bytes.
+  `/tx/:txid/raw` and `/block/:hash/raw` return arbitrary binary that can
+  legitimately begin with the same `<` byte an HTML document does, so
+  sniffing would reject valid responses. A missing or unparseable content
+  type falls through to the caller's own parser rather than being reported
+  as a misconfiguration.
+- Response bodies quoted back in an error are collapsed onto one line and
+  bounded to 256 bytes, cut on a rune boundary. A base URL naming a web
+  frontend answers with a full HTML page, and the unbounded form carried
+  kilobytes of markup through every layer up to the RPC client.
 - `scriptHashHex` hex-encodes the SHA256 digest in its natural byte order.
   Esplora's REST API differs from the Electrum wire protocol here, which
   reverses it, and a wrong order fails silently because the API answers an

How to apply

Save the block above to a file and git apply it, or run the doc skill
locally scoped to just this package:

/doc-gardening lwwallet

Note on `make doc-check`

make doc-check currently reports two failures on this branch, both
pre-existing on the base commit and untouched by this PR (vtxo/CLAUDE.md
and vtxo/AGENTS.md have diverged; a .claude-pr/ runner artifact directory
has a CLAUDE.md with no AGENTS.md). They are left alone here so the
advisory diff stays scoped to lwwallet; the nightly full sweep owns them.

https://github.com/lightninglabs/wavelength/actions/runs/31763675478

The package doc described the Esplora client's cache-integrity check as the only thing standing between a response and a parser. It now also passes an endpoint-shape guard, and neither that guard nor the new exported ErrNotEsploraAPI sentinel appeared anywhere in the doc. Record both, along with the two decisions a future reader is most likely to want to undo: that the guard keys on Content-Type alone rather than sniffing the body, because the /raw endpoints return binary that can legitimately start with '<', and that a body quoted back in an error is bounded to 256 bytes.

litbot-9000 added 2 commits August 14, 2026 02:08

litbot-9000 force-pushed the esplora-html-endpoint-guard branch 2 times, most recently from bd76151 to bfa3169 Compare August 14, 2026 02:25

Navigation Menu

Uh oh!

lwwallet: report a non-Esplora endpoint instead of parsing its HTML - #1145

lwwallet: report a non-Esplora endpoint instead of parsing its HTML#1145
litbot-9000 wants to merge 3 commits into
mainfrom
esplora-html-endpoint-guard

litbot-9000 commented Aug 14, 2026

Uh oh!

litbot-9000 commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

litbot-9000 commented Aug 14, 2026

What this fixes

The change

Why content type and not the body

What this does not change

Verification

Uh oh!

litbot-9000 commented Aug 14, 2026

📘 Doc drift advisory — lwwallet

How to apply

Note on make doc-check

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

📘 Doc drift advisory — `lwwallet`

Note on `make doc-check`