Make secp256k1 backend pluggable (coincurve/wallycore/pure-python) - #746
Merged
Conversation
- Add btcrecover/crypto_backends.py with auto-selection of coincurve, wallycore, or a bundled pure-Python fallback (with a startup warning) - Support BTCR_BACKEND env override to force a backend - Route EC operations through the backend in btcrseed, btcrpass (incl. Electrum 2.8 ECIES) and P2TR_tools - requirements.txt: coincurve optional, wallycore>=1.0.0 added - Add benchmark_crypto_backends.py - Linux CI now tests all three backends (Latest/Weekly base+full, Termux) - Document the three backends and relative performance in INSTALL.md
|
- coincurve install now installs requirements minus coincurve first (pulling in wallycore as fallback) and only then attempts coincurve, tolerating build failures (e.g. Python 3.14) instead of hard-failing the install step - Drop wallycore from the Termux matrix: it has no Android/aarch64 wheel and its source build fails under Termux, so only coincurve and pure-python are tested there
…lback - Verify via PyPI: neither coincurve 20.0.0 (wheels cp38-cp312) nor 21.0.0 (wheels cp39-cp313) ships a Python 3.14 wheel, and both fail to build from source on 3.14. So coincurve simply cannot be installed on 3.14 today. - Keep the CI coincurve install as 'coincurve==21.0.0 || coincurve==20.0.0 || true' (gets a wheel on 3.10-3.13) with '|| true' so 3.14 falls back to the already installed wallycore backend instead of failing the install step - Update docs/INSTALL.md and skills/install-btcrecover/windows/SKILL.md: the previous claim that 'coincurve==20.0.0 has a 3.14 wheel' is incorrect; on 3.14 users should install wallycore (or rely on the pure-Python fallback)
- 3.13 -> coincurve==21.0.0 (only version with a 3.13 wheel) - 3.10/3.11/3.12 -> coincurve==21.0.0 || 20.0.0 (both have wheels) - 3.14 -> no coincurve wheel; install allowed to fail and the job runs on the already-installed wallycore fallback backend
Use PEP 508 python_version markers so pip installs the coincurve release that
actually has a prebuilt wheel for the running interpreter:
- python_version < 3.13 -> coincurve==20.0.0 (wheels cp38-cp312)
- python_version >= 3.13 and < 3.14 -> coincurve==21.0.0 (cp313 wheel)
- python_version >= 3.14 -> omitted (no coincurve wheel exists); BTCRecover
falls back to the wallycore backend instead
This replaces the previous platform_machine-based split, which is no longer
needed since both releases now ship aarch64 and x86_64 wheels. Updated in
requirements.txt, requirements-full.txt, and the commented example in
requirements-walletfinder.txt.
… only - test_walletfinder.py: the 3 argument-parsing tests referenced old argparse attribute names (args.wallet_mode / args.text_mode) that no longer exist after the backward-compat flags were renamed to *_compat. Update them to the current API (skip_wallet_mode default, text_mode_compat, wallet_mode_compat). These 3 errors were failing every Linux/Windows/macOS test run (pre-existing on master). - termux-tests.yml: on Android/aarch64 neither coincurve nor wallycore has a wheel or builds from source (and bip-utils forces coincurve), so Termux can only run the bundled pure-Python backend. Test that backend only, excluding the unbuildable C libraries from the install.
…ve, which cannot build on aarch64)
Termux/aarch64 cannot use coincurve (ofek/coincurve#189: build/runtime incompatibility), so restore the original single-backend Termux jobs. Exclude wallycore from the main install so only coincurve is attempted (matching the historical Termux setup), avoiding the wallycore-from-source failure introduced by adding wallycore to the base requirements.
…emp file On Windows, 'pip install -r /dev/stdin' fails with 'Could not open requirements file: /proc/self/fd/0' (no /proc). Write the filtered requirements to $RUNNER_TEMP/btcr-reqs.txt and install from that file instead. This makes the coincurve install step (which previously failed on all Windows Python versions) work, falling back to the wallycore wheel on 3.14.
The YAML block scalar preserved the backslash-escaped quotes as literal characters, so the shell received a path wrapped in literal backslashes and the temp requirements file was never created. Use plain double quotes.
bip-utils (and py_crypto_hd_wallet, slip10, stellar_sdk) hard-require coincurve, which has no wheel and cannot build on Python 3.14. Exclude that subtree from the Full requirements install on 3.14 so the job installs using the wallycore fallback instead of failing on a coincurve source build.
…ch64) Mirrors the Weekly/Latest Full fix: bip-utils and its dependents hard-require coincurve, which has no aarch64 wheel and fails to build on Termux, so exclude that subtree from the Termux Full install.
- btcrecover/aes_backends.py: Unified AES & ChaCha20-Poly1305 backend (pycryptodome -> pure-python fallback) - lib/aes_gcm.py: Pure-python AES-GCM using pyaes + GHASH - lib/chacha20_poly1305.py: Pure-python ChaCha20 + Poly1305 (RFC 8439) - lib/keccak.py: Pure-python Keccak-256 (Ethereum-compatible) - lib/eth_hash/backends/purepython.py: eth_hash backend wrapping lib.keccak - btcrpass.py: Use aes_backends instead of direct Crypto.Cipher imports; improve protobuf error handling (exit -> ValueError/False) - emip3.py: Use chacha20_poly1305_new from aes_backends - requirements.txt: Document optional dependencies - CI: Exclude pycryptodome/protobuf in purepython backend scenario so the pure-python fallbacks are actually exercised
load_aes256_library() returns _AESModule (the pure-python fallback), but callers access __name__ on the result to estimate password speed. Without __name__, the CI pure-python scenario crashed with: AttributeError: '_AESModule' object has no attribute '__name__'
The old 5-limb carry-chain decomposition was broken — when h values
exceeded 32 bits, it produced integers too large for struct.pack('<I', ...),
causing OverflowError in the pure-python backend CI scenario.
Rewrite _poly1305_mac using clean big-integer arithmetic (int.from_bytes /
to_bytes) that is provably correct and matches RFC 8439 test vectors.
Also fix a minor struct.pack call in the section about patating the other.
Bug found by CI testing with pycryptodome excluded.
- _inc32 now increments only the low 32 bits (GCM spec), preventing values >128 bits that caused OverflowError: int too big to convert - _compute_j0 handles arbitrary-length IVs via GHASH (SP 800-38D), fixing Metamask/btc_com wallets that use 16-byte IVs
…backends in under 24h
Sets BTCR_BACKEND env var for all subprocesses so users can compare coincurve, wallycore, and pure-python performance from a single tool.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace the hard
coincurvedependency with a pluggable secp256k1 backend so BTCRecover works in environments without pre-built C-extension wheels (e.g. Termux, unusual platforms).btcrecover/crypto_backends.pywith automatic backend selection:requirements-fulldep)lib/ecpy) — prints a startup warningBTCR_BACKENDenv var forces a backend (coincurve/wallycore/purepython), falling back if unavailable.btcrseed.py,btcrpass.py(incl. Electrum 2.8 ECIES + pickling) andlib/p2tr_helper/P2TR_tools.py.requirements.txt: coincurve is now optional;wallycore>=1.0.0added.benchmark_crypto_backends.pycomparing pubkey derivation, P2TR tweak, and ECIES multiply across backends.Latest-*base+full,Weekly-Base,termux-tests).Performance
For a standard BIP-39 recovery (public-key derivation dominated), the C backends are ~150× faster than pure-Python (~34–36k ops/s vs ~240 ops/s). Electrum 2.8 ECIES is only accelerated by coincurve (~48k ops/s); wallycore and pure-Python fall back to the bundled pure-Python EC code (~2k ops/s).
Test plan
🤖 Generated with opencode