Modernize library: feature parity with Python reference, embedded Tor, full CLI#13
Open
v1b3coder wants to merge 11 commits intoopentimestamps:masterfrom
Open
Modernize library: feature parity with Python reference, embedded Tor, full CLI#13v1b3coder wants to merge 11 commits intoopentimestamps:masterfrom
v1b3coder wants to merge 11 commits intoopentimestamps:masterfrom
Conversation
This is the MSRV of the latest bitcoin_hashes.
Almost no problems here, which is nice.
These shouldn't be `# Heading`s because those display poorly in docs.rs output. Just use a word for the first line and let rustdoc decide how to format it.
Rust 2021 started with Rust 1.56. Our MSRV is 1.63. So move to 2021. No changes except the module improvements from Rust 2018.
This dependency is only used in the binary, not the library. To properly separate this we should use workspaces, but as an easy first step just make the dep optional. While we are at it, update to the latest version of env_logger.
Rewrite the core data model from a linear step-list to a proper DAG (BTreeMap<Op, Timestamp> + BTreeSet<Attestation>) matching the Python reference client's architecture. This enables correct round-trip serialization of complex multi-calendar timestamps. Core changes: - timestamp: DAG model with merge(), all_attestations(), nonce(), prune_to_height(), discard_attestations() methods - op: execute() now returns Result with input/output size limits (MAX_OP_LENGTH=4096); add Ord, Hash impls for BTreeMap keys - attestation: add Ord, Hash impls for BTreeSet storage; payload size cap (8192); verify_against_blockheader(); height type u64; sub-buffer parsing with trailing-byte rejection - ser: add from_hash(), hash_reader() for streaming file hashing; fix varint overflow; fix check_eof(); fix Serializer doc comment - error: add MsgTooLong, ResultTooLong, MergeMismatch, VerificationError, CommitmentNotFound, Http, Config variants; #[non_exhaustive]; replace deprecated cause() with source() - hex: add encode() function - lib: re-export new modules New modules: - merkle: cat_then_unary_op(), cat_sha256(), make_merkle_tree() - verify: DAG-walking verification with callback for block headers - bitcoin: make_timestamp_from_block() builds proof from block txns - calendar: RemoteCalendar client, UrlWhitelist, SOCKS5 proxy support, DEFAULT_AGGREGATORS (feature: calendar) - upgrade: upgrade_timestamp() resolves pending attestations via calendar servers (feature: calendar) - packetstream: 255-byte framed PacketReader/PacketWriter (feature: calendar) - rpc: BitcoinRpc JSON-RPC client, bitcoin.conf/cookie auth parsing (feature: cli) - cache: TimestampCache persistent on-disk cache (feature: cli) - bin/ots: unified CLI with info, verify, stamp, upgrade, prune subcommands (feature: cli) - tests/example_files: integration tests against .ots fixtures Remove ots-info binary, superseded by `ots info` subcommand. Build system: - features: calendar=["ureq"], socks5, cli=["clap","calendar", "env_logger","getrandom","serde","serde_json","dirs"] - fix renamed clippy lints (empty_enums, unchecked_time_subtraction, match_on_vec_items removed) - .gitignore: exclude vendor/ - Display formats now match Python output (lowercase op/hash names) 135 tests (122 unit + 13 integration), 0 clippy warnings.
Add `tor` feature with embedded Arti client (src/tor.rs). The new `--tor` CLI flag bootstraps a Tor circuit and routes all calendar connections through it — no external SOCKS5 proxy needed. `--tor` and `--socks5-proxy` are mutually exclusive (enforced via clap `conflicts_with`); both remain available. Migrate from ureq 2.x to ureq ~3.2, updating all call sites: AgentBuilder → config_builder, .set() → .header(), .send_bytes() → .send(), .into_reader() → .body_mut().as_reader(), Error::Status → Error::StatusCode. Explicitly select TlsProvider::NativeTls on every agent construction site, since arti-ureq pulls in ureq's rustls-no-provider feature (rustls without a CryptoProvider), which would panic at runtime if the default TLS provider were used. Replace per-struct SOCKS5 URL plumbing with agent injection. RemoteCalendar and BitcoinRpc now hold a ureq::Agent directly and expose .with_agent() to accept a pre-configured one (Tor-routed, SOCKS5-proxied, or custom). Remove the old build_agent(), with_socks5(), and the `socks5` feature flag. Route .onion Bitcoin RPC addresses through the Tor agent automatically. build_bitcoin_rpc() checks is_onion_address() and injects the Tor agent when the RPC URL is a hidden service; non-onion RPC connects directly (almost always localhost/LAN). Requesting a .onion RPC without --tor or --socks5-proxy now returns a clear error. Harden proxy failure handling: --tor and --socks5-proxy now exit the process on setup failure instead of silently falling back to direct connections, which would violate the user's privacy expectations. Hoist resolve_tor_agent() to command level so the agent is created once and shared across calendar upgrade and Bitcoin RPC, avoiding redundant Tor bootstraps in cmd_verify and per-file loops in cmd_upgrade.
- Bump version to 0.4.5, rewrite README for current feature set - Make Timestamp::new() fallible, enforce message size limits - Add prune module and CLI command, remove unused packetstream - Fix cache path to ~/.cache/ots, matching Python client - Add CLI integration tests for verify, upgrade, prune, and cache
Add GitHub Actions workflow that builds the ots CLI for five targets (x86_64/aarch64 Linux, x86_64/aarch64 macOS, x86_64 Windows) and creates a GitHub release with attached binaries on tag push. Decouple embedded Tor (Arti) from the cli feature: --features cli produces a ~4.5MB binary, --features cli-tor includes Arti (~26MB). Switch Arti deps from vendor paths to crates.io, vendor OpenSSL for cross-platform builds, use native-tls consistently.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey! I've been frustrated that the only full OpenTimestamps CLI implementation is in Python, and dealing with Python dependency hell every time I want to timestamp something got old fast. So I took Andrew's pull request and built it out into a full implementation on par with the Python reference (v0.4.5).
The result is ~6,500 new lines across 11 commits — a complete
otsCLI that can stamp, verify, upgrade, prune, and inspect.otsfiles, with optional Tor support baked in. No Python required.Key changes
Timestampis now a proper directed acyclic graph withmerge(),all_attestations(),prune_to_height(), anddiscard_attestations()DetachedTimestampFilewith support for all hash types, including round-trip fidelity with the Python implementationverify()walks the DAG and checks Bitcoin attestations;make_timestamp_from_block()builds proofs from raw blocksRemoteCalendar, URL whitelist,upgrade_timestamp()with multi-calendar supporttorfeature that routes calendar traffic through Tor for improved privacyotsCLI binary —stamp,upgrade,verify,info,prunesubcommands with persistent timestamp cache and Bitcoin Core RPC integrationcat_sha256(),make_merkle_tree()for batched stamping with nonce isolationDependency updates
bitcoin_hashesupgraded to 0.16ureq3.xFeature flags
calendar— calendar client (ureq)tor— embedded Tor via Articli— full CLI binary (includes calendar)cli-tor— CLI with Tor supportThe library without features enabled remains lightweight with only
bitcoin_hashesandlogas dependencies.Test plan
cargo test— 141 tests passcargo clippy --all-features— 0 warnings.otstest vectors fromopentimestamps-clientots stamp/ots upgrade/ots verifyend-to-end against live calendars