solution: ability to archive as plain JSON (or HEX) files with dir pe…

[splix]

…r height

[Copilot]

Pull request overview

This PR adds a new JSON-per-field archival format (plus .hex raw-tx files) and introduces a new storage capability split so commands can target backends that can scan for missing heights without requiring full file read/merge support.

Changes:

• Add JSON-per-field format encoder (formats::json) plus new filesystem and S3 (object_store) storage backends for that layout.
• Refactor storage capabilities by introducing ScanTarget (resume/missing detection) separate from ReadTarget (list/open/delete), and update stream/fix to depend on scanning only.
• Add --format {avro,json} CLI switch and reject verify/compact for JSON format at startup.

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/storage/mod.rs	Adds JSON storage constructors, introduces ScanTarget, and factors out find_incomplete_by_listing
src/storage/fs.rs	Implements ScanTarget for filesystem Avro backend via shared listing-based scanner
src/storage/objects.rs	Implements ScanTarget for S3 Avro backend via shared listing-based scanner
src/storage/json_fs.rs	New filesystem JSON-per-field backend + tests
src/storage/json_objects.rs	New object_store JSON-per-field backend + tests
src/formats/mod.rs	Exposes formats::json
src/formats/json.rs	Implements JSON-per-field encoding + missing-height heuristics + tests
src/args.rs	Adds --format option and Format enum
src/main.rs	Dispatches commands based on backend capabilities and selected format
src/command/stream.rs	Relaxes target bound from ReadTarget to ScanTarget
src/command/fix.rs	Relaxes target bound from ReadTarget to ScanTarget
src/archiver/filenames.rs	Adds height_dir() helper for per-height directory layouts
src/record.rs	Clarifies TxRaw semantics (decoded bytes; re-encode per chain on output)
src/blockchain/ethereum.rs	Adjusts raw-tx handling to decode hex payload into bytes
README.adoc	Documents new --format flag in CLI help output
Cargo.toml / Cargo.lock	Adds tempfile for new filesystem JSON storage tests

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[coveralls]

Coverage Report for CI Build 26260196243

Coverage increased (+6.9%) to 49.769%

Details

• Coverage increased (+6.9%) from the base build.
• Patch coverage: 112 uncovered changes across 5 files (139 of 251 lines covered, 55.38%).
• 10 coverage regressions across 5 files.

Uncovered Changes

File	Changed	Covered	%
src/storage/mod.rs	75	27	36.0%
src/main.rs	23	0	0.0%
src/storage/json_objects.rs	59	40	67.8%
src/formats/json.rs	50	34	68.0%
src/storage/json_fs.rs	40	34	85.0%

Coverage Regressions

10 previously-covered lines in 5 files lost coverage.

File	Lines Losing Coverage	Coverage
src/command/verify.rs	3	45.93%
src/metrics/mod.rs	3	10.53%
src/archiver/range_group.rs	2	83.33%
src/archiver/blocks_config.rs	1	43.75%
src/archiver/datakind.rs	1	81.82%

---

Coverage Stats

Relevant Lines:	2166
Covered Lines:	1078
Line Coverage:	49.77%
Coverage Strength:	1.6 hits per line

---

💛 - Coveralls