Skip to content

gibz104/Rethix

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

13 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Rethix ๐Ÿš€

High-Performance Ethereum Blockchain Indexer Built on Reth

Real-time blockchain data extraction at unprecedented speed

Rust PostgreSQL Ethereum

Build Reth Version


๐ŸŽฏ What is Rethix?

Rethix (Reth + Index) is a cutting-edge Ethereum blockchain indexer that leverages Reth's Execution Extensions (ExEx) framework to deliver unparalleled indexing performance. Built in Rust for maximum efficiency, Rethix transforms raw blockchain data into queryable PostgreSQL tables in real-time.

Why Rethix?

  • โšก Lightning Fast: Process thousands of blocks per second with native Rust performance
  • ๐Ÿ”„ Real-time & Historical: Seamlessly handles both live tip-following and historical backfilling
  • ๐Ÿ“Š Modular Datasets: Choose exactly what data you need - from basic headers to complex DeFi analytics
  • ๐Ÿ› ๏ธ Production Ready: Battle-tested with automatic recovery, gap detection, and resumable processing
  • ๐Ÿ”Œ Full Reth Integration: Access all standard Reth node capabilities plus advanced indexing

๐Ÿš€ Quick Start

Prerequisites

  • Rust 1.75+
  • PostgreSQL 14+
  • ~2TB SSD for full Ethereum mainnet sync
  • 32GB+ RAM recommended

Installation

# Clone the repository
git clone https://github.com/yourusername/rethix
cd rethix

# Build with optimizations
cargo build --release

# Run with indexing enabled
./target/release/rethix indexer \
  --chain mainnet \
  --enabled-datasets Headers,Transactions,Logs \
  --database-conn-str postgresql://user:pass@localhost:5432/rethix_db

๐Ÿ“ฆ Available Datasets

Rethix offers granular control over what blockchain data you index:

Core Data

  • Headers - Block headers with metadata
  • Transactions - Complete transaction details
  • Logs - Smart contract event logs
  • Traces - Internal transaction traces

Token Analytics

  • Erc20Transfers - ERC-20 token transfers
  • Erc20Metadata - Token names, symbols, decimals
  • NativeTransfers - ETH transfers (including internal)

Smart Contracts

  • Contracts - Contract deployments and bytecode

DeFi Analytics

  • UniV2Pools - Uniswap V2 pool creation events
  • UniV3Pools - Uniswap V3 pool deployments
  • UniV4Pools - Uniswap V4 hooks and pools
  • UniV2PoolVolumeTVL - V2 volume and liquidity metrics
  • UniV3PoolVolumeTVL - V3 volume and liquidity metrics

๐ŸŽฎ Usage Examples

Basic Indexing

# Index only headers and transactions
rethix indexer --enabled-datasets Headers,Transactions \
  --database-conn-str $DATABASE_URL

Full DeFi Analytics

# Index all Uniswap data with high parallelism
rethix indexer \
  --enabled-datasets UniV2Pools,UniV3Pools,UniV2PoolVolumeTVL,UniV3PoolVolumeTVL \
  --backfill-workers 8 \
  --backfill-batch-size 200 \
  --database-conn-str $DATABASE_URL

Custom Performance Tuning

rethix indexer \
  --enabled-datasets Headers,Transactions,Logs,Traces \
  --backfill-workers 16 \
  --backfill-batch-size 500 \
  --backfill-queue-size 2000 \
  --db-pool-size 64 \
  --db-write-batch-size 50000 \
  --database-conn-str $DATABASE_URL

๐Ÿ”ง Configuration

Command-Line Options

Option Description Default
--enabled-datasets Comma-separated list of datasets to index None
--database-conn-str PostgreSQL connection string $DATABASE_URL
--backfill-workers Number of parallel backfill workers 4
--backfill-batch-size Blocks per backfill batch 100
--backfill-queue-size Maximum pending batches 1000
--db-pool-size Database connection pool size 32
--db-write-batch-size Rows per database write 1000

Configuration File

Create a config.yaml for Uniswap pool configurations:

# Uniswap V2 pool configurations (for volume/TVL tracking)
UniV2PoolVolumeTVL:
  pools:
    - "0xb4e16d0168e52d35cacd2c6185b44281ec28c9dc"  # USDC/ETH
    - address: "0x0d4a11d5eeaac28ec3f61d100daf4d40471f1852"  # ETH/USDT
      start_block: 10008355  # Optional: specify when pool was created

# Uniswap V3 pool configurations
UniV3PoolVolumeTVL:
  pools:
    - "0x8ad599c3a0ff1de082011efddc58f1908eb6e6d8"  # USDC/ETH 0.3%
    - "0x88e6a0c2ddd26feeb64f039a2c41296fcb3f5640"  # USDC/ETH 0.05%

๐ŸŒ RPC Interface

Rethix exposes RPC methods for runtime control:

Manual Backfill

# Backfill specific dataset for block range
curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"rethix_startDatasetBackfill","params":["Headers",1000000,2000000],"id":1}' \
  http://localhost:8545

Control Commands

# Pause all backfilling
curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"rethix_pauseBackfill","params":[],"id":1}' \
  http://localhost:8545

# Check status
curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"rethix_backfillStatus","params":[],"id":1}' \
  http://localhost:8545

๐Ÿ“Š Database Schema

Rethix automatically creates optimized tables for each dataset:

-- Example: Headers table
CREATE TABLE headers (
    block_number BIGINT PRIMARY KEY,
    block_hash BYTEA NOT NULL,
    parent_hash BYTEA NOT NULL,
    timestamp BIGINT NOT NULL,
    gas_used NUMERIC(20,0),
    gas_limit NUMERIC(20,0),
    base_fee_per_gas NUMERIC(20,0),
    -- ... additional fields
);

-- Automatic indexing for performance
CREATE INDEX idx_headers_timestamp ON headers(timestamp);
CREATE INDEX idx_headers_hash ON headers(block_hash);

๐Ÿ—๏ธ Architecture

Two-Phase Processing

  1. Tip Following ๐Ÿ”„

    • Real-time processing of new blocks
    • Sub-second latency from block commitment
    • Automatic chain reorganization handling
  2. Backfilling ๐Ÿ“š

    • Parallel historical data processing
    • Resumable from any interruption
    • Automatic gap detection and filling

Performance Features

  • Work-Stealing Queue: Dynamic load balancing across workers
  • Watermark Tracking: Efficient resumable processing
  • Batch Processing: Optimized database writes
  • Memory Management: Automatic backpressure and flow control

๐Ÿ“ License

Licensed under the MIT License.

๐Ÿ™ Acknowledgments

Built on the incredible foundation of Reth by Paradigm.


Rethix - Indexing Ethereum at the Speed of Rust ๐Ÿฆ€

About

Ethereum indexer using Reth Execution Extensions (ExEx)

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages