Skip to content

README.md

Latest commit

 

History

History
131 lines (92 loc) · 4.28 KB

README.md

File metadata and controls

131 lines (92 loc) · 4.28 KB
Raw

Amplifier Ingester

The Amplifier Ingester is a component that consumes events from message queue and forwards them to the Amplifier API. It acts as a bridge between blockchain networks and the Axelar Amplifier protocol.

Overview

The ingester is designed to:

  • Consume events from a message queue (NATS or GCP Pub/Sub)
  • Process events concurrently for optimal throughput
  • Forward events to the Amplifier API with proper authentication
  • Handle retries and error scenarios gracefully

Architecture

The ingester follows a modular architecture where the core ingestion logic is separated from the infrastructure components. This allows for easy integration with different message queue systems.

Blockchain Subscriber → Events → Message Queue → Ingester → Amplifier API

Usage

For Blockchain Integration

Important: When integrating a new blockchain, you don't need to modify the amplifier-ingester crate. Instead:

  1. Implement a blockchain-specific subscriber that:

    • Monitors blockchain events (smart contract events, transactions, etc.)
    • Transforms blockchain data into Amplifier event format
    • Publishes events to the configured message queue (NATS/GCP) for amplifier ingester to process

  2. The amplifier-ingester will automatically:

    • Consume your published events from the queue
    • Forward them to the Amplifier API
    • Handle authentication, retries, and error scenarios

Configuration

The ingester requires a configuration file with the following sections:

# General ingester configuration
concurrent_queue_items = 100

[amplifier]
url = "https://amplifier-api.example.com"
chain = "ethereum"

[amplifier.identity]
# TLS identity configuration - see main README for details
# Option 1: Direct certificate (development)
# identity = "..."
# Option 2: Use with GCP KMS (production) - configure [gcp.kms] section

# For NATS
[nats]
urls = ["nats://localhost:4222"]
stream_name = "amplifier_events"
stream_subject = "events.*"
stream_description = "Amplifier event stream"
consumer_description = "Amplifier ingester consumer"
deliver_group = "ingester_group"

# For GCP Pub/Sub
[gcp]
project_id = "your-project-id"
subscription_id = "amplifier-events-sub"

Running

The ingester supports two message queue backends that are mutually exclusive:

# With GCP Pub/Sub (default)
cargo run --bin amplifier-ingester -- --config config.toml

# With NATS (requires disabling default features)
cargo run --bin amplifier-ingester --no-default-features --features nats -- --config config.toml

supervisor feature should be enabled if you want add it to development supervisor binary to start all relayer components as a single binary

Features

  • Concurrent Processing: Configurable number of concurrent amplifier api clients pushing data
  • Multiple Queue Support: NATS and GCP Pub/Sub implementations
  • TLS Authentication: Secure communication with Amplifier API
  • Error Handling: Automatic retries with exponential backoff
  • Observability: Integrated metrics and tracing
  • BigInt Precision: Supports forwarding BigInt features to amplifier-api for blockchain-specific numeric precision (see below)

Blockchain-Specific BigInt Configuration

The ingester forwards BigInt features to amplifier-api. See the main README for details.

# Solana with GCP
cargo build --bin amplifier-ingester --features bigint-u64

# Solana with NATS
cargo build --bin amplifier-ingester --no-default-features --features "nats,bigint-u64"

Development

Adding Support for New Message Queues

To add support for a new message queue system:

  1. Implement the Consumer trait from the infrastructure crate
  2. Add a new feature flag in Cargo.toml
  3. Create a new component module similar to components/nats.rs
  4. Update the main binary to support the new feature

Testing

Since GCP and NATS features are mutually exclusive, test each backend separately:

# Test with GCP (default)
cargo test

# Test with NATS
cargo test --no-default-features --features nats

Related Components