Skip to content

nxm-rs/apiary

BranchesTags

Repository files navigation

apiary

CI License: AGPL-3.0

The hive. The enclave. Where Swarm bees live, work, and get stress-tested.

A Kurtosis package that spins up a complete Ethereum Swarm environment in seconds. Reth, Bee nodes, contracts, and an apiarist ready to poke everything.

One-liner Deployment

kurtosis run github.com/nxm-rs/apiary

That's it. You now have:

  • Reth in dev mode (because Geth is so 2020)
  • A bootnode and 3 Bee workers
  • Smart contracts deployed and ready
  • Apiarist standing by to torture-test everything
  • Observability stack (Prometheus, Grafana, Tempo for distributed tracing)

Configuration

# Use a configuration file
kurtosis run github.com/nxm-rs/apiary --args-file network_params.json

# Or inline JSON (positional argument after package path)
kurtosis run github.com/nxm-rs/apiary '{"bee_count": 5, "verbosity": 3}'

# Run locally
kurtosis run .

Parameters

Parameter Type Default What it does
full_node_count int 4 Number of full Bee nodes
light_node_count int 0 Number of light Bee nodes
bee_image string ethersphere/bee:latest Bee Docker image
reth_image string ghcr.io/paradigmxyz/reth:latest Reth Docker image
network_id int 0 Swarm network ID
enable_swap bool true Enable SWAP payment protocol
verbosity int 5 Log verbosity (0-5)
enable_observability bool true Deploy Prometheus/Grafana stack
checks_to_run list ["pingpong", "peercount", "fullconnectivity", "smoke"] Tests to run
check_options object {} Per-check configuration (see below)
apiarist_image string ghcr.io/nxm-rs/apiarist:latest Apiarist Docker image
check_timeout string 10m Max time for all checks

Integration Tests

Apiary runs integration tests via Apiarist, a Rust-based testing tool that runs inside the enclave.

Available Checks

P0 - Core Protocol:

Check Description
pingpong Peer connectivity and RTT measurement
peercount Validate minimum peer counts
fullconnectivity Full mesh connectivity test

P1 - Data Operations:

Check Description
smoke Upload/download sanity check (supports long-running mode)

Coming Soon:

  • pushsync, retrieval, fileretrieval

Smoke Check Configuration

The smoke check supports both single-shot and long-running modes:

Single-shot (default): Uploads 1KB, downloads from all nodes, exits.

Long-running (like beekeeper): Continuously uploads/downloads for a specified duration.

{
  "checks_to_run": ["smoke"],
  "check_timeout": "13h",
  "check_options": {
    "smoke": {
      "duration_secs": 43200,
      "file_sizes": [1024, 10240, 102400, 1048576],
      "sync_wait_ms": 60000,
      "iteration_wait_secs": 300
    }
  }
}
Option Default Description
duration_secs 0 (single-shot) Total run duration in seconds
data_size 1024 File size in bytes (single size)
file_sizes [1024] Multiple file sizes to test per iteration
sync_wait_ms 2000 Wait after upload for network sync
iteration_wait_secs 5 Wait between iterations
seed random PRNG seed for reproducible tests
batch_id auto Specific postage batch ID to use

Running Specific Checks

# Run only pingpong
kurtosis run github.com/nxm-rs/apiary '{"checks_to_run": ["pingpong"]}'

# Run all P0 checks (default)
kurtosis run github.com/nxm-rs/apiary

# Skip checks entirely
kurtosis run github.com/nxm-rs/apiary '{"checks_to_run": []}'

Monitoring Test Progress

# View apiarist logs
kurtosis service logs <enclave> apiarist -f

# Check status API
PORT=$(kurtosis port print <enclave> apiarist api | cut -d':' -f2)
curl http://localhost:$PORT/status

# View Prometheus metrics
curl http://localhost:$PORT/metrics

Example Configurations

Minimal Setup (2 nodes, no SWAP, no observability):

{
  "full_node_count": 2,
  "enable_swap": false,
  "enable_observability": false,
  "verbosity": 3,
  "checks_to_run": ["pingpong"]
}

Stress Test (10 nodes, all checks):

{
  "full_node_count": 10,
  "verbosity": 5,
  "checks_to_run": ["pingpong", "peercount", "fullconnectivity", "smoke"]
}

Long-Running Smoke Test (12 hours, like beekeeper):

{
  "full_node_count": 4,
  "checks_to_run": ["smoke"],
  "check_timeout": "13h",
  "check_options": {
    "smoke": {
      "duration_secs": 43200,
      "file_sizes": [1024, 10240, 102400, 1048576],
      "sync_wait_ms": 60000,
      "iteration_wait_secs": 300
    }
  }
}

Use network_params_smoke_longrun.json for the long-running configuration.

Services

Reth (reth-swap)

Ethereum execution client in development mode:

  • HTTP RPC: http://reth-swap:8545
  • WebSocket RPC: ws://reth-swap:8546
  • Metrics: http://reth-swap:9001/metrics
  • Block time: 1 second

Bee Bootnode

Bootstrap node with hardcoded keys:

  • API: http://bootnode:1633
  • P2P: tcp://bootnode:1634

Bee Nodes (bee-0, bee-1, ...)

Regular Bee nodes:

  • API: http://bee-N:1633
  • P2P: tcp://bee-N:1634

Apiarist

The testing service:

  • Status API: http://apiarist:8080/status
  • Results: http://apiarist:8080/results
  • Health: http://apiarist:8080/health

Observability Stack

  • Grafana: http://grafana:3000 (anonymous admin access)
  • Prometheus: http://prometheus:9090 (metrics)
  • Tempo: http://tempo:3200 (distributed tracing)

Logs

Kurtosis doesn't allow Docker socket mounting, so in-enclave log collection is not possible. Access logs via:

# Kurtosis native logs
kurtosis service logs <enclave> bootnode
kurtosis service logs <enclave> bee-0 -f

# Or run grafloki before deployment for external Loki
kurtosis grafloki start

Architecture

                            +------------------------------------------+
                            |              Kurtosis Enclave            |
                            +------------------------------------------+
                            |                                          |
  +----------------+        |   +--------------+                       |
  |  Observability |        |   |    Reth      |  (dev mode, 1s)       |
  |  Stack         |        |   |   reth-swap  |                       |
  |                |        |   +------+-------+                       |
  |  Grafana <-----+--------+          |                               |
  |  Prometheus    |        |          | RPC                           |
  |  Tempo         |        |          v                               |
  |   (tracing)    |        |   +--------------+                       |
  +-------^--------+        |   |   Bootnode   | <-------------+       |
          |                 |   +------+-------+               |       |
          | traces                     | P2P                   |       |
          |                 |          v                       |       |
          |                 |   +------+------+--------+       |       |
          |                 |   |             |        |       |       |
          +-----------------+   v             v        v       |       |
                            | +-----+     +-----+   +-----+    |       |
                            | |bee-0|     |bee-1|   |bee-2|    |       |
                            | +-----+     +-----+   +-----+    |       |
                            |                                  |       |
                            |   +--------------------------+   |       |
                            |   |       Apiarist           |---+       |
                            |   |   (stress-tests all)     |           |
                            |   +--------------------------+           |
                            |                                          |
                            +------------------------------------------+

Kurtosis Commands

# List enclaves
kurtosis enclave ls

# View logs
kurtosis service logs <enclave> reth-swap
kurtosis service logs <enclave> bootnode
kurtosis service logs <enclave> bee-0 -f

# Access services from host
PORT=$(kurtosis port print <enclave> bootnode api | cut -d':' -f2)
curl http://localhost:$PORT/health

# Clean up
kurtosis enclave rm <enclave>
kurtosis clean -a

Why Kurtosis?

Beekeeper exists and it's fine. But have you tried not running Kubernetes locally?

Aspect Beekeeper Apiary
Backend Kubernetes only Docker or Kubernetes
Setup Time K8s startup time However fast Docker starts
Configuration YAML files Programmable Starlark
Local Dev Requires K3s/K8s Works with plain Docker

Why Reth?

  1. It's Rust - You know why
  2. Dev mode is nice - No consensus layer shenanigans
  3. Prometheus metrics - For when you want to stare at dashboards
  4. Modern codebase - Written this decade

Etymology

An apiary is where bees live. This is where your Bee nodes live. The metaphor is complete. You're welcome.

Development

apiary/
+-- kurtosis.yml              # Package metadata
+-- main.star                 # Entry point
+-- Makefile                  # Development helpers
+-- network_params*.json      # Config examples
+-- src/
    +-- reth.star             # Ethereum client
    +-- bee.star              # Bee nodes
    +-- contracts.star        # Contract deployment
    +-- apiarist.star         # Testing integration
    +-- funding.star          # Node funding
    +-- observability.star    # Grafana/Prometheus
    +-- keygen.star           # Key generation

Local Development with Apiarist

Use locally-built apiarist without publishing Docker images:

# Build local apiarist and run full test suite
make run-local

# Run with minimal config (faster iteration)
make build-apiarist  # Build apiarist:local
kurtosis run . --args-file network_params_local.json

# Monitor and debug
make logs      # Follow apiarist logs
make status    # Check test status
make results   # Get full test results
make metrics   # View Prometheus metrics

# Cleanup
make clean     # Remove enclave

Available configurations:

  • network_params.json - Full setup (4 nodes, observability, all checks including smoke)
  • network_params_minimal.json - Minimal (2 nodes, no observability, pingpong only)
  • network_params_local.json - Local dev (4 nodes, uses apiarist:local image)
  • network_params_large.json - Large cluster (10 nodes, all checks)
  • network_params_smoke_longrun.json - Long-running smoke test (12 hours, like beekeeper)

License

AGPL-3.0-or-later - because sharing is caring.

About

Spin up a decentralized storage network faster than your K8s cluster boots. Reth + Bee + vibes.

nxm.rs

Topics

testing devops ethereum swarm bee kurtosis

Resources

Readme

License

AGPL-3.0 license

Contributing

Contributing
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages