Skip to content

[M2] Live NEAR Lake S3 streaming with shard filtering #5

@Fosurero

Description

@Fosurero

Description

Connect to the real NEAR Lake S3 bucket (testnet and mainnet) and stream blocks with shard filtering.

Tasks

  • AWS credential configuration (env vars + config file)
  • Live testnet streaming end-to-end
  • Mainnet support
  • Configurable S3 endpoint for custom providers
  • Backfill mode from arbitrary start height
  • Integration tests with real data snapshots

Acceptance Criteria

  • Successfully stream 100k+ consecutive testnet blocks
  • Only selected shards are processed
  • Checkpoint/resume on restart

Metadata

Metadata

Assignees

No one assigned

    Labels

    M2Milestone 2: Live NEAR Lake IntegrationenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions