A lightweight, extensible cryptocurrency price aggregator service.
Built to pull real-time and historical crypto prices from external APIs (starting with CoinGecko), with architecture designed to easily support new data providers in the future. Designed for reliability, extensibility, and ease of local development.
Current Provider: CoinGecko Public API (no API key required)
This service fetches and caches cryptocurrency pricing data and exposes it via RESTful API endpoints and an interactive UI.
Core functionality includes:
- Fetching real-time crypto prices.
- Fetching historical prices for specific dates.
- Fetching price history over a date range (optimized for API limits).
- Caching all fetched prices locally in SQLite for fast, repeated access and reduced API calls.
- Always serving data from the local cache when available, and only falling back to the live API when necessary.
- Extensible provider architecture to easily add more APIs (e.g., CoinMarketCap).
- Local-first development with no cloud dependencies, no API keys needed.
- Backend API: FastAPI (Python)
- Frontend UI: Streamlit
- Database: SQLite (local, file-based)
- Testing: pytest
- Extensible Provider Layer: Abstract base class to swap in new data providers easily.
- Caching Layer: SQLite-based, with future Redis/memory-based caching possible.
You can download and watch the demo video here.
crypto_price_aggregator/
├── api_server/
│ ├── app/
│ │ ├── main.py # FastAPI app and routes
│ │ ├── routes.py # API route handlers
│ │ ├── services.py # Business logic, database access
│ │ ├── models.py # Pydantic models
│ │ ├── db.py # SQLite database utilities
│ │ ├── utils.py # Common utilities (e.g., retry logic)
│ │ └── providers/
│ │ ├── base.py # Abstract base provider
│ │ └── coingecko_provider.py # CoinGecko implementation
│ ├── tests/
│ │ ├── conftest.py # Test setup, dependency overrides
│ │ ├── test_api.py # API endpoint tests
│ │ └── test_services.py # Service layer tests
├── ui/
│ └── app.py # Streamlit frontend app
├── requirements.txt # Python dependencies
├── Makefile # CLI commands (run, test, clean)
└── README.md
| Endpoint | Method | Description | Query Params |
|---|---|---|---|
/latest |
GET | Get the latest price for a given asset | asset (e.g., bitcoin) |
/history/date |
GET | Get the price of an asset on a specific date | asset, date (YYYY-MM-DD) |
/history |
GET | Get price history over a date range | asset, from_date, to_date |
/assets |
GET | Get supported asset list | – |
/health |
GET | Health check endpoint | – |
CoinGecko's free tier imposes strict API limits.
- Retries: On 429 (rate limited) responses, we apply exponential backoff retry logic.
- Throttling: Requests to fetch historical data are rate-limited at the code level with controlled sleeps (e.g.,
time.sleep(1.5)) between requests to avoid triggering rate limits.
- Checking Local Cache (SQLite): Reuse previously cached data where available.
- Smart Sampling: When the range is too large, sample a subset of dates (e.g., 10 dates evenly spaced) instead of fetching every single day.
- On-Demand Fetch: Only fetch missing dates via the API, reducing total API calls and load.
For users scaling beyond the free API tier or needing high-frequency, real-time updates, we optionally integrate a Rust-based orchestration layer:
- Runs a fast background service to periodically fetch and push prices to the database at regular intervals (sub-minute granularity).
- Streams live metrics to a terminal dashboard (using Ratatui).
- Why Rust? Low overhead, highly concurrent fetching, minimal resource usage compared to Python threads.
- When Needed: For serious production setups using Pro API Subscriptions with higher rate limits. For local development and demo purposes, this orchestration layer is optional and not enabled by default.
- Python 3.11+
- pip3
- (Optional) Make
-
Clone the repo
git clone https://github.com/yourusername/crypto_price_aggregator.git cd crypto_price_aggregator -
Install dependencies
make install # Or manually: pip install -r requirements.txt -
Run the Application
make run
- FastAPI backend available at: http://localhost:8000
- Streamlit UI available at: http://localhost:8501
-
Run Tests
make test
- Unit tests for the service layer (business logic, price fetching, caching).
- Integration tests for the FastAPI REST API.
- All external API calls are mocked during tests — no real API calls during test runs.
- Provider Abstraction: Easily add new providers by implementing a simple interface.
- Caching Layer: Prices are stored in SQLite for fast retrieval.
- Rate-Limited API Usage: Smart retry logic with exponential backoff.
- Historical Range Optimization: Minimized API calls through local caching and sampling.
- Streamlit Frontend: User-friendly frontend to query and visualize price data.
- Minimal Local Setup: No Docker required, runs entirely locally.
- Extensible Design: Additional assets and providers can be added with minimal changes.
- (Optional) Rust Orchestrator: For continuous high-frequency ingestion in production environments.
- SQLite was chosen to keep the project lightweight and easy to run locally.
- FastAPI provides automatic OpenAPI documentation and async support.
- Streamlit offers rapid UI prototyping with minimal overhead.
- Provider Interface (Abstract Base Class) allows future integrations with providers like CoinMarketCap with minimal code changes.
- Retry & Backoff Mechanisms ensure resilience against API failures and rate limits.
- Rust Orchestration (Optional) is designed for future scaling to production workloads.
- In-memory Testing with pytest ensures fast, isolated tests with no real API dependency.
- Add support for multiple providers (CoinMarketCap, Binance, etc.).
- Add Redis/Memcached for faster caching layer.
- Rate-limit management using token bucket algorithms.
- API authentication with API keys or OAuth2.
- Dockerize the project for easier deployment.
- Advanced orchestrator for parallel, multi-asset, high-frequency ingestion.