BTC RL Trader

Production-grade reinforcement learning trend-following trading platform for BTC/USD.

Overview

This system implements a sophisticated Double DQN agent with dueling architecture and prioritized experience replay to learn optimal trading strategies from historical BTC/USD data. The platform is designed for continuous learning and deployment with comprehensive risk management.

Key Features

• Double DQN with Dueling Architecture: Reduces overestimation bias and improves value estimation
• Prioritized Experience Replay: Focuses learning on important transitions
• Walk-Forward Training: Robust out-of-sample validation
• Risk-Aware Rewards: Optimizes for Sharpe ratio and drawdown control
• Production API: FastAPI service with Prometheus metrics
• Real-Time Data: OANDA v20 API integration with retry logic
• Comprehensive Backtesting: Full performance analytics and visualization

Architecture

graph TB
    A[OANDA API] -->|Market Data| B[Data Pipeline]
    B --> C[Feature Engineering]
    C --> D[RL Environment]
    D --> E[DQN Agent]
    E -->|Actions| D
    D -->|Rewards/States| E
    E --> F[Model Checkpoints]
    F --> G[FastAPI Service]
    G --> H[Trading Decisions]
    G --> I[Prometheus Metrics]
    I --> J[Grafana Dashboard]

Loading

Performance Targets

Metric	Target	Hard Fail
Sharpe Ratio	≥ 2.0	< 1.5
Max Drawdown	≤ 5%	> 10%
Hit Rate	≥ 55%	< 45%
Avg Trade P/L	> 0	≤ 0

Quick Start

Prerequisites

• Python 3.11+
• CUDA 12.3+ (for GPU support)
• Docker & Docker Compose
• OANDA API credentials

Installation

# Clone repository
git clone https://github.com/yourusername/btc-rl-trader.git
cd btc-rl-trader

# Install dependencies
poetry install

# Set environment variables
export OANDA_ACCOUNT_ID="your_account_id"
export OANDA_API_KEY="your_api_key"

Training

# Run walk-forward training
python training_pipeline.py

# Monitor training
tensorboard --logdir=logs

Deployment

# Build and run with Docker
docker-compose up -d

# Check health
curl http://localhost:8000/health

# Get prediction
curl -X POST http://localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"use_latest": true}'

Project Structure

btc-rl-trader/
├── data/               # Data fetching and cleaning
├── features/           # Technical indicators and features
├── env/                # Gym trading environment
├── agents/             # DQN implementation
├── backtest/           # Backtesting engine
├── serve/              # FastAPI deployment
├── tests/              # Unit tests
├── notebooks/          # Jupyter demos
└── config/             # Configuration files

Configuration

Edit config/config.yaml to adjust:

• Risk parameters (stop loss, position sizing)
• Model architecture (layers, learning rate)
• Training settings (episodes, batch size)
• API endpoints and credentials

API Endpoints

• GET /health - Service health check
• POST /predict - Get trading prediction
• POST /trade - Execute trade (dry run by default)
• GET /metrics - Performance metrics
• GET /metrics/prometheus - Prometheus format

Testing

# Run all tests
poetry run pytest

# With coverage
poetry run pytest --cov=. --cov-report=html

# Type checking
poetry run mypy .

# Linting
poetry run flake8 .
poetry run black --check .

Monitoring

Access monitoring dashboards:

• Grafana: http://localhost:3000 (admin/admin)
• Prometheus: http://localhost:9091

Risk Management

The system implements multiple risk controls:

• Position Sizing: Kelly criterion with volatility adjustment
• Stop Loss: Automatic 2% stop loss per position
• Drawdown Limits: Episode termination at 20% drawdown
• Leverage Limits: Maximum 95% capital utilization
• Funding Costs: Realistic funding rate simulation

Performance Analysis

Generate comprehensive performance reports:

from backtest import PerformanceVisualizer

visualizer = PerformanceVisualizer()
visualizer.create_performance_report(
    portfolio_values,
    trades,
    positions,
    metrics,
    save_path="reports/performance.pdf"
)

Contributing

1. Fork the repository
2. Create feature branch (git checkout -b feature/amazing-feature)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing-feature)
5. Open Pull Request

License

This project is licensed under the MIT License - see LICENSE file for details.

Disclaimer

IMPORTANT: This software is for educational and research purposes only. Do not trade real money without extensive testing and validation. Past performance does not guarantee future results. Trading cryptocurrencies involves substantial risk of loss.

Acknowledgments

• OpenAI Gym for the RL framework
• PyTorch team for the deep learning library
• OANDA for market data API
• All contributors and researchers in algorithmic trading