Skip to content

Aurelien7877/Deep-Attention-BTC

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Bitcoin Volatility Engine Logo

Bitcoin Volatility AI Prediction

Python TensorFlow XGBoost Scikit-Learn

BTC-USD

Bitcoin Volatility Forecasting: A Comparative Study of Bi-LSTM with Attention and Ensemble Methods

1. Abstract

This research project implements a robust framework for predicting Bitcoin (BTC/USD) volatility using high-frequency 15-minute OHLCV data aggregated into 6-hour intervals. The objective is to classify future volatility states—defined by rolling quantiles—using a comparative approach involving Deep Learning (Bi-LSTM with Self-Attention), Gradient Boosting (XGBoost), and Support Vector Classification (SVC). The study emphasizes the prevention of data leakage through causal feature scaling and dynamic, rolling thresholding.

Reminder :

Beat 55% of accuracy on Bitcoin prediction is very hard. This threshold is very difficult to overpass due to noise of market.

Actual Best bitcoin volatility performances :

Category Model Technique Metric Classification
Hybrid SOTA CNN-LSTM + Attention PyTorch / TensorFlow Accuracy: 58-62% F1-Score: 0.60+
Transformer Temporal Fusion Transformer (TFT) (Google) PyTorch (PyTorch Forecasting) Accuracy: 57-60% AUC-ROC: 0.64
Ensemble ML XGBoost / LightGBM Scikit-learn / LightGBM Accuracy: 54-56% F1-Score: 0.55
Hybrid Stat SVR-GARCH R (rugarch) / Python (arch) Accuracy: 53-55% Precision: 0.54

2. Methodology

2.1 Data Preprocessing

The raw dataset consists of BTC/USD 15-minute candlesticks. To reduce stochastic noise and focus on mid-term trends, the data is aggregated into 6-hour bars using standard OHLCV resampling rules.

  • Interval: $ \Delta t = 6 $ Hours (aligned on UTC 00, 06, 12, 18).
  • Feature Engineering: Eight technical indicators are computed using a strictly causal approach:
    • Log Returns: $ r_t = \ln(P_t / P_{t-1}) $
    • Historical Volatility: $ \sigma_{t,n} = \sqrt{\frac{1}{n-1} \sum_{i=0}^{n-1} (r_{t-i} - \bar{r})^2} $ for $ n \in {6, 12} $.
    • Momentum: $ RSI_{14} $, $ MACD_{(12,26,9)} $.
    • Volume Dynamics: Z-score of volume relative to its 6-period moving average.

2.2 Causal Labeling Logic

Volatility classification is performed by evaluating the absolute log-return of the subsequent bar. To maintain a causal and adaptive threshold, a rolling quantile is utilized. Let $ R_{t+1} $ be the return at $ t+1 $. The target $ Y_t $ is defined as:

$$ Y_t = \begin{cases} 1 & \text{if } |R_{t+1}| > Q_{t}(k) \ 0 & \text{otherwise} \end{cases} $$

Where $ Q_{t}(k) $ represents the $ k $-th quantile ($ k=0.66 $) calculated over a look-back window of $ W=100 $ periods. This prevents look-ahead bias and ensures the model adapts to changing market regimes.


3. Model Architectures

3.1 Bi-directional LSTM with Attention

A deep learning architecture designed to capture temporal dependencies in both forward and backward directions.

  • Input Layer: Sliding windows of $ L=24 $ bars (6 days of history).
  • Recurrent Layer: Bi-LSTM with 64 units and $ tanh $ activation.
  • Attention Mechanism: A Self-Attention layer scores the hidden states $ h_i $ to generate a context vector $ c $: $$ e_i = a(h_i), \quad \alpha_i = \frac{\exp(e_i)}{\sum \exp(e_j)}, \quad c = \sum \alpha_i h_i $$
  • Regularization: Dropout (0.3) and Early Stopping based on Validation ROC-AUC.

3.2 Gradient Boosting (XGBoost)

A tabular approach utilizing flattened sequences. This model serves as a benchmark for non-linear relationship extraction between indicators.

  • Objective: binary:logistic
  • Scale Pos Weight: $ \frac{\sum y=0}{\sum y=1} $ to address class imbalance.

3.3 Support Vector Classifier (SVC)

A kernel-based method (RBF) used to identify optimal hyperplanes in a high-dimensional feature space.

  • Kernel: Radial Basis Function: $ K(x, x') = \exp(-\gamma |x - x'|^2) $.

4. Experimental Setup and Integrity

  1. Temporal Splitting: 80% Training, 20% Testing. Data is split chronologically to preserve time-series causality.
  2. Causal Scaling: StandardScaler parameters $ \mu_{train} $ and $ \sigma_{train} $ were computed strictly on the training set and applied to the test set: $$ z_{test} = \frac{x_{test} - \mu_{train}}{\sigma_{train}} $$
  3. Metrics: Primary evaluation is based on ROC-AUC to measure discriminative power independent of the decision threshold.

5. Performance Evaluation

The following results were obtained during the causal backtest:

Model Accuracy ROC-AUC
XGBoost 0.5857 0.5913
SVC 0.6576 0.5823
BiLSTM + Attention 0.6382 0.5777

5.1 Discussion

The empirical results indicate that XGBoost provides the highest discriminative power (AUC: 0.5913). The Bi-LSTM + Attention model exhibits significant overfitting patterns, as the validation AUC plateaus while the training AUC continues to rise. This suggests that for a 6h horizon using only OHLCV data, the signal-to-noise ratio is insufficient to justify high-complexity sequential models. The SVC demonstrates high accuracy through a "Majority Class Bias," failing to capture the underlying volatility variance effectively.


6. Conclusion

This study concludes that while Deep Learning models provide sophisticated architectures, tabular Gradient Boosting remains more robust for Bitcoin volatility classification under strict causal constraints. Future research should incorporate exogenous data such as Funding Rates or Liquidation Volume to enhance the predictive signal.

About

Deep Learning & Attention Mechanisms for Bitcoin Volatility.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors