Multi-Scale Hypergraph Meets LLMs: Aligning Large Language Models for Time Series Analysis

✨ This repository provides the official implementation of MSH-LLM that aligns large language models for time series analysis.

1 The framework of MSH-LLM

MSH-LLM focuses on reprogramming an embedding-visible large language model, e.g., LLaMA and GPT-2, for general time series analysis, while accounting for the multi-scale structures of natural language and time series. MSH-LLM consists four main parts: Multi-Scale Extraction (ME) Module, Hyperedging Mechanism, Cross-Modality Alignment (CMA) Module, and Mixture of Prompts (MoP) Mechanism. The framework of MSH-LLM is shown as follows:

2 Prerequisites

• Python 3.8.5
• PyTorch 1.13.1
• math, sklearn, numpy, torch_geometric

To install all dependencies:

pip install -r requirements.txt

3 Datasets && Description

You can access the well pre-processed datasets from [Google Drive] [Tsinghua Cloud], then put the downloaded datasets under the folder ./datasets.

4 Running

4.1 Install all dependencies listed in prerequisites

4.2 Download the dataset

4.3 Training

🚀 We provide experiment scripts for demonstration purpose under the folder ./scripts.

# the default large language model is LLaMA-7B

# long-term forecasting
bash ./scripts/ETTh1.sh

# short-term forecasting
bash ./scripts/M4.sh

# classification
bash ./scripts/EthanolConcentration.sh

# few-shot learning
bash ./scripts/ETTh1.sh

# zero-shot learning
bash ./scripts/m3_m4.sh
bash ./scripts/m4_m3.sh

5 Main results

The proposed method outperforms other models on most tasks, including long-term forecasting, short-term forecasting, classification, few-shot learning, and zero-shot learning.

5.1 Long-term forecasting

5.2 Short-term forecasting

5.3 Classification

5.4 Few-shot learning

5.4.1 Few-shot learning results under 5% training data.

5.4.2 Few-shot learning results under 10% training data.

5.5 Zero-shot learning

The final code will be released soon!!!