Skip to content

README.md

Latest commit

 

History

History
139 lines (96 loc) · 4.92 KB

README.md

File metadata and controls

139 lines (96 loc) · 4.92 KB
Raw

Fine-Tuning Qwen 2.5 Coder 7B with QLoRA on Custom Enterprise Data

A practical project demonstrating how to fine-tune a large language model (LLM) on domain-specific enterprise data using QLoRA (4-bit quantization + LoRA adapters), optimized to run on consumer-grade GPUs with as little as 8 GB of VRAM.

Overview

This project takes the Qwen 2.5 Coder 7B Instruct model and fine-tunes it on a custom dataset of enterprise coding standards, internal API patterns, and company-specific best practices. After training, the model learns to generate code that follows proprietary conventions — such as using internal logging libraries, vault-based secret management, and standardized API response formats.

Key Highlights

  • Model: Qwen 2.5 Coder 7B Instruct (4-bit quantized)
  • Method: QLoRA (Quantized Low-Rank Adaptation) via Unsloth
  • Hardware: Runs on a single NVIDIA RTX 4060 (8 GB VRAM)
  • Dataset: Custom JSONL with instruction/input/output triplets
  • Framework: Hugging Face transformers, trl (SFTTrainer), peft

Project Structure

├── train.py                  # Fine-tuning script (QLoRA + SFTTrainer)
├── test.py                   # Inference comparison: base model vs. fine-tuned
├── dati.jsonl                # Custom enterprise training dataset
├── qwen-modello-aziendale/   # Saved LoRA adapters (after training)
├── outputs/                  # Training checkpoints and logs
└── README.md

How It Works

1. Training (train.py)

The training script:

  1. Loads Qwen 2.5 Coder 7B in 4-bit precision to fit in limited VRAM
  2. Attaches LoRA adapters (rank 16) to all attention and MLP projection layers
  3. Formats training examples using an Alpaca-style instruction template
  4. Trains with SFTTrainer using memory-efficient settings:
    • Batch size 2 with gradient accumulation (effective batch size = 8)
    • AdamW 8-bit optimizer
    • Gradient checkpointing via Unsloth
  5. Saves the trained LoRA adapters locally

2. Evaluation (test.py)

The evaluation script compares the base model against the fine-tuned model on the same prompts. It:

  1. Loads the base Qwen model
  2. Runs inference on sample enterprise questions
  3. Loads the trained LoRA adapters on top of the base model
  4. Runs the same questions again to show the difference

3. Dataset Format

The training data follows the Alpaca instruction format in JSONL:

{
  "instruction": "Initialize the enterprise logger to record a critical error in the payments module.",
  "input": "",
  "output": "from core_utils.logging import get_nexus_logger\n\nlogger = get_nexus_logger(__name__)\nlogger.critical('Critical error in payments module', extra={'alert_level': 'P1', 'slack_notify': True})"
}

Each entry teaches the model a specific enterprise pattern — internal libraries, coding standards, infrastructure conventions, etc.

Getting Started

Prerequisites

  • Python 3.10+
  • NVIDIA GPU with ≥ 8 GB VRAM (CUDA 12.1+)
  • Linux (recommended)

Installation

# Clone the repository
git clone https://github.com/<your-username>/train-llm.git
cd train-llm

# Create a virtual environment
python -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install unsloth transformers trl datasets peft bitsandbytes accelerate

Run Training

python train.py

Training runs for 60 steps by default (~5–10 minutes on an RTX 4060). Adjust max_steps in the script for longer training.

Run Evaluation

python test.py

This will print a side-by-side comparison of the base model vs. fine-tuned model responses.

Configuration

Key training parameters (in train.py):

Parameter Value Description
max_seq_length 2048 Maximum token length per example
r (LoRA rank) 16 Rank of the low-rank adaptation matrices
lora_alpha 16 Scaling factor for LoRA
per_device_train_batch_size 2 Batch size per GPU
gradient_accumulation_steps 4 Virtual batch size multiplier
max_steps 60 Total training steps
learning_rate 2e-4 Learning rate
optim adamw_8bit Memory-efficient 8-bit optimizer

Technologies

  • Unsloth — 2x faster fine-tuning with optimized kernels
  • QLoRA — 4-bit quantization + Low-Rank Adaptation
  • Hugging Face TRL — Supervised Fine-Tuning Trainer
  • PEFT — Parameter-Efficient Fine-Tuning
  • Qwen 2.5 Coder — State-of-the-art coding LLM

License

This project is released under the MIT License.

Author

Built as a portfolio project to demonstrate practical LLM fine-tuning on consumer hardware.

Paolo Chignoli

17 March 2026