LLM & VLM Fine-Tuning with LoRA and QLoRA

Self-contained utilities for preparing NLP/VLM datasets, fine-tuning with LoRA or QLoRA, and running everything from a Typer CLI. Code lives in scripts/, datasets are stored under data/, and models/adapters are saved under models/.

Project Layout

• scripts/ — all Python modules (cli.py, config, data prep, training, etc.). Run via python -m scripts.cli ... from the repo root.
• data/ — prepared/tokenized datasets (data/processed/<task> by default).
• models/ — model artifacts. Set caches here to keep base downloads local (see .env.example); fine-tuned adapters default to models/lora and models/qlora.
• requirements.txt — install dependencies into your venv.
• .env.example — optional env vars for caching/GPU selection.

Setup

python -m venv .env
source .env/bin/activate
pip install -r requirements.txt

(Optional) Create .env from .env.example to pin caches:

HF_HOME=./models/hf_cache
HF_DATASETS_CACHE=./data/hf_datasets
CUDA_VISIBLE_DEVICES=0

Quickstart

1. Prepare data (defaults store processed data in data/processed/<task>)

python -m scripts.cli prepare-data \
  --task question_answering \
  --dataset-name squad \
  --model-type llm \
  --max-length 384

For local files: --data-files '{"train": "train.json", "validation": "valid.json"}'

2. LoRA fine-tune (adapters saved to models/lora unless overridden)

python -m scripts.cli finetune-lora \
  --task question_answering \
  --model-type llm \
  --train-data-dir data/processed/question_answering \
  --output-dir models/lora/qa-finetune \
  --num-epochs 3 \
  --batch-size 4 \
  --learning-rate 2e-4 \
  --gradient-checkpointing  # optional: saves memory, sets use_cache=False and uses non-reentrant checkpointing

3. QLoRA fine-tune (4-bit base, adapters saved to models/qlora)

python -m scripts.cli finetune-qlora \
  --task question_answering \
  --model-type llm \
  --train-data-dir data/processed/question_answering \
  --output-dir models/qlora/qa-finetune \
  --num-epochs 3 \
  --batch-size 4 \
  --learning-rate 2e-4 \
  --gradient-checkpointing  # optional

VLM Notes

• Default VLM: Salesforce/blip2-flan-t5-xl.
• Expect image column image (PIL-compatible) and text prompt via text/question/prompt. Labels via answer/caption/output.
• Use --model-type vlm; QLoRA loads 4-bit automatically. Set caches to models/hf_cache in .env to keep downloads inside models/.

Implementation Highlights

• Models: Seq2Seq LLMs via AutoModelForSeq2SeqLM; VLMs via AutoModelForVision2Seq + AutoProcessor; optional 4-bit with BitsAndBytesConfig.
• Data Prep: Task-specific tokenization; auto train/validation split if missing; saved to data/processed/<task>.
• Training: Seq2SeqTrainer + DataCollatorForSeq2Seq; LoRA via get_peft_model; QLoRA uses prepare_model_for_kbit_training first. Adapters/tokenizers saved in models/ under chosen output dir.
• Reproducibility: set_seed seeds Python/NumPy/Torch; CLI loads .env automatically.

Tips

• For CPU-only use, avoid --model-type vlm or --finetune-qlora unless you have the RAM; bitsandbytes needs GPU.
• If CUDA OOM, lower --batch-size, raise gradient accumulation, or shorten --max-length during prepare-data.
• Keep datasets and caches tidy: data/ for datasets, models/ for all model weights/adapters and HF cache.