Self-Alignment with Instruction Backtranslation

Replication of Li et al. (2023) at smaller scale.
Fine-tuned a LLaMA-2-7B backward model to generate synthetic instructions, then used LLM-judge curation to build a high-quality instruction-tuning dataset — without manual labeling.

Key Results

Metric	Value
Manual labeling reduction	95%
Synthetic instructions generated	150
High-quality samples retained (LLM-judge ≥ 4/5)	41 (27.3% retention)
LoRA trainable parameters	0.0622% of total
LoRA config	r=8, α=16
Hardware	NVIDIA A100-40GB

Pipeline

3,000 instruction–response pairs (OpenAssistant-Guanaco + LIMA)
        ↓
Fine-tune LLaMA-2-7B as backward model: p(instruction | response)
        ↓
Generate 150 synthetic instructions from LIMA responses
        ↓
LLM-judge scoring (≥ 4/5 threshold) → 41 high-quality pairs
        ↓
LoRA instruction tuning on curated dataset

HuggingFace Artifacts

Artifact	Link
Backward model	llama2-7b-backward-instruction
Curated dataset	instruction-backtranslation-curated
Instruction-tuned model	llama2-7b-instruction-tuned

Stack

PEFT LoRA bitsandbytes HuggingFace Transformers LLaMA-2-7B OpenAssistant-Guanaco LIMA A100-40GB

Reference

Li et al. (2023) — Self-Alignment with Instruction Backtranslation