PersonalVoice AI

Fine-tune an open-source LLM on your own writing so the model produces text in your voice — your vocabulary, sentence rhythm, formality, and habits — not the generic LLM house style.

The project implements the full pipeline from raw Google Drive / Gmail exports to a trained QLoRA adapter and an inference UI, with evaluation harnesses for automatic metrics, AI-detector checks, and blind human A/B testing.

What's in this repo

Stage	Module	CLI entry point
Google Drive ingest	src/ingestion/drive_downloader.py	python -m src.ingestion.drive_downloader
File conversion	src/ingestion/file_converter.py	python -m src.ingestion.file_converter
Gmail mbox → text	scripts/convert_mbox_to_text.py	python scripts/convert_mbox_to_text.py
Text cleaning + PII	src/preprocessing/text_cleaner.py	python -m src.preprocessing.text_cleaner
Dataset construction	src/preprocessing/dataset_creator.py	python -m src.preprocessing.dataset_creator
QLoRA fine-tuning	src/training/finetune_qlora.py	python -m src.training.finetune_qlora
Hyperparameter sweep	src/training/sweep.py	python -m src.training.sweep
Ablation studies	src/training/ablations.py	python -m src.training.ablations
Generation (CLI)	src/inference/generator.py	python -m src.inference.generator
Generation (API)	src/inference/server.py	python -m src.inference.server
Generation (UI)	ui/app.py	streamlit run ui/app.py
Style metrics	src/evaluation/metrics.py	python -m src.evaluation.metrics
AI-detector testing	src/evaluation/detector_test.py	python -m src.evaluation.detector_test
Blind A/B harness	src/evaluation/ab_test.py	python -m src.evaluation.ab_test
Colab training notebook	notebooks/train_colab.ipynb	Open in Colab, runtime → T4 GPU

Quickstart — train on your writing

1. Install. ./setup.sh (creates venv, installs requirements, downloads spaCy en_core_web_sm).
2. Configure. cp configs/config.example.yaml configs/config.yaml and fill in Google Drive folder IDs. If you'd rather skip Drive, drop your .txt/.docx/.pdf files straight into data/raw/.
3. Ingest + clean + build dataset. ./run_pipeline.sh (interactive; runs each stage with your confirmation) produces data/datasets/train.jsonl and test.jsonl in Alpaca format.
4. Train. Either:
   • Free GPU (Colab, recommended): open notebooks/train_colab.ipynb, upload your JSONL files, and run all cells. The notebook trains Llama-3.2-3B-Instruct with QLoRA on a T4 in ~30–60 minutes and hands you back personalvoice_adapter.zip. Unzip into models/adapters/.
   • Local GPU / Azure ML: python -m src.training.finetune_qlora --config configs/config.yaml, or python scripts/azure_submit.py --submit for managed training.
5. Generate. streamlit run ui/app.py, or CLI: python -m src.inference.generator --prompt "Write an email to my professor asking for an extension."

Evaluating your model

Hardware note. A fine-tuned 3B-parameter model on CPU takes 5–30 minutes per 48-token generation, which makes CPU-based evaluation impractical. Running 30 test generations takes hours on CPU vs. ~5 minutes on a Colab T4. Use Colab/GPU for the generation step; the scoring step (metrics, A/B) runs fine anywhere.

A. Run it all on Colab GPU (recommended)

Open notebooks/eval_colab.ipynb, upload your adapter + test.jsonl to MyDrive/personalvoice/, runtime → T4 GPU, run all cells. Produces generated.jsonl, reference.jsonl, and results.json with BERTScore, ROUGE, style similarity, and perplexity.

B. Run the full pipeline locally (GPU required)

# End-to-end: loads adapter, generates, then scores. ~5 min on an A100/T4.
python scripts/run_evaluation.py --n 30 --output-dir eval/

C. Metric-pipeline smoke test (CPU OK)

Validates BERTScore, ROUGE, and style-similarity produce monotonic results across identity/paraphrase/unrelated text pairs — confirms the scoring code works end-to-end without needing a GPU-driven generation:

python scripts/smoke_test_metrics.py
# Expected: identity_sim=1.00 > paraphrase_sim~0.77 > unrelated_sim~0.05 → PASS

D. AI-detector testing (requires API keys)

python -m src.evaluation.detector_test --input generated.jsonl \
    --gptzero-key $GPTZERO_KEY --originality-key $ORIGINALITY_KEY

E. Blind A/B test harness (no GPU needed)

python -m src.evaluation.ab_test build  --prompts prompts.jsonl \
    --generated generated.jsonl --reference reference.jsonl --output eval/pairs.json
python -m src.evaluation.ab_test rate   --pairs eval/pairs.json \
    --output eval/ratings.json --rater-id alice
python -m src.evaluation.ab_test score  --truth eval/pairs_truth.json \
    --ratings eval/ratings.json

Hyperparameter sweeps and ablations

The proposal's Aim 3 calls for sweeps over learning rate, LoRA rank, and epochs, and ablations across data size, genre mix, instruction diversity, and adapter placement. Both are implemented:

python -m src.training.sweep --dry-run                # print the 48-cell grid
python -m src.training.sweep --output-root runs/sweep # actually train each cell

python -m src.training.ablations --dry-run            # print all ablation cells
python -m src.training.ablations --only emails_only articles_only mixed

Each run writes its own adapter directory, training losses, and eval losses to the results JSON so runs can be compared after the fact.

Architecture

Google Drive / Gmail Takeout
         │
         ▼
   [ingestion]  ─── raw .txt / .docx / .pdf / .mbox ───►  data/raw/
         │
         ▼
  [preprocessing]  ─── cleaning + PII + TF-IDF dedup ───►  data/processed/
         │
         ▼
  [dataset creator]  ─── Alpaca-format instruction pairs ──►  data/datasets/{train,test}.jsonl
         │
         ▼
   [training]  ─── QLoRA (4-bit NF4 + LoRA adapters) ───►  models/adapters/
         │
         ▼
   [inference]  ─── generator + optional RAG + humanizer ──►  CLI / FastAPI / Streamlit
         │
         ▼
  [evaluation]  ─── metrics, detector tests, blind A/B ───►  evaluation results

Privacy

• Training happens in your own environment (local, Colab, or Azure under your account).
• The preprocessing.anonymize_pii flag replaces persons, orgs, locations, dates, emails, and phone numbers with bracketed placeholders via spaCy NER before anything is written to the training dataset.
• No telemetry. The project makes zero outbound calls unless you opt in to a detector API (GPTZero / Originality.ai) during evaluation.
• Train only on text you own or have rights to use. Voice mimicry can be abused for impersonation — disclose AI-assisted writing when the context calls for it.

Testing

pytest tests/

Unit tests cover text cleaning, dataset construction, genre/intent heuristics, the humanizer, sweep configuration, and the A/B scoring math. Tests run without a GPU.

License

MIT — see LICENSE. Base-model weights are covered by their own licenses (Llama 3.2 Community License, etc.); redistribute adapters accordingly.