Skip to content

README.md

Latest commit

 

History

History
97 lines (65 loc) · 2.31 KB

README.md

File metadata and controls

97 lines (65 loc) · 2.31 KB
Raw

VHgrad

An educational light deep learning framework built step by step from scratch. Inspired by Karpathy’s micrograd, but expanded into a full journey.

Project Roadmap

This repository is structured as a staged learning + engineering journey. Each stage builds on the previous one, gradually transforming a toy autograd engine into a portfolio-grade deep learning framework.

1. Micrograd (done)

  • Similar reproduction of Karpathy’s Micrograd with some small differences.

  • Implements a scalar-based autograd engine with:

    • Value class (forward/backward)
    • Core ops: +, -, *, /, ^, tanh
    • Computation graph visualization (Graphviz)
    • Minimal Neuron, Layer, MLP

4. VHgrad (not released)

Goal: Polish into a professional-grade mini-framework.

  • Core API:

    • Module, Parameter, Sequential
    • Linear, Dropout, activations (ReLU, Tanh, LeakyReLU)

  • Training utilities:

    • SGD, Adam optimizers
    • LR schedulers (step, cosine)
    • Gradient clipping, early stopping

  • Reliability:

    • Save/load checkpoints
    • Deterministic seeds

  • Metrics: accuracy, precision/recall, confusion matrix

  • Docs site (mkdocs)

  • Examples: XOR, Spiral, MNIST (≥95% acc), CIFAR-10 tiny baseline

  • Packaged on PyPI (pip install vhgrad)

  • CI tests (GitHub Actions)

Quickstart

pip install vhgrad  # after vhgrad stage is released
from vhgrad import MLP, SGD, mse

# simple 2-3-1 MLP
model = MLP(2, [3, 1])
opt = SGD(model.parameters(), lr=0.1)

for epoch in range(1000):
    ypred = [model(x) for x in xs]
    loss = mse(ys, ypred)
    opt.zero_grad()
    loss.backward()
    opt.step()

Benchmarks (targets)

  • XOR: ≥95% acc within 5k steps
  • Spiral: ≥90% acc
  • MNIST (1k subset): ≥90% acc ≤10 epochs
  • MNIST (10k subset): ≥95% acc ≤20 epochs
  • Speedup: ≥5× vs scalar baseline

Definition of Done per stage

  • micrograd: scalar engine reproduces tutorial
  • vhgrad: polished framework, docs, PyPI, ≥95% MNIST acc, CI tests

License

License: MIT — free to use, learn from, and extend.

Credits