TinyTorch v0.1.10 — Publication-Grade Lab Guide

TinyTorch v0.1.10

Largest Lab Guide upgrade since the series began, plus substantive framework work: new Tensor API surface (view, masked_fill, ndim, numel, contiguous), no_grad() context manager, Python 3.10+ baseline, 28 security alerts resolved, seven batches of module-audit fixes, reproducibility via seeded default_rng(7), and 847/847 tests green.

✨ New Features

Lab Guide PDF (the flagship of this release)

• Glossary back matter: 90 alphabetical entries covering tensor/memory, autograd, training systems, architecture, optimization, and ML basics, with module cross-references (see glossary.qmd).
• Module opener hooks: every module starts with a 2–3 sentence systems-first framing paragraph leading with memory, bandwidth, arithmetic intensity, HBM, or roofline implications before the ML story.
• Code listing captions + List of Listings: ~60 substantive code blocks carry Listing N.M — Description captions; populates a new List of Listings in the front matter.
• List of Figures + List of Tables in front matter (151 newly-captioned tables).
• Running headers with chapter + section: verso Chapter N · Title, recto N.M · Section, wordmark centered. H&P / CLRS convention.
• Personal instructor note signed callout at the end of the conclusion.
• Single-source build: make install-deps && make pdf works identically locally and in CI; deps live in pdf/apt-requirements.txt and pdf/tex-requirements.txt.

TinyTorch Framework

• Tensor API expansion: added view(), masked_fill(), Tensor stacking, ndim, numel(), and contiguous() (closes #1298; PR #1392 by @Shashank-Tripathi-07). PyTorch-compat test coverage added for all new methods.
• no_grad() context manager: autograd now supports with no_grad(): inference blocks plus graph cleanup between passes.
• Tito CLI: module path --about renamed to module path --guide and repointed at the Quarto chapter (consistent with the Lab Guide becoming the reference).

🐛 Framework Bug Fixes

Autograd and training

• Tanh wired into enable_autograd() — was silently producing zero gradients.
• Trainer.evaluate accuracy for regression models corrected (was misreporting).
• GELU gradient mismatch + float32 test precision fixed.
• Trainer init: guard requires_grad loop against non-Tensor params; ensure model params have requires_grad=True (2 commits).
• M06 _reduce_broadcast_grad aligned with module conventions.

Module-level

• Quantization (M15): constant tensor quantized to all-zeros, losing the original value (#1444).
• MaxPool2d: API mismatch in milestone 04 CIFAR script fixed (#1278).
• Export paths: corrected for modules 09 and 13.
• Token constants: refactor cleanup from PR #1279 (#1256).
• M19 benchmarking: MLPerf trademark attribution added, educational-purposes disclaimer, table alignment fix, addresses feedback from #1196.
• M02 activations: improved activation graph visualization.

Module audit fixes

• Seven batches of audit fixes landed: batches 1–7 covering critical fixes, medium/low documentation and accuracy, and test-infrastructure cleanup. Final state: 847/847 tests passing.

🔬 Tests

• Finite-difference gradient correctness tests added for Module 06.
• Module 08 training infrastructure coverage tests added.
• Gradient correctness suite restored with per-op tolerances (#1342).
• Module 10 tokenization tests now use real Tensor params instead of raw numpy arrays.
• Module 08 scheduler lr assertion corrected (epoch 0, not 1).

🔧 Engineering

• Reproducibility: migrated from legacy np.random to default_rng(7) — seeded, per-call RNG across all modules.
• Python baseline: minimum version bumped to 3.10. Milestone 05 docs updated.
• src/*/ABOUT.md cleanup: 20 stale duplicates deleted (−20,876 lines); the single-source ABOUT.md now lives in the correct companion-doc location.
• Security: all 28 GitHub code-scanning alerts resolved.
• Tito: register --tinytorch pytest flag in conftest; fix UnicodeDecodeError on Windows in tito module complete (#1184); null-synced_modules guard in submission progress response.

📖 Content Improvements

• Systems-first narrative: every module hook leads with the systems angle (memory, bandwidth, compute, hardware utilization) before pivoting to ML theory.
• Check Your Understanding callouts: converted from prose sections to callout-tip format with 3–5 technical-specific checkboxes per module.
• Key Takeaways: 3–4 bullet recap plus next-module hook at the end of every module chapter.
• Systems Implication callouts unified to callout-note across all 21 instances; answers converted to callout-tip collapse="true" across 101 Q&A pairs.
• Cross-reference audit: 216 orphan table/figure/listing labels got natural prose references (87% coverage).
• Further Reading hyperlinks: 20 external URLs verified and linked (Jay Alammar, arXiv papers, Karpathy's blog, Jurafsky & Martin SLP3).
• Broadcasting pitfall now taught alongside the broadcasting feature (M01 tensor).
• log_softmax implementation cleaned up with clearer variable names and reuse.
• Type hints: added to M03 layers, M04 losses, M05 dataloader (#1167).
• Big-picture diagram: redesigned as a 4-layer stack (Capstone → Optimization → Architecture → Foundation) in neutral palette with MIT-red Capstone accent.

🎨 Design and Typography

• Book-style typography: linestretch: 1.1, first-line indent (parindent: 1.2em), tight parskip. Stripe-Press / Swift-Book density.
• Thin single orange header rule (previously double rule).
• 23 module-diagram SVGs aligned to the book palette via a Gemini multimodal audit pass.
• Arraystretch 1.2 + enumitem for table and list breathing room.
• Text-only callout titles (no stripped emojis; class semantics drive visual distinction).

🐛 Lab Guide Bug Fixes

• Tokenization module: restored missing ```{python} fence that caused Pandoc to render Python variable-definition comments as chapter headings.
• Single-PDF guarantee: Makefile self-heals when Quarto's post-render cleanup strands the artifact at pdf/ instead of pdf/_build/. Build-end banner prints the canonical path.
• 3 broken URLs fixed: GPT-2 cloudfront → OpenAI CDN, PyTorch .md → .html, mlu-explain /relu/ → /neural-networks/.
• Duplicate trailing ## Get Started removed from 4 modules (copy-paste artifact).
• Orphan big-picture-module-flow.svg removed from images/diagrams/ (canonical lives at images/svg/).

🔧 CI / Infrastructure

• Single-source deps: make install-deps reads pdf/apt-requirements.txt and pdf/tex-requirements.txt — same command works locally and in CI.
• tinytorch-build-pdfs.yml and tinytorch-update-pdfs.yml simplified to make install-deps && make pdf (no inline tlmgr package list).
• make clean extended to remove stale *_files/ directories at the Quarto project root.

📚 Documentation

• Early Explorer callout removed from getting-started.qmd — no longer appropriate now that the Lab Guide is shipping.
• Callout convention documented in the preamble: six semantic callout types keyed off Quarto's five shipped classes plus title conventions.
• README tables converted from markdown to HTML format for consistent rendering across GitHub and the Lab Guide.

---

👥 Contributors

Thanks to everyone who contributed to this release:

• @profvjreddi — editorial direction and polish across all 20 modules
• @hzeljko — sustained code, diagram, and infrastructure contributions
• @Shashank-Tripathi-07 — Tensor PyTorch-compat API (ndim, numel, view, contiguous, masked_fill; PR #1392; first-time contributor!)
• @farhan523 — ongoing documentation and module improvements
• @adityamulik — null synced_modules fix in tito submission progress
• @harishb00 — type hints across M03/M04/M05

🆕 New Contributors

• @Shashank-Tripathi-07 made their first code contribution (PR #1392)

---

Full Changelog: tinytorch-v0.1.9...tinytorch-v0.1.10

Website: https://mlsysbook.ai/tinytorch/

PDF: https://mlsysbook.ai/tinytorch/assets/downloads/TinyTorch-Guide.pdf

Glossary (new): https://mlsysbook.ai/tinytorch/glossary.html