README.md

Perception for Robotics and Autonomous Vehicles

A first-principles roadmap and technical reference for perception — 51 chapters across twelve parts, from linear algebra through sensors, classical vision, deep learning, 3D perception, fusion, SLAM, and the modern frontier.

It serves two uses: a structured path into the field, and a reference to return to while implementing or researching.

Download

Prebuilt PDFs, rebuilt on every merge to main:

• perception-guide.pdf — the handbook
• math-primer.pdf — the assumed mathematics, built from nothing
• notation.pdf — every symbol, and the ones that collide

The documents

File	Words	What it is
perception-guide.md	~83k	The handbook. Start here.
math-primer.md	~11k	Every prerequisite the guide assumes, derived from scratch. Start here if a chapter's math is unfamiliar.
notation.md	~4k	Symbol reference.

Structure

• Part 0 — Orientation: what perception is, the autonomy stack, the three hard problems
• Part I — Mathematical Foundations
• Part II — Sensors: From Physics to Data
• Part III — Calibration and Time Sync
• Part IV — Classical Computer Vision
• Part V — Deep Learning for Perception
• Part VI — 3D Perception
• Part VII — Estimation, Tracking, and Fusion
• Part VIII — SLAM, Localization, and Maps
• Part IX — The Modern Frontier
• Part X — Systems Engineering
• Part XI — Study Plan and Reference

Each chapter opens with a prereq block naming what it depends on, then follows the same arc: Intuition (the picture, 2–4 paragraphs) → Model (the mathematics) → In practice → Failure modes → Reference card → Practice. Cross-references use §X.Y and are checked mechanically.

Building the PDFs

Requires pandoc 3.7+ (for --syntax-highlighting) and XeLaTeX:

sudo apt-get install pandoc texlive-xetex texlive-latex-recommended \
  texlive-latex-extra fonts-texgyre fonts-texgyre-math fonts-dejavu-mono

tools/build.sh perception-guide.md build/perception-guide.pdf

With no arguments, tools/build.sh builds the guide next to its source. Typesetting options live in tools/header.tex.

Linting

python3 tools/check.py

Reports unresolved §X.Y references, symbols missing from notation.md, chapters missing part of the skeleton, over-long Intuition sections, and the word count per chapter. It is a linter for a document, not a parser — heuristics, and every finding cites file:line, so a false positive costs one glance. It always exits 0 and never gates a build.

Layout

perception-guide.md   math-primer.md   notation.md
tools/    build.sh, header.tex, check.py, baseline.md
build/    generated PDFs (gitignored)