Synapsa Commons › Track lessons
Lessons that belong to a subject track rather than to one industry programme. Each is a notebook you do, with the same exercises, checks and hints as every other lesson here, and each opens in Colab, Kaggle or Binder from the links beside it, or in Codespaces or your own Jupyter as the README describes; the worked solution and the full rubric come with enrolment on Synapsa.
New here? Take the beginner path, or pick a track.
Begin with some Python and take these in order; each needs only the lessons before it.
- The 8 GB track · ~40 minutes · Python · no lesson to do first
Build a profiler that measures what any piece of Python really costs in time and memory, rank four implementations of one task on the numbers, and reproduce the compute-tier check every lesson here has to pass.
- Byte-level BPE from scratch · ~60 minutes · Python · builds on The 8 GB track
Build byte-level byte pair encoding from scratch, and measure the token tax it imposes on five writing systems.
- The CPU slice of nanolm · ~75 minutes · Python · builds on The 8 GB track
Train a character-level language model on a CPU with numpy alone, its backward pass derived by hand and checked against finite differences.
- Retrieval from scratch · ~70 minutes · Python · builds on The 8 GB track
Build BM25, a dense retriever with no external model and hybrid fusion, scored by an evaluation harness whose intervals say when one retriever is really better.
- The agent loop from scratch · ~55 minutes · Python · builds on The 8 GB track
Build an agent harness with validated tools, budgets, retries and traces, and measure how per-step reliability compounds over long tasks.
Start from nothing, on a laptop. Measure what code costs on an 8 GB machine, keep a model registry honest through a model swap, and write one client for three backends.
- The 8 GB track · ~40 minutes · Python · no lesson to do first
Build a profiler that measures what any piece of Python really costs in time and memory, rank four implementations of one task on the numbers, and reproduce the compute-tier check every lesson here has to pass.
- A model registry that survives a swap · ~40 minutes · Python · builds on The 8 GB track
Build a role-based model registry, a retirement detector run against a dated snapshot of a provider's model pages, and a golden test that survives a model swap.
- One client, three backends · ~50 minutes · Python · builds on The 8 GB track
Build the normalising half of a provider-agnostic LLM client, which reads tool calls, reasoning and JSON out of whatever shape a backend returns, and a compatibility suite that shows what three stand-in backends keep, rename, drop or reject.
- Living on free tiers · ~90 minutes · Python · builds on The 8 GB track
Build a token-bucket limiter, jittered retries, a pacer and a capacity planner for a class on free API tiers, and measure how many learners finish, how many requests are wasted and what a quota cut does
Statistics by simulation: sampling distributions, bootstrap intervals, permutation tests and power, the habits every later lesson leans on.
- Statistics by simulation · ~90 minutes · Python · builds on The 8 GB track
Build a simulation toolkit for experiment analysis -- sampling distributions, bootstrap intervals, a permutation test and power calculator -- and audit how peeking and testing twenty metrics inflate false positives
- How models lie · ~50 minutes · Python · builds on The 8 GB track and Statistics by simulation
Build a leakage clinic — plant five leaks (target, preprocessing, duplicate, group and temporal) in a seeded customer book, measure how much each inflates cross-validated AUC against an oracle ceiling, and write the split, feature and pipeline checks that catch all five
Build automatic differentiation from scratch, a scalar engine and a tensor engine, and the gradient check that keeps both honest.
- Autodiff from scratch · ~75 minutes · Python · builds on The 8 GB track
Build a scalar and a tensor reverse-mode autodiff engine, plus a forward-mode one, and hold every gradient to a finite-difference check
- PyTorch core idioms · ~90 minutes · Python · builds on Autodiff from scratch
Write a PyTorch training loop with a checkpoint that survives a restart, and measure what each missing piece of state costs
Tokenisers from scratch, the BPE merge loop in C++, and a language model trained on a CPU with numpy alone.
- Byte-level BPE from scratch · ~60 minutes · Python · builds on The 8 GB track
Build byte-level byte pair encoding from scratch, and measure the token tax it imposes on five writing systems.
- The BPE merge loop, in C++ · ~60 minutes · C++ · builds on Byte-level BPE from scratch
Write the BPE merge loop in C++, and measure its speed-up over the Python trainer on one identical corpus.
- The CPU slice of nanolm · ~75 minutes · Python · builds on The 8 GB track
Train a character-level language model on a CPU with numpy alone, its backward pass derived by hand and checked against finite differences.
Post-training from scratch: GRPO in numpy on a tiny model, and the traps that make a reward mislead you, measured.
- GRPO from scratch on a tiny model · ~60 minutes · Python · builds on The 8 GB track
Build GRPO from scratch in numpy on a tiny model, and measure its traps: zero-variance groups, length and standard-deviation normalisation, group size and a reward hack.
- Chat templates and loss masking · ~90 minutes · Python · builds on Byte-level BPE from scratch and The CPU slice of nanolm
Plant five chat-template, loss-masking, end-token and packing bugs in a numpy fine-tuning pipeline, measure what each does to held-out behaviour, and write the five tests that catch them before training starts
- Build a cost and quality decision calculator that prices prompting, retrieval, fine-tuning, distillation and a mix over a year, and finds the volumes at which each overtakes the last · ~45 minutes · Python · builds on The 8 GB track
Build a cost and quality decision calculator that prices prompting, retrieval, fine-tuning, distillation and a mix over a year in three currencies and finds the volumes at which each overtakes the last
- LoRA from scratch · ~120 minutes · Python · builds on The CPU slice of nanolm and Chat templates and loss masking
Implement a LoRA linear layer with hand-derived gradients, adapt a numpy transformer to a new vocabulary at six ranks, and measure the rank at which the adapter stops helping against full fine-tuning
- Rubrics as rewards · ~60 minutes · Python · builds on GRPO from scratch on a tiny model
Build a rubric reward for open-ended answers — a weighted checklist score, a false-credit audit and an agreement test against a hidden truth — train a tiny policy against it, and measure how it learns to game the grader and what a penalty and a held-out monitor each repair
- DPO from the RLHF objective · ~2.5 hours · Python · builds on The CPU slice of nanolm and Chat templates and loss masking
Follow the derivation of the DPO loss from the KL-regularised reward objective, implement it on a numpy nanolm with a hand-checked gradient, and measure what it does to likelihood, margin, length and distance from the reference
- An environment for a tool-using agent · ~2 hours · Python · builds on GRPO from scratch on a tiny model and The agent loop from scratch
Build a tool environment with reset, step and reward, train a tabular agent on it with a group-relative update, and measure what happens when the reward pays for tool calls instead of outcomes
- Train a Bradley-Terry reward model and audit what it really learned · ~60 minutes · Python · builds on DPO from the RLHF objective, Judges that agree with humans and The challenger, built from scratch
Train a Bradley-Terry reward model and audit what it really learned — write its loss and gradient, measure accuracy against two sets of labels, isolate each shortcut with counterfactual pairs, run best-of-n against the learned reward, and put an interval on the drop
Serve a language model on modest hardware: a KV cache, paging and continuous batching on a CPU, and what each one buys.
- Serving a small model on a CPU · ~90 minutes · Python · builds on The 8 GB track and The CPU slice of nanolm
Serve a small transformer on a CPU the way an inference engine does, with a KV cache and static batching, and account for the operations, memory and latency each one buys.
- nanoserve, part two · ~2 hours · Python · builds on The CPU slice of nanolm and Serving a small model on a CPU
Build a paged KV cache and a continuous-batching scheduler for a small language model, prove that they change no token, and measure what they buy against a contiguous cache with static batching.
- GPTQ and AWQ from the maths · ~2.5 hours · Python · builds on The CPU slice of nanolm
Implement round-to-nearest, GPTQ and AWQ quantisation from the maths in numpy, quantise the weight matrices of a small language model trained inside the lesson to 4 and 3 bits, and measure the calibration-set size at which each method stops improving
- 4-bit number formats from scratch in C++ · ~150 minutes · C++ · builds on The BPE merge loop, in C++ and The CPU slice of nanolm
Write the INT4, MXFP4 and NVFP4 encoders and decoders in C++, and measure what each costs and what each loses on a small model's weights.
Build search and RAG you can measure: retrieval, chunking, the context budget, reranking and agent memory.
- Retrieval from scratch · ~70 minutes · Python · builds on The 8 GB track
Build BM25, a dense retriever with no external model and hybrid fusion, scored by an evaluation harness whose intervals say when one retriever is really better.
- Chunking and the context budget · ~55 minutes · Python · builds on The 8 GB track and Retrieval from scratch
Chunk documents and assemble context under a token budget, measured by whether the evidence an answer needs actually reaches the model.
- Reranking, and what it costs · ~75 minutes · Python · builds on The 8 GB track, Retrieval from scratch and Chunking and the context budget
Build a learned reranker, pointwise and pairwise, state its gain with its interval, measure what reranking depth costs, and meet the trap of tuning on test queries.
- Agent memory · ~45 minutes · Python · builds on The 8 GB track, Retrieval from scratch and The agent loop from scratch
Build memory for a long-running assistant: what to store, how recency, importance and relevance combine at retrieval, and why an updated fact must beat the stale one.
Build AI agents that hold up: the agent loop from scratch, MCP and A2A from the wire up, evaluation with intervals, and constrained decoding.
- The agent loop from scratch · ~55 minutes · Python · builds on The 8 GB track
Build an agent harness with validated tools, budgets, retries and traces, and measure how per-step reliability compounds over long tasks.
- The Model Context Protocol, from the wire up · ~50 minutes · Python · builds on The 8 GB track and The agent loop from scratch
Build a Model Context Protocol server and client from the specification, checked by a conformance harness built from the same specification.
- Evaluating agents · ~50 minutes · Python · builds on The 8 GB track and The agent loop from scratch
Evaluate agents with numbers that carry intervals: pass^k against pass@k, trajectory metrics, a failure taxonomy from traces, and a paired comparison of two configurations.
- The Agent2Agent (A2A) protocol, from the wire up · ~75 minutes · Python · builds on The 8 GB track, The agent loop from scratch and The Model Context Protocol, from the wire up
Build an Agent2Agent (A2A) Agent Card validator, task-lifecycle server and streaming client from the specification, checked by a conformance harness built from the same specification.
- Constrained decoding from scratch · ~45 minutes · Python · builds on Byte-level BPE from scratch and The agent loop from scratch
Build a JSON-schema constrained decoder, a token mask over a BPE vocabulary, and measure what the mask does to the output distribution.
- Build an Agent Client Protocol agent's session lifecycle and drive it from three toy editors · ~50 minutes · Python · builds on The Model Context Protocol, from the wire up
Build an Agent Client Protocol agent's session lifecycle over JSON-RPC and drive it from three toy editors
- Build a sandbox you would trust · ~90 minutes · Python · builds on The agent loop from scratch
Build a sandbox you would trust — a process-group supervisor, a symlink-resolving jail check and a deny-by-default audit policy, measured by twenty-one seeded escape attempts against every combination of the three layers
- Vet before you connect · ~75 minutes · Python · builds on The Model Context Protocol, from the wire up
Build an MCP tool-description scanner, a content-hash pin and a rug-pull detector, and measure the scanner's precision and recall on planted and paraphrased attacks
- Designing tools an agent can use · ~75 minutes · Python · builds on The agent loop from scratch and Evaluating agents
Specify the same tools badly and well, build a corrective validator, a spec linter and an episode loop, and measure with exact counts which design lever changes a scripted agent's results
- Give your agent its own identity · ~60 minutes · Python · builds on The Model Context Protocol, from the wire up and Vet before you connect
Build a token verifier, an on-behalf-of exchange and a replay-safe resource server, and count which attacks a static key and a delegated, scoped, expiring token each stop
Run agents like production software, with OpenTelemetry traces and a cost-per-task report rebuilt from the traces alone.
- Observability for agents · ~50 minutes · Python · builds on The agent loop from scratch and Evaluating agents
Instrument the agent loop with OpenTelemetry GenAI spans, and rebuild a cost-per-task report from the spans alone.
- From 100 traces to a failure taxonomy · ~60 minutes · Python · builds on Evaluating agents and Observability for agents
A failure taxonomy built from 100 agent traces by open coding, with two coders' kappa, five executable checks and a per-check CI gate
- Five workflow patterns in plain Python, each with an eval · ~75 minutes · Python · builds on The agent loop from scratch and Evaluating agents
Build the five workflow patterns (chaining, routing, voting, orchestration, evaluate-and-refine) over a scripted model, and measure each against a single call for accuracy, calls and failure modes
- Cost per successful task · ~75 minutes · Python · builds on Evaluating agents and Observability for agents
Build a cache-aware prompt layout, an exact token-level cost ledger and a latency-budgeted cost per successful task, and measure which layout changes cost the most
Measure models honestly: an EU AI Act evidence pack, evaluation across languages, and a mini arena whose ratings carry intervals.
- The conformity evidence pack · ~60 minutes · Python · no lesson to do first
Turn an AI system registry into a machine-checkable EU AI Act conformity evidence pack.
- Evaluating across languages · ~90 minutes · Python · builds on The 8 GB track and Byte-level BPE from scratch
Build the instruments that evaluate a system across six generated languages: token fertility, cost per correct answer, paired bootstrap intervals, aggregation traps, a translation-artefact detector and a power curve.
- A mini arena · ~50 minutes · Python · builds on The 8 GB track
Build a mini arena's ranking engine: Bradley-Terry ratings from pairwise votes, a position-bias fit, bootstrap intervals, and the number of votes before neighbouring systems stop swapping places.
- Metrics that punish correct answers · ~45 minutes · Python · builds on The 8 GB track
Build edit distance, BLEU and chrF from scratch, measure how often each ranks a rejected answer above an accepted one, and repair the metric with a normaliser and a rubric check
- How good is your retriever? Recall, MRR and nDCG from scratch, the pooling trap, and synthetic labels with a known error · ~50 minutes · Python · builds on Retrieval from scratch and Reranking, and what it costs
Build the retrieval-evaluation toolkit — recall, MRR and nDCG from scratch, a judgement pool and its bias, and a scripted labeller whose error you measure against the planted truth
- How many items does an eval need? Power, ranking uncertainty and a 100-item estimate of a 1,000-item score · ~120 minutes · Python · builds on Evaluating agents and Statistics by simulation
Build the statistics of an honest benchmark — the items needed to detect a gap, rank ranges over items and prompt formats, and a 100-item IRT estimate of a 1,000-item score
- Judges that agree with humans · ~60 minutes · Python · builds on How many items does an eval need? Power, ranking uncertainty and a 100-item estimate of a 1,000-item score, Evaluating agents and Review queues that survive contact with humans
Calibrate an LLM-as-judge against human labels — probe it for position and length bias, resolve its verdicts across a swap, and correct its win rate with an interval
- Threat-model an agent · ~45 minutes · Python · builds on The agent loop from scratch and The Model Context Protocol, from the wire up
Threat-model an agent as data — find where outside text travels, build a risk register mapped to the OWASP agentic list, generate the mitigation checklist, and run checks that fail until each mitigation is present
Do AI for science on real data: find the first gravitational wave in the raw LIGO data, and model rainfall to runoff honestly.
- Find GW150914 in the raw LIGO data · ~50 minutes · Python · builds on The 8 GB track
Find the first gravitational wave ever detected in the raw LIGO data, on a laptop: noise spectra, whitening, a chirp template and matched filtering, from scratch.
- Rainfall to runoff, honestly · ~60 minutes · Python · builds on The 8 GB track
Build a four-parameter rainfall-runoff model, a calibrator and a lagged-rain baseline, and measure the hold-out, split and flow-band traps on two real Australian catchments.
Synapsa Commons · © 2026 RealAI · licensed under CC BY-NC-SA 4.0. You may use this material to learn and to teach, and copy, fork, share and adapt it. You must credit "Synapsa Commons by RealAI" with a link to https://github.com/TarrySingh/Artificial-Intelligence-Deep-Learning-Machine-Learning-Tutorials, say what you changed, and share adaptations under the same licence. You may not use it, or anything adapted from it, for commercial advantage or payment; for a commercial licence, contact RealAI. The full summary is in NOTICE; the licence's legal code governs.