Every module is a lab a student does: a notebook they run and fill in, or a C exercise they
compile. Every module declares a compute tier and is held to it by tools/execute.py.
Environment constraint, stated once. The whole programme runs on Python 3.12 with numpy,
matplotlib, tokenizers and pytest, plus clang and make. There is no PDF library, no OCR
engine and no model API on any required path, so every corpus is generated deterministically
inside its lesson from a fixed seed. The labs below are designed within that, not around it.
11 of 11 modules built. A module is built when its lesson directory exists and has passed all 14 gates in QUALITY.md and an independent review; everything else is specified here and has no directory. Seconds and peak memory are measured by tools/execute.py; rubric size is read from each lesson's tests/test_lesson.py. Regenerate with python tools/status.py.
- Field extraction you can measure · built · ~55 minutes · Python
measured 0.2 s · 30 MiB · rubric 90 points in 24 cases - Ingestion and layout without a PDF library · built · ~60 minutes · Python
measured 0.2 s · 31 MiB · rubric 94 points in 23 cases - From rules to a tagger · built · ~60 minutes · Python
measured 0.8 s · 35 MiB · rubric 90 points in 20 cases - Table extraction: structure and content are two different scores · built · ~70 minutes · Python
measured 7.2 s · 30 MiB · rubric 94 points in 21 cases - Contract clause classification and calibrated abstention · built · ~80 minutes · Python
measured 2.1 s · 196 MiB · rubric 100 points in 29 cases - Human-in-the-loop review queues that survive contact with humans · built · ~70 minutes · Python
measured 2.9 s · 31 MiB · rubric 90 points in 26 cases - Error analysis and slice discovery · built · ~75 minutes · Python
measured 10.4 s · 211 MiB · rubric 92 points in 26 cases - Drift monitoring for a live document pipeline · built · ~70 minutes · Python
measured 8.8 s · 128 MiB · rubric 96 points in 27 cases - Cost per document, and the routing budget that sets it · built · ~75 minutes · Python
measured 0.8 s · 83 MiB · rubric 100 points in 30 cases - The scanner in C · built · ~90 minutes · C
measured 4.5 s · 65 MiB · rubric 39 points in 12 cases - Capstone: the conformity pack · built · ~100 minutes · Python
measured 5.7 s · 29 MiB · rubric 149 points in 37 cases
lessons/P02-L01-extraction-evaluation/ · tier cpu8 · prerequisites T00-L01-the-8gb-track
The lab. The student is handed 180 short synthetic remittance advices with gold labels, generated inside the lesson from a fixed seed, and a deliberately mediocre stand-in extractor with a seeded, documented error model — it reformats dates and amounts into its own house style, drops fields, invents payment terms that were never printed, smudges a supplier name the way a scan does, and gets a digit wrong in an amount, an invoice number or a payment term. They then implement, in pure numpy and stdlib:
normalise_value— a per-field-type normalisation policy (money, date, id, integer, text), including the trap that a comma is a thousands separator in one convention and a decimal point in another: a lone comma is decimal only when one or two digits follow it and end the number.match_value— exact, normalised and fuzzy matching, with fuzzy confined to text fields and its threshold inclusive. A later cell counts how many genuinely wrong amounts a fuzzy money matcher would wave through on this corpus.score_fieldandmacro_f1— per-field precision, recall and F1, where a wrong value counts as both a false positive and a false negative, and an unweighted macro average that refuses to let the field on every page bury the rare one.classify_cellandconfusion_by_field_type— five mutually exclusive labels per cell (correct, miss, spurious, wrong value, true negative), aggregated by field type under the matching mode it is given, so one F1 number becomes a work order.review_queueandapply_reviews— a confidence-ordered routing policy under a fixed budget in which every cell is a candidate, a confident silence included, ties break on(doc_id, field), and corrections land in new records rather than the ones being scored.
They finish by sweeping the budget and reading a quality/cost table: macro F1 against a random routing control, cost per document from placeholder rates they are told to replace, and the marginal F1 bought per unit of cost. On this corpus the yield rises then falls, because the lowest-confidence cells are all misses and the next band brings in wrong values — and fixing a wrong value is worth strictly more than fixing a miss. That is measured in the notebook, not asserted.
Why this is module 1 rather than module 3. A May 2026 study of structured contract
extraction found its domain-trained model weakest on currency fields requiring normalisation,
and scored its models with no currency parsing at all (both are sourced in the
lesson's claims.yaml). A student who has not written the normaliser cannot tell a formatting
difference from a defect, and will spend the rest of the programme filing bugs against models
that were already right.
lessons/P02-L02-layout-reading-order/ · tier cpu8 · prerequisites T00-L01-the-8gb-track,
P02-L01-extraction-evaluation
Lab: reading order from geometry. The student is given synthetic pages, laid out inside the
lesson from a fixed seed, as token boxes (text, x, y, w, h, page) covering
single-column, two-column and table-bearing layouts. They implement recursive XY-cut
segmentation (find_gaps, xy_cut), a lines-first sort within each block
(order_within_block) and a header/footer detector that masks digits (detect_header_footer);
then score their reading order against the gold sequence with Kendall's tau and a block-level
boundary F1 (kendall_tau, boundary_f1), and must show a setting where the two disagree: a
page shredded into blocks keeps a perfect tau. They finish by measuring, with module 1's scorer
reproduced inside the lesson, how much field-extraction F1 moves when reading order is wrong —
naive raster order scores 0.745 against 1.000 for gold order — which is the argument for caring
about layout at all.
lessons/P02-L03-sequence-labelling/ · tier cpu8 · prerequisites T00-L01-the-8gb-track,
P02-L01-extraction-evaluation, P02-L02-layout-reading-order
Lab: build an averaged structured perceptron in numpy over BIO tags on a document token
stream by implementing its parts: the features (word_shape, token_features — shape, affix,
neighbouring token, line head, line position and one head|shape conjunction), decode_spans,
sequence_scores, viterbi_decode and perceptron_update, whose transition half is graded
because one wrong tag breaks two bigrams. The training loop and the weight averaging are given.
Train it, then score it on held-out suppliers with module 1's harness — same
scorer, different extractor, which is the whole point — and compare it field by field with a
given rule-based baseline (improvement_table, fields_improved) to identify which field types
the tagger actually improved; here it still loses to the rule on counterparty. Two deliberate
departures: the token stream is regenerated inside the lesson from a fixed seed rather than
imported from module 2, since a student bundle is one lesson directory, and the weights are
averaged once per training document rather than once per update. One unplanned result is kept
in the open: greedy decoding beats Viterbi on held-out macro F1, because the learned transitions
cut the longer held-out supplier names short.
lessons/P02-L04-table-extraction/ · tier cpu8 · prerequisites T00-L01-the-8gb-track,
P02-L01-extraction-evaluation, P02-L02-layout-reading-order, P02-L03-sequence-labelling
Lab: on synthetic two-page tables laid out inside the lesson from a fixed seed rather than
imported from module 2, reconstruct rows and columns with one projection profile (ink_runs)
on both axes — row_bands on y, and columns on x read from body ink only, because a spanning
header's continuous ink hides the gaps beneath it. cells_in_band must cluster a band into
cells before assigning columns, and merge_continuations stitches a row split across a page
break, matching cells on (col, colspan). Then two scorers, content_f1 over the multiset of
cell texts and teds_like (built on row_distance, with a text-blind TEDS-Struct variant), and
a case where content is perfect while structure is wrong — every body row cut in two scores
content F1 1.000, TEDS-Struct 0.367 — and a case where the reverse is true. Ends with a rule: name the
consumer, and gate on the smallest set of scores that catches the defects that would break it.
teds_like is deliberately not TEDS — a two-level alignment over rows and cells, no rowspan,
clamped at zero — and the lesson says where the resemblance ends.
lessons/P02-L05-clause-classification-abstention/ · tier cpu8 · prerequisites
T00-L01-the-8gb-track, P02-L01-extraction-evaluation
Lab: 6,000 synthetic clauses of six types (governing law, limitation of liability,
termination for convenience, assignment, indemnity, change of control), some of them hybrid,
smudged or truncated, deduplicated as one pool before being split into training, calibration,
test and a fourth "next quarter" split, so no clause appears twice. The byte-pair-encoding
vocabulary is trained with tokenizers on the training split by given code, since filling in a
BpeTrainer call is API lookup rather than a skill this module teaches, and the lesson trains
it twice to show that it is byte-identical. The student implements the featuriser
(bag_of_subwords), a multinomial logistic regression in numpy trained by full-batch gradient
descent to a stated stopping rule (softmax, loss_and_gradient, train_softmax_regression),
per-clause precision, recall and F1 with abstentions counted as false negatives
(score_classes, macro_f1), temperature scaling and expected calibration error
(fit_temperature, expected_calibration_error), the coverage/accuracy curve and an
abstention threshold (coverage_accuracy_curve, choose_threshold, predict_or_abstain),
and the policy with its hand-off (fit_policy, to_review_records). The temperature and the
threshold may come only from the calibration split, and that ordering is graded directly:
replacing the training and test splits must leave the fitted policy unchanged. Abstained
clauses go to module 1's review queue, carried into the lesson under the same names and record
format, with their real calibrated confidence.
Two results are printed rather than assumed. The fitted temperature comes out below 1, and the student is asked what that means instead of being handed the familiar line that models are overconfident. And temperature scaling leaves every clause's predicted type unchanged but does reorder confidences between clauses; the lesson counts the reversed pairs. The module warns that even an honestly fitted threshold is a point estimate chosen to just meet its accuracy target: over 100 reshuffles of the held-out clauses it meets the target on unseen clauses only about as often as a coin flip. It names selective classification with a guaranteed risk as the fix this lesson does not implement.
lessons/P02-L06-review-queues-real-reviewers/ · tier cpu8 · prerequisites
T00-L01-the-8gb-track, P02-L01-extraction-evaluation
Lab: module 1's reviewer was perfect; these are not. Module 1's 180 documents and 1,080
(document, field) cells are re-created record for record from its own seeds, and three
simulated reviewers, two at tier 1 and one senior, label them with disagreement and with
fatigue that grows through a shift. The student implements raw agreement and Cohen's kappa over
the union of two reviewers' labels, including a label one of them never uses
(agreement_table, raw_agreement, chance_agreement, cohens_kappa), an SLA-aware queue
that serves arrived work earliest deadline first and reports its breaches (sla_schedule,
breach_report), a two-tier escalation rule (escalate), and the quality ceiling the harness
can see (harness_ceiling). On the day's 240 routed cells, earliest deadline first breaches no
SLA, against 26 for first-in-first-out and 59 for module 1's confidence order. Averaged over 20
simulated days, the rota chosen against a 0.05 breach-rate target is 60-item shifts: 35.6 items
an hour, no breaches, 7.5 routed cells still wrong a day, and macro F1 0.962 against the truth
where module 1's perfect reviewer gave 0.967. The rota is chosen on cells still wrong, not on
corpus macro F1, which moves by 0.007 across the sweep while cells still wrong roughly triple.
One departure from the original spec: the ceiling is not the reviewers' agreement itself. Their
kappa is 0.561; a perfect system scored against gold one of them wrote reaches about
sqrt(kappa), 0.749, against a measured 0.737 between the simulation's truth and reviewer A,
and a perfect extractor scored against A's gold reaches macro F1 0.976, not 1.000. The lesson
grades that form, states its assumption of independent mistakes, and shows the ceiling rising
to 0.935 when the second reviewer shares A's mistakes. Exercise 3's hint warns about the
readiness test that makes the scheduler loop forever, since the grader has no per-test timeout.
lessons/P02-L07-slice-discovery/ · tier cpu8 · prerequisites T00-L01-the-8gb-track,
P02-L01-extraction-evaluation
Lab: 1,600 synthetic documents carrying module 1's six fields and five attributes (vendor,
layout family, language, page count, scan quality), with two given extractor versions scored by
module 1's harness, carried into the lesson. The student implements slice discovery over single
attributes and two-attribute crosses above a minimum size (find_slices), module 1's macro F1
computed from per-document counts for every replicate at once (macro_f1_from_counts), a
paired bootstrap over documents for the change in a slice's macro F1, with a percentile
interval and a one-sided p-value that is never zero (paired_bootstrap), Holm's step-down and
Benjamini-Hochberg's step-up adjustments (holm_adjust, bh_adjust), the sample size a slice
needs before it may block a release (required_slice_size), and the release decision
(release_decision), in which a slice below that size is "too small to say" whatever its
p-value. On this release only the fax slice blocks; the size gate is what stops a 13-document
vendor slice (change −0.120, Holm-adjusted p 0.033) from blocking as well.
The regression is planted in one place, digit fields on faxed documents, and every slice inherits a diluted share of it. The lesson re-runs the whole pipeline on 20 null releases, one retrain of v1 against another, and counts how often each rule blocks when nothing changed: any drop 20 times, a drop beyond the margin 15, an uncorrected p-value 14, and Holm, with or without the size gate, never. It also discloses that the seeds were picked so that this one release shows every failure the lesson teaches, the Holm-flagged small slice among them. Depth-2 crosses are implemented and graded but not used for the gate: 253 slices raise the required size from 162 to 214 documents and put the smallest Holm-adjusted p-value that 2,000 bootstrap replicates can produce at 0.1264, above 0.05, so a search that deep could not block anything.
lessons/P02-L08-pipeline-drift-monitor/ · tier cpu8 · prerequisites T00-L01-the-8gb-track,
P02-L01-extraction-evaluation
Lab: a synthetic replay year of 364 days, 200 documents a day and a random audit of 25,
carrying a vendor template change, a seasonal mix shift and one real model regression. The
student implements the population stability index and KL divergence over field-value and
confidence distributions, on edges cut once on a baseline quarter and floored on both sides so
that KL(a‖e) + KL(e‖a) = PSI exactly (population_stability_index, kl_divergence,
fit_edges, drift_scores); the audited defect rate as a daily quality proxy, from module 1's
miss, spurious and wrong-value labels over reviewed cells only (daily_defect_rate); an upper
one-sided CUSUM with reset (cusum); the most sensitive threshold whose false-page rate on 200
simulated no-change years stays within a budget of one false page a quarter
(false_pages_per_quarter, choose_threshold); and a rule that names a change by which
distribution moved and whether the audited quality moved (triage). The chosen threshold is
printed beside the handbook's rules of thumb, h = 4 and 5, with the false pages each costs and
the documents each lets through before it pages.
Whether the monitor paged for an event is decided by counterfactual, not by calendar: the year is generated so that a copy with one event switched off differs from the log only by that event. On this replay the regression pages the day after it goes live, neither harmless change causes a page, and there are 0 unexplained pages in 105 event-free monitored days where the budget allowed about 1.2; paging on PSI above 0.25 does the reverse, paging for both harmless changes and never for the regression. Two warnings close the lab. The no-change years are resampled from the baseline quarter, because a real pipeline has no generator to draw from, so their tail is thinner than reality's. And a rolling baseline erases the template change within four weekly reports (confidence PSI from 0.3769 to 0.0053), which makes it invisible, not resolved.
lessons/P02-L09-cost-routing-budget/ · tier cpu8 · prerequisites T00-L01-the-8gb-track,
P02-L01-extraction-evaluation, P02-L03-sequence-labelling,
P02-L06-review-queues-real-reviewers, P02-L07-slice-discovery
Lab: a three-tier router (cheap extractor, expensive extractor, human review) priced from
named placeholders: module 1's machine cost and reviewer rate, module 6's reviewing pace, and a
per-page price for the expensive tier. It routes an eight-segment traffic fixture whose per-tier
accuracies are shaped on what modules 1–4 and 6–8 measured, each segment labelled with the
module it mirrors. That is the departure from the spec: a notebook runs alone and cannot import
another lesson's results, so these are fixtures, not the student's own measurements. The
student implements cost per document and field accuracy of a routing plan as volume-weighted
means over segments (evaluate_plan), an exhaustive sweep and the most accurate plan within a
cap (sweep_plans, best_within_budget), the Pareto frontier as a staircase rather than a
convex hull (pareto_frontier), a greedy marginal-yield allocator that records its path
(greedy_allocation), a segment mix on which the sweep beats greedy, graded against reference
implementations so that it must be a genuine counterexample (greedy_counterexample), and the
figures of a recommendation paragraph whose wording is given and adds no number of its own
(recommend).
Of 6,561 plans, 736 fit the placeholder cap of 0.30 per document. The sweep's plan costs 0.2920 per document; greedy, held to the same cap, leaves 2,436 more fields wrong a month; and the next step up the 48-plan frontier costs 1.74 per wrong field avoided. Greedy matches the sweep at 213 of 241 caps, at all 15 of its own breakpoints and on 130 of 150 random six-segment mixes, so the module does not call it unreliable: it measures where greedy is exact and has the student construct a case where it is not. Every price is a placeholder the student is told to replace before anyone signs, and swapping module 6's reviewing pace for module 1's placeholder alone moves one of the eight segments to another tier.
lessons/P02-L10-streaming-scanner-in-c/ · tier cpu8 · prerequisites T00-L01-the-8gb-track,
P02-L01-extraction-evaluation
Lab: implement, in lesson.c, a streaming field scanner in constant memory over a synthetic
export the program generates from a fixed seed: field_push (a fixed buffer, overflow counted
per field, not per byte), field_complete, scan_chunk (quoted fields, escaped quotes and
embedded newlines, with counters that must not depend on where the stream was split — graded
down to one-byte chunks) and scan_finish (a truncated final record versus a legal one with no
trailing newline). It is built with make and graded against a Python oracle on identical
bytes, and on peak RSS staying flat as the export grows. They measure throughput and peak RSS
against that Python scanner and learn where each factor comes from: C buys the throughput, the
chunk loop the memory. Two departures: the Python scanner is the one given in this lesson,
since module 1 has no scanner, and the export stops short of multiple gigabytes, because the flat
memory line is already unambiguous. Its common-mistakes section tells students to run
make clean after moving or rebuilding their checkout, as QUALITY.md requires.
lessons/P02-L11-capstone-conformity-pack/ · tier cpu8 · prerequisites
T00-L01-the-8gb-track, P02-L01-extraction-evaluation, P02-L02-layout-reading-order,
P02-L03-sequence-labelling, P02-L04-table-extraction,
P02-L05-clause-classification-abstention, P02-L06-review-queues-real-reviewers,
P02-L07-slice-discovery, P02-L08-pipeline-drift-monitor, P02-L09-cost-routing-budget,
P02-L10-streaming-scanner-in-c
Lab: the harness, the residual-by-class analysis, the drift alarm, the escalation policy
and the cost model from the earlier modules are carried in, given, and assembled into one
evidence pack for a synthetic pipeline of 240 documents; the seven graded exercises are the
validator that checks it. The student implements a reproduction gate that extracts every figure
from the pack's text and requires each one to reproduce, within the precision the text itself
claims, from a computed evidence bundle (extract_figures, computed_values, trace_gate);
three provenance checks: the residual-by-class table recomputed from the raw records on the
split the pack claims (check_residual_provenance), every cost figure presented as measured
cited to a matching entry in a measurement registry (check_cost_provenance), and an alarm
threshold calibrated strictly before the day it is credited with catching
(check_threshold_timing); and a validator that refuses the pack unless all four pass, graded
as an AND for every check rather than a majority (validate_pack). Module 10's C scanner is a
prerequisite by programme order but is not carried into the pack.
Four defects are planted, each built to be caught by exactly one check: a residual table computed on the training split and labelled as the evaluation split, a derived cost figure labelled measured with no source, a threshold calibrated through day 258 for an alarm on day 253, and a typed commentary figure that nothing computed. The lesson plants each alone in a clean pack and prints which check catches it. The reproduction gate alone catches one of the four, because the other three figures are correct for the data the preparer actually used and trace against the preparer's own evidence bundle. The module warns that repairs which only make a validator quieter (widening a tolerance, loosening the timing comparison by one step, dropping a check from the combination) leave the pack no more honest, and the rubric fails each of them.
Synapsa Commons · © 2026 RealAI · licensed under CC BY-NC-SA 4.0. You may use this material to learn and to teach, and copy, fork, share and adapt it. You must credit "Synapsa Commons by RealAI" with a link to https://github.com/TarrySingh/Artificial-Intelligence-Deep-Learning-Machine-Learning-Tutorials, say what you changed, and share adaptations under the same licence. You may not use it, or anything adapted from it, for commercial advantage or payment; for a commercial licence, contact RealAI. The full summary is in NOTICE; the licence's legal code governs.