Stress-testing the autorouter on 45 real-world open-source boards

[drandyhaas]

Stress-testing the autorouter on 45 real-world open-source boards

To find out where the router actually breaks — not on toy boards but on real designs — we've been running it against a corpus of 45 downloaded open-source KiCad boards and grading the results against the boards' own (human-routed) copper. This post explains the idea, what it's surfaced, and the state of the fixes.

The idea

Most autorouter tests are synthetic. Real PCBs fail in ways you can't anticipate: power/mixed-type copper layers, KiCad 6/7/8/9/10 format quirks, fine-pitch BGAs and QFNs, USB-C connectors with coincident flip pins, DDR byte lanes, exotic net-class rules, parenthesized layer names, footprints that lie about their pad count. So:

1. Download real boards from their upstream GitHub repos (FPGA boards, SoCs, DDR test beds, keyboards, SDRs, motor controllers…).
2. Strip the existing routing (tracks/vias/pours) to get a clean unrouted board, but keep a copy of the original, routed version.
3. Re-route from scratch following the normal plan-pcb-routing methodology (fanout → diff pairs → signal routing → power planes → repair → verify), one board per agent.
4. Grade against the original. The human routing is ground truth for "what a manufacturable, DRC-clean board looks like."

What we grade against (and why it matters)

Two refinements turned out to be essential and are worth highlighting on their own:

• Ground-truth DRC, not zero. We DRC the original routed board at the design clearance and use that as the floor (real boards carry a few clearance-independent hole-to-hole/pad violations). Comparing our output to the original's count, instead of to 0, is the honest delta.
• Clearance/track/via come from the real routing, not blindly from the net class. Measuring the originals showed the netclass clearance is frequently not the effective DRC rule — e.g. ulx3s routes at 0.1 mm while its netclass says 0.2; several boards' project min_clearance is a stale editor floor, not the copper rule. A new compare_to_original.py step contrasts our routing with the original (via count, total copper, track-width strategy, layer balance) and emits concrete suggestions.

We also record per-step and total agent timing so runs are comparable over time.

The corpus (three 15-board sets)

• Set 1: glasgow, hackrf, ottercast, bitaxe, lumenpnp, castor_pollux, lpddr4-testbed, neo6502, smartknob, megadesk, haxophone, upsy-desky, piantor, splitflap, urchin.
• Set 2: ulx3s, cynthion, butterstick (8-layer!), cparti (Artix-7), schoko, tinytapeout, ddr5-testbed, caravel, system76-launch, crkbd, sofle, lily58, free-dap, usb-sniffer, nitrokey.
• Set 3: daisho (USB3, 8-layer), orangecrab (ECP5 + DDR3, 6-layer), keks & eis (ECP5 SoCs), upduino & fomu (iCE40), tigard (FT2232H debug), watchy (ESP32 e-paper), lora_v3 (STM32WLE LoRaWAN), a20_can (Allwinner SBC), lna3030 (RF LNA), olimex_lora, esp_prog, mod_bme280, throwing_star (all-through-hole Ethernet tap). Spans L2→L8 and 9→392 footprints, and deliberately heavy on old KiCad 4/5 formats rescued via the pcbnew normalize round-trip.

(The harness, runbook, and corpus manifest live under tests/stress/.)

What it's fixed so far

The stress runs have already driven 100+ fixes. Grouped:

Parser robustness — power/mixed copper layers dropped (#76), Edge.Cuts cross-matching (#77), KiCad 6/7 fp_text reference collapse (#78), mixed numeric/name net refs silently dropping copper (#79), unnumbered pads treated as targets (#94), oval/slot drills misread as SMD (#106), and parenthesized layer names ("In1(GND).Cu") parsing as 0 copper layers, plus an unbalanced ( in a property value swallowing downstream footprints' pads (#113), and — new from Set 3 — rotated-package SMD pads modelled 90° off their true shape: a 90°-rotated TSSOP/QFN row collapsed into one phantom copper wall that boxed in every pin (caught on lora_v3's I²C sensor row; 157/157 rotated pads on that board were mis-modelled). Fixed in v0.15.14 — pad geometry is now resolved at the pad's true board orientation — together with exact rotated/non-orthogonal BGA collision in fanout (#137). Since then: custom pads are modelled by their real copper extent rather than the anchor size (rotated custom-pad sizes and slot-drill diameters were under-reported — fixed in both the file parser and the live-board build_pcb_data path), then a step further — the real comb/finger polygon, not just its bounding box (#188): a MOSFET's large exposed source/drain pad is a notched comb, and filling those notches solid walled in the gate/source pads that route through the channels (bitaxe Q1/Q2 had four signal/power nets plus five GND plane pads unroutable, all reporting the misleading "no rippable blockers found"). The obstacle map and DRC now rasterize the true outline — parsed from the file's gr_poly primitives and, in the GUI, from pcbnew's effective polygon (degrading safely to the bounding box when the geometry cache is unavailable headless) — and those nets route. And locked tracks are now honoured as obstacles (#150) so the router no longer lays copper straight through existing locked copper.

Fanout — KiCad-10 numeric net refs (#80), B.Cu components getting F.Cu stubs (#96), self-blocking escape bands (#97), defaulting to F.Cu/B.Cu regardless of layer count (#98), and the big set-2 one — fully-populated BGAs silently dropping deep interior balls, now solved by a new under-pad escape engine (#122, highlighted below). Set-3 follow-ups: LGA and fine-pitch land / chip-scale arrays are now fanout candidates instead of stranding their interior pads (#144), and escape-stub ends now snap to the routing grid with their end-jog obstacle-checked so the downstream router picks them up cleanly (#149). Plus a geometric fit-up pass: escape via/track are sized to the actual pitch budget (a 0.5 mm via won't fit an 0.8 mm half-pitch at 0.1 clearance), and bga_fanout warns when they don't, so escape stubs stop grazing the adjacent fanout vias (#158). And the under-pad via-drop escape — first added for BGAs — is now available for QFN / fine-pitch edges too (qfn_fanout --escape-method underpad), for crowded rows where the surface 45° fan has no room (#164). And qfn_fanout's 45° escape stubs no longer graze adjacent diff-pair pads at 0.4 mm pitch — it emits a post-fanout drc_grazes summary and the planner shrinks the stub width until the escape is DRC-clean (#179). And two v0.17.0 escape-geometry fixes: qfn_fanout ramps its fan angle to 45° at the corners to spread the stub tips on crowded edges (#200), and a via-in-pad is clamped to its pad edge at placement so it can't bulge past the pad (#202).

Routing core — stub layer swaps moving SMD stubs off their pad (#83), partial multipoint nets mishandled on retry (#85), no output when 0 nets route (#86), negative success counts (#87), one unroutable pad abandoning a whole multipoint net (#101), retry chaining never offering prior tracks for rip-up (#103), boxed-in hint recommending an OOM-inducing grid (#105), multi-point layer swaps (#17), incremental output so a killed long run keeps its routed nets (#100), board-global fine-grid (0.05 mm) OOM on large boards (#109), and wide-pitch through-hole headers / keyswitch / diode footprints misdetected as BGA exclusion zones (keyboards 0% routable at defaults, #82). Newer core work tightened via obstacle geometry — vias are stamped and cleared at their actual size (off-grid vias, a via-via clearance float floor, and same-net via-in-pad reuse instead of a redundant adjacent via) — added a post-route redundant-cycle prune (check_cycles) so rip-up/retry no longer leaves a signal net carrying a closed copper loop, necks wide power segments that graze a foreign pad at a terminal, connects terminal segments to the exact pad endpoint instead of a grazing grid cell, and dropped the now-redundant appendix-collapse pass (sweep_dead_ends covers it, #148). The off-grid terminal connector then turned out to be an active hazard — a long diagonal from a relocated terminal to an off-grid fanout-stub end ran unchecked across foreign vias (keks stranded /SRAM_A6 over an /SRAM_A17 via) — so terminal endpoints are no longer relocated off-grid (#158/#159), and the geometric fix_self_intersections and collapse_appendices post-passes were removed outright rather than guarded (#159/#162): they straightened or extended copper in ways that traded one short for another, and on-grid routing plus the dead-end sweep make them unnecessary. A later set-3 audit — driven by grading at the minimum routed clearance plus a whole-set A/B replay (ab_replay_grade.py / --regrade) — surfaced a cluster of sub-clearance grazes the obstacle map was rounding away: grid-quantized clearance and via-keepout radii were floored where they needed ceiling (#154/#155), terminal/endpoint and plane-tap connection segments grazed foreign pads (#157), the track-block obstacle lacked the diagonal margin the via-block already had and kept out from the configured net width instead of a segment's actual width (#172), board-edge via keep-out swept only the track band rather than the full via band (#170), and tap rip-up re-routed a victim straight through an already-placed foreign via (#171). With those fixed, the remaining real different-net shorts and via-over-track overlaps on the densest set-3 boards (eis, daisho, esp_prog) — down to tens of microns — are caught and cleared (#169/#174). The latest batch adds a via-in-pad unblock: a signal pad boxed against same-layer copper drops a via in its pad and escapes to a free layer (generic across single-ended, diff-pair and plane routing, triggered when a net gets stuck), with failed via sizes memoised so the retry doesn't blow up; a fine-grid retry that routed a different net through an existing trace — a real inner-layer short — is caught by re-routing the nested-cascade victims that cross a kept tap (#186); the multipoint pad counter that under-reported connected pads (flagging as disconnected pads that check_connected confirms whole) now derives its tally from the final-board union-find (#184); on dense 2-layer boards the default B.Cu layer-cost over-biased F.Cu (excess vias, poor balance), so --layer-costs is exposed and the plan now derives per-layer costs from the planned plane assignment (#178/#185); and literal !-prefixed (active-low) net names are now selectable instead of being parsed as exclusion patterns (#177). v0.17.0 then replaced the approximation underneath all of this: a track or via keep-out is now modelled as the exact swept capsule of the segment/via at its real width — the exact keep-out is the default (the earlier opt-in gate removed), at route.py ↔ plane-router parity, with the rip-up blame map built from the same capsule (#203) — so the grid-quantized floor/ceiling/diagonal-margin patches above collapse into one correct geometry that neither over-blocks a threadable channel nor flags it. This is the Rust change behind the v0.17.0 crate bump (#173/#156).

Diff pairs — explicit P/N pairs (#54), pair self-crossing near termination resistors (#56), and the important one: a pair reported "routed" while its P/N copper crossed 4× (real shorts, #102). Plus inner-layer BGA-escaped stubs stranded by an F.Cu/B.Cu --layers default (#116) power/diff-pair detection gaps on real-world net names (#91), diff-pair globs not matching hierarchical (slash-separated) net names (#120), and route_diff unable to launch a pair from inner-layer / BGA-escaped fanout stubs (#121). And the connector-cluster case (#90): coincident USB-C flip pins and other tightly-clustered multi-terminal pairs now route. Terminals pair same-component first — a connector's own D+/D- pins pair with each other rather than a geometrically-nearer neighbouring pad (which used to cross-couple and run a leg straight across the partner pad, shorting it) — and electrically-short legs route single-ended instead of forcing a coupled pair through near-coincident pads: a leg shorter than ~5× the connector setback has no real coupled section (only fan-in/out), so it is deferred and finished by the single-ended pass at matching width. All three set-1 USB-C boards (castor_pollux, smartknob, upsy-desky) — previously 0 coupled pairs routed — now route their D+/D- pairs. A later detection/robustness batch: DP/DN, indexed P0/N0, and comma-separated --diff-pairs now parse (#143), TXP/TXN legs pair up (#151), crystal/oscillator P/N net names no longer false-positive as differential (#145), and a pair that self-crossed after a bare-pad target swap is fixed (#142). Passive ± terminals (buzzers, Net-(REF-±)) no longer false-positive, and USB D+/D- carried on chip-path (hierarchical) net names — which don't follow the P/N suffix convention — are now paired (#181). The biggest set-3 diff-pair work is the connector launch path (#165): setback/launch segments from a connector pad used to graze the partner net's pad (the obstacle map excluded the whole pair from itself), so legs now launch from the pad edge along the escape direction, terminals use optimal min-cost pairing, and multipoint terminal-relocation no longer loses vias or collides in dense pad fields — verified end-to-end on a real-world board (Tigard, FT2232H USB, covering bare + under-pad-scoped + non-scoped-stub fanout). And on name-referenced (non-KiCad-10) boards the polarity-swap stub relabel matched 0 segments and left a swapped pair's stubs on the pre-swap net (reported disconnected though the copper was whole) — now fixed by matching the net-token format the file actually uses (#163). The connector rework briefly regressed two boards' USB source-escape (esp_prog #166, watchy #167) and overshot the setback past a long source pad — both fixed — and a diff pair that routed with sub-clearance spacing to its own test point (orangecrab #168, including the stub layer-swap clearing the partner polarity's test point) is now clean. v0.17.0 then rewrote the escape itself into a direct-coupled-middle router: a pair's coupled middle routes on any layer, each leg drops its own escape via, and the terminal legs go point-to-point, so a pair escapes and fully routes inside route_diff (inner-layer launch included, with the P/N offset side chosen to minimise terminal-leg crossings). Boxed endpoints now escape by exact geometry (#197), and under-pad-escaped diff pads that used to strand connect via a via-reachable multilayer launch (#195). A 3 mm (λ/10 @ 5 GHz) electrical floor on the coupled-length test sends genuinely short legs single-ended, with a GUI "Hide short routes" toggle to match.

Power planes — pad-level plane-connection repair (#99), fine-pitch pad taps (#104), and Voronoi multi-net planes silently skipping through-hole pads that land in another net's region (#107), pouring zones for all-through-hole nets that place no stitching via (#114), restoring a ripped net's original trace when --rip-blocker re-route fails instead of dropping it (#88), zone-fill segments and stitching vias placed outside the board outline (#119), same-net stitching vias below the hole-to-hole drill minimum (#125), and connected-pad accounting that disagreed with geometric connectivity — an offset via-in-pad now credited as connected (#89). Two more: route_disconnected_planes --reroute-ripped-nets now reroutes every unrouted net, including ones ripped by the earlier route_planes pass (#141), and dense inner plane layers no longer place stitching vias / connection traces inside signal clearance (#146). And plane repair no longer boxes in nearby protected-net pads with its own stitching (#180): the distant-trace reach is unified to the --max-search-radius parameter, a stranded pad now traces to an escaped same-net SMD pad or to existing plane copper before dropping a redundant via, and a fine-pitch repair retry ladder (smaller via, then finer grid) recovers pads the first pass leaves disconnected.

Connectivity checking — power-net skip list hiding unrouted nets + co-located pads misreported (#92), footprint built-in thermal vias not credited (#108).

Memory — board-edge polygon rasterization OOM (#81).

Windows / console — --bus and the GUI stdout redirector no longer crash with UnicodeEncodeError on a cp1252 console, and UTF-8 console output is forced where the terminal allows it (#152).

Design rules / DRC grading — the dominant set-1/set-2 DRC source. KiCad net-class track_width/via_diameter are drawing defaults, not DRC minima, so the tools now read the DRC-enforced Board Constraints (design_settings.rules) and emit a small working via down to min_via_diameter instead of the net-class nominal (#115), and grade DRC at the manufacturing floor (board Constraint or JLC fab min) rather than the inflated net-class clearance — fine-pitch escapes routed to that floor stop reading as board-wide violations (#111). And to close the loop inside KiCad itself, routing now writes the board's DRC design rules back to match the floors it actually used — a complete Default net class plus the constraint set (clearance, hole-to-hole, copper-to-edge, min via/track), with non-routing categories (thermal, courtyard, mask) demoted — so opening the routed board and running the interactive DRC shows the same honest count, not hundreds of stock-default false positives. It is auto-invoked after routing from both the CLI and the GUI (fix_kicad_drc_settings, shared logic) (#160) — across every routing entry point now (single-ended route.py, route_diff.py, route_planes.py, and route_disconnected_planes.py; in the GUI a single shared toggle covers single-ended, diff-pair, and plane create/repair), with --keep-thermal (an Advanced-tab checkbox) to leave thermal-relief severity untouched when you'd rather keep it. And the fab-floor track-width and via/hole-size checks now grade against the fine-via floor, so the small working-via escapes don't read as track/via-too-small violations (#176).

A/B verification (same router; only via-size + clearance + grading changed) on two fine-pitch 2-layer boards — the new working-via + manufacturing-floor approach both completes more of the board and grades clean:

• free_dap (TQFP-64, 0.4 mm pitch): old netclass params route 15/24 single nets (9 fail on the 0.4 mm escape); new params route 24/24, all 68 pads. DRC on the new board: 2 at the manufacturing floor vs 20 at the old netclass stick (the extra 18 are manufacturable escape copper). Human original: 0.
• smartknob_base (QFN-20, 0.4 mm): old 47/57 single nets (10 fail); new 57/57. DRC: 0 at the floor vs 282 at the netclass stick — the same false explosion the earlier run reported (114 → 212), now gone, and cleaner than the human original (28 at the floor).

And on two 4-layer boards run through the full pipeline (fanout → signal → planes → repair) with the working via + floor — which also exercises the small-via lever (#115), since these place real vias:

• bitaxe_ultra (QFN, 93 nets): 58/59 signal nets + 190/192 multipoint pads + planes (GND 136/140, /5V 24/24); all 198 vias are the 0.5 mm working via, not the netclass 0.6. DRC 21 at the manufacturing floor vs 632 at the netclass stick (Per-net/region clearance: global --clearance forces board-wide DRC when escaping fine-pitch parts #111); human original 1.
• tinytapeout_qfn (QFN-64 0.4 mm + dual 0.5 mm board-to-board connectors — a genuinely dense board, so still partial): 138 working vias placed; DRC 69 at the floor vs 1134 at the netclass stick, and cleaner than the human original (164 at the floor).

Net effect on the hardest boards: glasgow's diff pairs went 0 → all 13 FPGA pairs (now escaped coupled through the BGA, #182), its plane repair stopped crashing (90 → 18 unconnected plane pads), and several boards' completion jumped 15–20 points (bitaxe 70 → 90%, lumenpnp 75 → 95%).

Dense-BGA under-pad escape (#122)

The biggest set-2 fanout finding: the channel-template fanout confines every layer to the gaps between ball rows, so on a fully-populated array a few channels over-subscribe (conflict cliques far exceeding the layer count) and the deepest balls can't escape. ulx3s (22×22, 0.8 mm, 4-layer) dropped ~23 balls, which then failed signal routing as “no rippable blockers” and dominated its shortfall.

A new escape engine — --escape-method underpad — fixes it by exploiting a fact the channel model ignores: SMD pads block only their own layer, so inner-layer copper can run straight under the pad field. Each signal ball drops a via in its pad and escapes on an inner layer (deepest-first, inside-out), while near-rim balls escape via-less on the BGA's own layer to keep the rim clear so the deeper balls can run beneath them. Power/gnd balls tap their plane rather than fanning out.

Result: it escapes every dense board the channel router couldn't — ulx3s 22×22 and cparti 24×24 (Artix-7) on just 4 layers → 0 dropped; cynthion/schoko/butterstick come out DRC-clean at the manufacturing floor. It works top- and bottom-side, and composes with diff-pairs- or power-planes-fanned-first (it auto-skips already-fanned nets and routes clear of their copper). It also escapes differential pairs coupled (#182) — each pair's P and N exit on one shared layer at the diff spacing (edge pairs via-free on top, deeper pairs end-on on an inner layer, a pair that can't fit a coupled corridor falling back to single-ended) — so route_diff re-pairs them instead of being handed two single-ended legs: on glasgow (BGA U30, 0.8 mm) all 13 FPGA Z-pairs now escape coupled and route, DRC-clean. We verified the capacity claim against the human-routed ulx3s, which uses dog-bone vias offset into the diagonal gaps; plain via-in-pad reaches the same 0 with a small via/track for the pitch (e.g. via 0.35 / track 0.12 at 0.8 mm). The lone residual — 4 via-vs-decoupling-cap overlaps where a cap sits exactly at a ball — is now handled by a decoupling-cap placement repair (#130): after fanout, a pass nudges each near-BGA decoupling cap or resistor off any foreign-net fanout via (and foreign track on its side) and pulls its pad toward the nearest same-net ball, so a power/GND via dropped there later lands on the cap too. Caps move as little as possible, never overlap each other, and any that can't clear are reported. It ships as place_fanout_clearance.py, as an "Optimize decoupling cap placement" option (with the placement knobs) on the GUI's BGA fanout tab, and as an auto-added step after BGA fanout in the Claude plan. On ulx3s U1 (22×22) it takes the fanned board from 4 PAD-VIA overlaps to fully DRC-clean. And when an escape leaves a sub-clearance graze even though every ball escaped, bga_fanout now surfaces it (a drc_grazes field in its JSON summary, DRC'd at the fanout clearance), so the planner retries the fanout with a smaller via / thinner track toward the fab floor for the via-over-track class.

What's still open — the big levers

The remaining failures cluster tightly. Set 2 (which deliberately loaded up on fine-pitch parts) made the #1 issue — fine-pitch escape DRC — unmistakable; that one is now addressed (#111/#115, above), leaving:

• Planes (route_planes: secondary power plane (Voronoi multi-net) starves the smaller net's peripheral pads #112). Secondary power planes starve peripheral pads. (The connected-pad counter that disagreed with geometry route_planes: connected-pad accounting disagrees with geometric connectivity #89, the skipped pour when a net has no stitching via route_planes: skips the zone pour when all of a net's pads are TH/already on the layer (0 vias placed) #114, out-of-outline plane vias route_disconnected_planes places zone-fill segments and stitching vias outside the board outline #119, and sub-minimum stitching-via spacing Same-net vias placed below the hole-to-hole drill minimum #125 are now fixed. And plane-repair now connects a pad that can't reach its plane because it is too small to drop a via in and a signal trace blocks the only corridor: it traces to an adjacent same-net pad — a connector GND pin to its shield pad — ripping the blocking net with --rip-blocker-nets and re-routing it, restoring any net that can't re-route. Fixed castor_pollux's stranded USB-connector GND pin J2.5; wired into the plugin's plane-repair tab too.)
• Quality/robustness (route.py multipoint: orphan dead-end stubs and same-net self-crossings on successfully routed nets #84, Router reports net fully routed but produces disconnected trace segments #8). Orphan stubs and a reopened intermittent "routed but disconnected" case. (Fine-grid sub-cell pad-clearance DRC Fine grid steps produce sub-cell clearance DRC violations (obstacle expansion floors where pad/track clearance needs ceil) #70 — connecting terminal segments to the exact pad endpoint plus ceil-correct obstacle expansion — and the fix_self_intersections / collapse_appendices geometric post-passes — long a source of trade-one-short-for-another "butterfly" regressions (fix_self_intersections straightens a same-net detour across foreign copper, creating a different-net short (RAM_A9); can't be guarded/deferred in place (rip-reroute butterfly) #147) — are now removed outright (Terminal connector to off-grid fanout-stub end is an unchecked long diagonal that crosses foreign vias (keks /SRAM_A6 232µm over /SRAM_A17 via) #159/Same-net self-crossings: find the source and remove them cleanly (delete-only, not geometric extension) #162), with clean delete-only same-net self-crossing removal tracked as the open follow-up (Same-net self-crossings: find the source and remove them cleanly (delete-only, not geometric extension) #162); incremental output route.py: no incremental output — killing a long run loses all routed work #100, board-global fine-grid OOM route.py: board-global route at fine grid (0.05mm) OOMs >1GB on large 4-layer boards #109, and rip-blocker restore route_planes: --rip-blocker-nets restores/re-routes ripped nets on top of new plane copper, with default params #88 already were.)

Discussion

A few things we'd love input on:

• Per-net/region clearance turned out not to be needed: we route each net class at its own clearance and grade DRC at the manufacturing floor (the board's DRC-enforced Constraints + JLC fab min), which is the rule the human board already passes. Is that the right call, or are there boards where genuinely local (rule-area) clearance is unavoidable?
• Worth teaching the tools to read routing parameters from an existing reference board when net classes are unreliable?
• Writing the routed board's DRC settings back to match the clearances actually used — so KiCad's interactive DRC shows the same honest count we grade — is now implemented and auto-applied after routing (Make KiCad DRC settings consistent with the router's assumed clearances after routing (CLI: edit .kicad_pro; GUI: set live design settings) #160). Still open as a direction: a per-board "tune" mode that adjusts the routing CLI options from each step's own failure feedback (plan-pcb-routing: optional "tune" mode that adjusts routing CLI options per-board from prior-step feedback #153).
• Other real-world boards we should add to the corpus (especially unusual stackups / packages)?

Findings and the full per-board scorecards are tracked in the issues linked above.