|
| 1 | +# W26 quality probe — 2026-05-01 |
| 2 | + |
| 3 | +First end-to-end run of the Phase 1 quality probe. No LLM calls; this is the |
| 4 | +data-quality floor against which classification + report generation will run |
| 5 | +in subsequent PRs. |
| 6 | + |
| 7 | +## Headline |
| 8 | + |
| 9 | +**63.3% of YC W26 analyzed** — 124 of 196 companies pass the data-quality bar. |
| 10 | + |
| 11 | +## Coverage breakdown |
| 12 | + |
| 13 | +| Source | Count | Notes | |
| 14 | +|---|---:|---| |
| 15 | +| YC W26 official (Demo Day, 2026-03-24) | 196 | Per the [VC Corner W26 breakdown](https://www.thevccorner.com/p/yc-w26-demo-day-2026-complete-breakdown). | |
| 16 | +| yc-oss/api fixture (last refreshed 2026-02-08) | 132 | 64 companies missing — upstream is stale by ~3 months. | |
| 17 | +| Tier A (full classification) | 120 | All required fields + website returned 2xx/3xx. | |
| 18 | +| Tier B (partial — website unreachable) | 4 | Required fields present; website 4xx/5xx. Kept in charts with a flag. | |
| 19 | +| Tier C (excluded) | 8 | Acknowledged in the dropped register below. | |
| 20 | +| **Analyzable (A + B)** | **124** | Feeds every chart in the dashboard. | |
| 21 | + |
| 22 | +**Coverage of upstream:** 93.9% (124 / 132). |
| 23 | +**Coverage of YC official:** 63.3% (124 / 196). ← **headline metric** |
| 24 | + |
| 25 | +## Why the gap |
| 26 | + |
| 27 | +### 1. Upstream staleness (the bigger problem — 64 companies) |
| 28 | + |
| 29 | +`yc-oss/api`'s `meta.json` reports `last_updated: 2026-02-08T01:49:11Z`. W26 Demo Day was **2026-03-24**, so the upstream was last refreshed ~6 weeks before the batch closed. The Demo Day–era cohort (~64 companies) is missing from the feed entirely. |
| 30 | + |
| 31 | +This is not a bug in `yc-ai-pulse` — `yc-oss/api` is community-maintained. Mitigations: |
| 32 | + |
| 33 | +1. **Already in place:** the dashboard surfaces this gap upfront ("Upstream gap" alert banner). |
| 34 | +2. **B003 (open in BACKLOG):** add a CI cron that warns if the upstream is >48h stale. The W26 case would have tripped it ~3 months ago. |
| 35 | +3. **Future:** consider a direct YC profile-page enrichment (allowed under robots.txt for `/companies/<slug>`) for slug lists discovered from elsewhere. Not in v0.1 scope. |
| 36 | + |
| 37 | +### 2. Per-company drops (8 companies) |
| 38 | + |
| 39 | +Eight companies in the upstream feed were excluded from charts because they're missing fields the analysis layer requires. They are listed by name: |
| 40 | + |
| 41 | +| Slug | Name | Reason | |
| 42 | +|---|---|---| |
| 43 | +| `protent` | Protent | `long_description` empty | |
| 44 | +| `byteport` | Byteport | `long_description` empty | |
| 45 | +| `zerosettle` | ZeroSettle | `long_description` empty | |
| 46 | +| `traverse` | Traverse | `long_description` empty | |
| 47 | +| `grade` | Grade | `long_description` empty | |
| 48 | +| `zymbly` | Zymbly | `long_description` empty | |
| 49 | +| `moda` | Moda | `long_description` 57 chars (below 80-char threshold) | |
| 50 | +| `condor-energy` | Condor Energy | `website` field empty | |
| 51 | + |
| 52 | +Auditable threshold: `MIN_DESCRIPTION_CHARS = 80` ([src/ycai/coverage.py](../src/ycai/coverage.py)). Lowering it to 50 would bring `moda` back; raising it to 120 would drop ~6 more borderline rows. The current threshold balances inclusion with the requirement that classification be evidence-backed. |
| 53 | + |
| 54 | +### 3. Dead websites (4 companies — kept as Tier B) |
| 55 | + |
| 56 | +Tier B keeps these companies in the analysis but flags them in the dashboard: |
| 57 | + |
| 58 | +- `maywood` — Maywood |
| 59 | +- `caretta` — Caretta |
| 60 | +- `arzule` — Arzule |
| 61 | +- `servo7` — Servo7 |
| 62 | + |
| 63 | +These had 4xx/5xx responses at probe time. Could be transient. The verifier reruns at report build time (PR #3 acceptance gate). |
| 64 | + |
| 65 | +## What we already know about the analyzable 124 |
| 66 | + |
| 67 | +Industry distribution (from the YC-supplied `industry` field, no LLM yet): |
| 68 | + |
| 69 | +| Industry | Count | |
| 70 | +|---|---:| |
| 71 | +| B2B | 80 | |
| 72 | +| Industrials | 18 | |
| 73 | +| Healthcare | 9 | |
| 74 | +| Fintech | 8 | |
| 75 | +| Consumer | 6 | |
| 76 | +| Real Estate and Construction | 3 | |
| 77 | + |
| 78 | +The B2B-heavy distribution lines up with the [thevccorner.com breakdown](https://www.thevccorner.com/p/yc-w26-demo-day-2026-complete-breakdown) (64% B2B for W26). Internal consistency check passes. |
| 79 | + |
| 80 | +## Verifier results |
| 81 | + |
| 82 | +- `ok` (2xx/3xx): **127** websites |
| 83 | +- `dead` (4xx/5xx): **4** websites |
| 84 | +- `slow` (>5s): 0 |
| 85 | +- `redirect` (>3 hops): 0 |
| 86 | +- `error` (network): 0 |
| 87 | + |
| 88 | +## Reproducing this run |
| 89 | + |
| 90 | +```bash |
| 91 | +PYTHONPATH=src python3 -m ycai.cli run-coverage \ |
| 92 | + --batch winter-2026 \ |
| 93 | + --yc-official-count 196 |
| 94 | +``` |
| 95 | + |
| 96 | +Output: `runs/2026-05-01-185520/{dashboard.html, coverage.json, companies.csv}`. |
| 97 | + |
| 98 | +## Implications for downstream PRs |
| 99 | + |
| 100 | +- **PR #2 (researcher + classifier):** must consume `coverage.json` directly so its denominator agrees with the dashboard. The LLM never sees Tier C rows. |
| 101 | +- **PR #3 (deck/memo):** the methodology slide must show the same 63.3% headline, same upstream-gap callout, same dropped-register table. CI should fail if the deck cites a different denominator. |
| 102 | +- **PR #5 (release):** consider adding a "data freshness" indicator to the README badge so users know if the latest cached run is from a stale upstream. |
| 103 | + |
| 104 | +## Open follow-ups (added to BACKLOG) |
| 105 | + |
| 106 | +- [B004] Tune `MIN_DESCRIPTION_CHARS`. 80 is a guess; a small calibration study against the 8 borderline companies would let us pick a defensible value. |
| 107 | +- [B005] Add a "what's missing" section to the dashboard that compares yc-oss slugs to a slug list discovered from the YC `/companies/<slug>` profile pages, so we can name the 64 missing W26 companies, not just count them. |
0 commit comments