You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[Bug]: upstream-coverage matches only the bare slug literal per single file, so directory-path / split-across-files upstream citations read as unreferenced — a construction run logs hundreds of false SENSOR_FAILED #557
v2 (v2 branch), 2.3.3. core/tools/aidlc-sensor-upstream-coverage.ts + the dispatcher core/tools/aidlc-sensor.ts (harness-neutral — lives in core/, so all harnesses).
AI-DLC Phase
Every phase whose stages declare consumes:; the volume concentrates in Construction's per-unit design stages (functional-design, nfr-requirements, nfr-design, infrastructure-design).
Summary
upstream-coverage checks that a stage output references each artifact in the stage's consumes: list. Its match is a case-insensitive regex anchored on the bare slug: \b<slug>\b OR [[<slug>]] (aidlc-sensor-upstream-coverage.ts, the per-slug loop). The dispatcher fires it per written file (aidlc-sensor.tshandleFire threads a single --output-path), and the per-slug loop tests that one file's body.
Two structural mismatches between what the sensor demands and how this framework's artifacts actually cite upstream make it report a legitimately-referenced upstream as unreferenced:
Citation form. Artifacts cite upstream by directory path or backticked filename in their provenance header, not by bare slug. A representative nfr-design output header reads (paraphrased): "basis/trace: construction/api-service/nfr-requirements/, functional-design/, decisions.md …".
That header covers performance-requirements, security-requirements, scalability-requirements, reliability-requirements (all produced under nfr-requirements/) by naming the producing-stage directory once. The slug performance-requirements never appears as a standalone token, so \bperformance-requirements\b misses it and the sensor reports it unreferenced. (Note: \b<slug>\bdoes already catch a backticked `<slug>.md` — the . is a word boundary — so filename citations of the exact artifact are fine; the gap is the directory-path form and item 2 below.)
Per-file vs per-stage. A stage emits several artifacts and splits its upstream citations across them: in nfr-requirements, business-rules.md cites business-logic-model but tech-stack-decisions.md does not — and should not; a tech-stack decision doc has no reason to name the domain model. Because the sensor fires per file and demands the full consumes: set in each, every file that legitimately omits an upstream irrelevant to it fails. Coverage is a property of the stage's whole output, not of each file.
Neither is a bug in a specific artifact — they are how the framework's authoring conventions and the sensor's matching model diverge. The result is that the sensor's failures are dominated by false negatives.
Evidence (one real Construction run, 2.3.3)
A greenfield MVP run that reached Construction logged, from its audit shard:
186 SENSOR_FAILED, of which 178 are upstream-coverage (the other 8: 7 linter, 1 required-sections). upstream-coverage fired-and-failed in 18 of the run's stages.
Re-scoring only the final deliverables (excluding *-questions.md and memory.md scaffolding) with the shipped regex: of the slugs the sensor marked unreferenced, ~97% are still unreferenced in the finished artifact — i.e. these are not mid-draft states that pass on the last save, they never pass.
I then re-scored the same deliverables against the two mismatches above (count a producing-stage directory path reference as coverage; evaluate coverage over the stage's sibling *.md union rather than the single fired file). The unreferenced slug count dropped ~81–89%, and the residue was dominated by genuinely-absent references (e.g. business-logic-model in nfr-design, which an NFR design legitimately may not cite) — i.e. the signal survives, the noise clears.
#529 (debounce sensor firing) addresses the volume of fires — the same tuple re-firing on every save — and proposes firing document-shape sensors once at the stage boundary. That is real and complementary, but it assumes the terminal verdict is correct and characterises the failures as "half-written draft … only passes on the last one." The evidence above contradicts that assumption for this sensor: the final state still fails ~97% of the time. Firing once at the boundary would therefore still emit a large batch of false SENSOR_FAILED at each stage's gate — the noise moves to the gate, it does not clear. This report is about matching accuracy, not firing frequency; the two fixes are orthogonal.
Impact
SENSOR_FAILED is advisory (aidlc-sensor.ts header: "Sensor outcomes are advisory") so this does not block the workflow. The cost is:
Audit-ledger noise — hundreds of SENSOR_FAILED rows (+ a findings detail file each) that are mostly false, which buries the minority of real coverage gaps this sensor exists to surface.
A. Broaden the match to the citation forms the framework actually uses. In addition to \b<slug>\b / [[<slug>]], count a reference to the artifact's producing-stage directory (e.g. nfr-requirements/) as coverage. The dispatcher already resolves each consume's producer (producersOf in aidlc-sensor.tspresentConsumes), so the producing-stage slug can be threaded to the sensor alongside the artifact slug without new graph plumbing.
B. Evaluate coverage over the stage's output, not one file. Score the union of the stage directory's deliverables rather than the single fired file, so a citation in a sibling artifact counts. This can be done without changing firing granularity (the sensor reads sibling *.md at evaluation time), which keeps it independent of [Feature]: debounce sensor firing - the Write/Edit hook re-fires the same sensor + stage + path tuple on every save, so doc-heavy stages accumulate hundreds of advisory fires per run #529's firing-cadence change. Caveat worth calling out: the union should exclude scaffolding (memory.md, *-questions.md) — otherwise a slug that appears only in the Q&A file or the rule memory would falsely count as covered, trading a false negative for a false positive.
A design question underneath both: is a directory-path citation intended to satisfy coverage, or should artifacts be expected to name each consumed slug explicitly? That's a call for the sensor's owner — the two fixes assume the former (matching the current authoring convention), but if the latter is intended, the fix belongs in the stage authoring guidance / templates instead, and the sensor is behaving as designed (just louder than useful).
Environment
AI-DLC v2 2.3.3
Evidence from a single greenfield MVP run reaching Construction, on the Kiro IDE harness (IDE 1.0.116). The harness is incidental here: the sensor + dispatcher are harness-neutral core/ code, and the finding was reproduced by reading the run's audit shard and re-running the sensor over the on-disk artifacts, so the matching behaviour is identical on every harness and independent of the IDE version.
AI-DLC Version
v2 (
v2branch), 2.3.3.core/tools/aidlc-sensor-upstream-coverage.ts+ the dispatchercore/tools/aidlc-sensor.ts(harness-neutral — lives incore/, so all harnesses).AI-DLC Phase
Every phase whose stages declare
consumes:; the volume concentrates in Construction's per-unit design stages (functional-design,nfr-requirements,nfr-design,infrastructure-design).Summary
upstream-coveragechecks that a stage output references each artifact in the stage'sconsumes:list. Its match is a case-insensitive regex anchored on the bare slug:\b<slug>\bOR[[<slug>]](aidlc-sensor-upstream-coverage.ts, the per-slug loop). The dispatcher fires it per written file (aidlc-sensor.tshandleFirethreads a single--output-path), and the per-slug loop tests that one file's body.Two structural mismatches between what the sensor demands and how this framework's artifacts actually cite upstream make it report a legitimately-referenced upstream as
unreferenced:Citation form. Artifacts cite upstream by directory path or backticked filename in their provenance header, not by bare slug. A representative
nfr-designoutput header reads (paraphrased): "basis/trace:construction/api-service/nfr-requirements/,functional-design/,decisions.md…".That header covers
performance-requirements,security-requirements,scalability-requirements,reliability-requirements(all produced undernfr-requirements/) by naming the producing-stage directory once. The slugperformance-requirementsnever appears as a standalone token, so\bperformance-requirements\bmisses it and the sensor reports it unreferenced. (Note:\b<slug>\bdoes already catch a backticked`<slug>.md`— the.is a word boundary — so filename citations of the exact artifact are fine; the gap is the directory-path form and item 2 below.)Per-file vs per-stage. A stage emits several artifacts and splits its upstream citations across them: in
nfr-requirements,business-rules.mdcitesbusiness-logic-modelbuttech-stack-decisions.mddoes not — and should not; a tech-stack decision doc has no reason to name the domain model. Because the sensor fires per file and demands the fullconsumes:set in each, every file that legitimately omits an upstream irrelevant to it fails. Coverage is a property of the stage's whole output, not of each file.Neither is a bug in a specific artifact — they are how the framework's authoring conventions and the sensor's matching model diverge. The result is that the sensor's failures are dominated by false negatives.
Evidence (one real Construction run, 2.3.3)
A greenfield MVP run that reached Construction logged, from its audit shard:
SENSOR_FAILED, of which 178 areupstream-coverage(the other 8: 7linter, 1required-sections). upstream-coverage fired-and-failed in 18 of the run's stages.*-questions.mdandmemory.mdscaffolding) with the shipped regex: of the slugs the sensor markedunreferenced, ~97% are still unreferenced in the finished artifact — i.e. these are not mid-draft states that pass on the last save, they never pass.I then re-scored the same deliverables against the two mismatches above (count a producing-stage directory path reference as coverage; evaluate coverage over the stage's sibling
*.mdunion rather than the single fired file). Theunreferencedslug count dropped ~81–89%, and the residue was dominated by genuinely-absent references (e.g.business-logic-modelinnfr-design, which an NFR design legitimately may not cite) — i.e. the signal survives, the noise clears.Why this is distinct from #529
#529 (debounce sensor firing) addresses the volume of fires — the same tuple re-firing on every save — and proposes firing document-shape sensors once at the stage boundary. That is real and complementary, but it assumes the terminal verdict is correct and characterises the failures as "half-written draft … only passes on the last one." The evidence above contradicts that assumption for this sensor: the final state still fails ~97% of the time. Firing once at the boundary would therefore still emit a large batch of false
SENSOR_FAILEDat each stage's gate — the noise moves to the gate, it does not clear. This report is about matching accuracy, not firing frequency; the two fixes are orthogonal.Impact
SENSOR_FAILEDis advisory (aidlc-sensor.tsheader: "Sensor outcomes are advisory") so this does not block the workflow. The cost is:SENSOR_FAILEDrows (+ a findings detail file each) that are mostly false, which buries the minority of real coverage gaps this sensor exists to surface.blockingseverity lets aSENSOR_FAILEDhalt a gate, this sensor's current false-negative rate would block on citations that are in fact present. Accuracy should be fixed before (or with) making it able to gate.Fix directions (two independent root causes)
These are separable — either helps, both compose:
\b<slug>\b/[[<slug>]], count a reference to the artifact's producing-stage directory (e.g.nfr-requirements/) as coverage. The dispatcher already resolves each consume's producer (producersOfinaidlc-sensor.tspresentConsumes), so the producing-stage slug can be threaded to the sensor alongside the artifact slug without new graph plumbing.*.mdat evaluation time), which keeps it independent of [Feature]: debounce sensor firing - the Write/Edit hook re-fires the same sensor + stage + path tuple on every save, so doc-heavy stages accumulate hundreds of advisory fires per run #529's firing-cadence change. Caveat worth calling out: the union should exclude scaffolding (memory.md,*-questions.md) — otherwise a slug that appears only in the Q&A file or the rule memory would falsely count as covered, trading a false negative for a false positive.A design question underneath both: is a directory-path citation intended to satisfy coverage, or should artifacts be expected to name each consumed slug explicitly? That's a call for the sensor's owner — the two fixes assume the former (matching the current authoring convention), but if the latter is intended, the fix belongs in the stage authoring guidance / templates instead, and the sensor is behaving as designed (just louder than useful).
Environment
core/code, and the finding was reproduced by reading the run's audit shard and re-running the sensor over the on-disk artifacts, so the matching behaviour is identical on every harness and independent of the IDE version.