Skip to content

Commit 11ac87b

Browse files
Your Nameclaude
andcommitted
feat(semantic-facts): split effects confidence into event/target (P3a)
Executes P3a of docs/plans/2026-08-08-derived-artifact-hardening-execution-plan.md. symbol_effects.confidence is split into event_confidence (certainty an effect happened -- always "exact" in v1, every extraction site fires only on a real syntactic raise/throw/write node) and target_confidence ("exact" | "none" -- certainty about what the target is). This recovers 3 Python throw facts the pre-P3 code silently dropped instead of recording with degraded confidence: - `raise e` (bound exception variable, PEP-8 casing heuristic can't resolve it to a type) - `raise factory()` (lowercase call, syntactically identical to `raise SomeException()`) - bare `raise` (re-raise) -- previously had no target text at all structurally, so was skipped entirely rather than recorded Java/TS/JS throw detection is untouched: those require a real object_creation_expression/new_expression constructor node structurally, so they're always target_confidence="exact" when they fire at all -- the PEP-8 casing uncertainty is Python-specific (verified by reading detect_java_throw/detect_tsjs_throw before assuming a blanket classifier would be correct). - schema.rs: symbol_effects gains event_confidence/target_confidence columns (fresh-install CREATE TABLE + migrate_add_column for existing installs; old confidence column left in place on upgraded DBs, matching every other migration in run_migrations). - SOURCE_EXTRACTION_VERSION bumped 1->2 -- the first live exercise of the P1 drift-guard mechanism: it correctly failed first (hash mismatch) on this exact change, then passed once the expected hash was updated alongside the version bump. - Confidence classification lives in walk_effects (language+kind aware), not inside each per-language detect_* function -- keeps all 7 existing detectors' signatures untouched. - New symbol_info_surfaces_effect_confidence_split end-to-end test; understand_surfaces_architecture_digest_and_t1_facts and P1's derived_artifact_versions.rs updated for the schema/output change. Deliberately deferred, not attempted: detect_go_write (P3b). Needs the enclosing method's receiver PARAMETER NAME tracked through walk_effects's recursive traversal (Go's receiver is a plain identifier, not a keyword like Rust's self, so it can collide with an unrelated same-named local in a different function) -- real design work the module's own doc comment already flags as not wired. Left as its own future slice rather than rushed alongside this phase. Verified: full `cargo test --workspace --features embeddings` green (1054 calm-core + 367 calm-server + all other packages, 0 failures), clippy -D warnings clean, rustfmt clean, locate/symbol_info/understand toolsnaps regenerated. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 4fe66bf commit 11ac87b

12 files changed

Lines changed: 337 additions & 45 deletions

File tree

crates/calm-core/src/db/schema.rs

Lines changed: 41 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -322,13 +322,26 @@ CREATE INDEX IF NOT EXISTS idx_type_relations_path ON type_relations(source_path
322322
-- call_sites.enclosing_qn already is. Same rebuild lifecycle as
323323
-- type_relations above -- no source_hash needed.
324324
CREATE TABLE IF NOT EXISTS symbol_effects (
325-
id INTEGER PRIMARY KEY AUTOINCREMENT,
326-
symbol_qn TEXT NOT NULL,
327-
effect_kind TEXT NOT NULL CHECK (effect_kind IN ('explicit_throw', 'write_field')),
328-
target_text TEXT NOT NULL,
329-
confidence TEXT NOT NULL DEFAULT 'syntax_exact',
330-
source_path TEXT NOT NULL,
331-
line INTEGER NOT NULL,
325+
id INTEGER PRIMARY KEY AUTOINCREMENT,
326+
symbol_qn TEXT NOT NULL,
327+
effect_kind TEXT NOT NULL CHECK (effect_kind IN ('explicit_throw', 'write_field')),
328+
target_text TEXT NOT NULL,
329+
-- P3 (docs/plans/2026-08-08-derived-artifact-hardening-execution-plan.md):
330+
-- split from the old single `confidence` column -- `event_confidence`
331+
-- is certainty THAT the effect happened (currently always 'exact': every
332+
-- extraction site fires only on a real syntactic raise/throw/write
333+
-- node); `target_confidence` is certainty about WHAT the target is
334+
-- ('exact' | 'none'). A Python `raise e`/`raise factory()`/bare `raise`
335+
-- is a real, certain throw EVENT whose exact exception TYPE isn't
336+
-- syntactically knowable without full resolution -- previously dropped
337+
-- entirely (see semantic_facts.rs's module doc comment history);
338+
-- target_confidence='none' now records the event honestly instead of
339+
-- losing it. write_field's target (the field name) is always exact
340+
-- once detected, so it's always 'exact' on both dimensions.
341+
event_confidence TEXT NOT NULL DEFAULT 'exact',
342+
target_confidence TEXT NOT NULL DEFAULT 'exact',
343+
source_path TEXT NOT NULL,
344+
line INTEGER NOT NULL,
332345
UNIQUE(symbol_qn, effect_kind, target_text, line)
333346
);
334347
CREATE INDEX IF NOT EXISTS idx_symbol_effects_symbol ON symbol_effects(symbol_qn);
@@ -883,6 +896,27 @@ fn run_migrations(conn: &Connection) -> rusqlite::Result<()> {
883896
OR definition_snapshot IS NULL",
884897
[],
885898
)?;
899+
// P3 (docs/plans/2026-08-08-derived-artifact-hardening-execution-plan.md):
900+
// splits the old `symbol_effects.confidence` into `event_confidence`/
901+
// `target_confidence` (see the fresh-install CREATE TABLE comment for
902+
// the semantics). The old `confidence` column is left in place on an
903+
// upgraded DB rather than dropped -- every migration in this function
904+
// is purely additive, and `symbol_effects` is fully DELETE-then-
905+
// reinsert on every reindex of the owning file anyway, so it becomes
906+
// dead weight the next time each row's file is touched, not a
907+
// permanent liability.
908+
migrate_add_column(
909+
conn,
910+
"symbol_effects",
911+
"event_confidence",
912+
"TEXT NOT NULL DEFAULT 'exact'",
913+
)?;
914+
migrate_add_column(
915+
conn,
916+
"symbol_effects",
917+
"target_confidence",
918+
"TEXT NOT NULL DEFAULT 'exact'",
919+
)?;
886920
Ok(())
887921
}
888922

crates/calm-core/src/indexer/edges.rs

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,10 @@ pub struct SymbolEffectData {
117117
pub symbol_qn: String,
118118
pub effect_kind: &'static str,
119119
pub target_text: String,
120+
/// P3 (docs/plans/2026-08-08-derived-artifact-hardening-execution-plan.md):
121+
/// "exact" | "none" -- see `RawEffect::target_confidence`, which this
122+
/// mirrors verbatim.
123+
pub target_confidence: &'static str,
120124
pub source_path: String,
121125
pub line: i64,
122126
}
@@ -152,15 +156,21 @@ pub fn insert_symbol_effects_batch(
152156
tx: &Transaction,
153157
effects: &[SymbolEffectData],
154158
) -> rusqlite::Result<()> {
159+
// event_confidence is a literal 'exact' here, not threaded from
160+
// SymbolEffectData -- see schema.rs's symbol_effects comment: every
161+
// current extraction site fires only on a real syntactic raise/throw/
162+
// write node, so the EVENT is always certain in v1. Only
163+
// target_confidence varies per-row.
155164
let mut stmt = tx.prepare(
156-
"INSERT OR IGNORE INTO symbol_effects (symbol_qn, effect_kind, target_text, source_path, line)
157-
VALUES (?1, ?2, ?3, ?4, ?5)",
165+
"INSERT OR IGNORE INTO symbol_effects (symbol_qn, effect_kind, target_text, event_confidence, target_confidence, source_path, line)
166+
VALUES (?1, ?2, ?3, 'exact', ?4, ?5, ?6)",
158167
)?;
159168
for e in effects {
160169
stmt.execute(rusqlite::params![
161170
e.symbol_qn,
162171
e.effect_kind,
163172
e.target_text,
173+
e.target_confidence,
164174
e.source_path,
165175
e.line,
166176
])?;

crates/calm-core/src/indexer/pipeline.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -844,6 +844,7 @@ fn extract_file_data(
844844
symbol_qn,
845845
effect_kind: re.effect_kind,
846846
target_text: re.target_text,
847+
target_confidence: re.target_confidence,
847848
source_path: rel.to_string(),
848849
line: re.line as i64,
849850
});

crates/calm-core/src/indexer/semantic_facts.rs

Lines changed: 87 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,14 @@ pub struct RawEffect {
6565
pub enclosing_line: usize,
6666
pub effect_kind: &'static str, // "explicit_throw" | "write_field"
6767
pub target_text: String,
68+
/// P3 (docs/plans/2026-08-08-derived-artifact-hardening-execution-plan.md):
69+
/// `"exact"` | `"none"` -- certainty about WHAT `target_text` names.
70+
/// Computed in `walk_effects` (language + `effect_kind`-aware), not
71+
/// here: only Python's `explicit_throw` detection currently has a
72+
/// textual-heuristic uncertain case (`raise e` / `raise factory()` /
73+
/// bare `raise`); every other detector is fully structural and always
74+
/// `"exact"` when it fires at all.
75+
pub target_confidence: &'static str,
6876
pub line: usize,
6977
}
7078

@@ -84,7 +92,7 @@ pub struct RawEffect {
8492
/// `derived_artifact_versions::source_extraction_fixture_is_pinned_to_its_version`
8593
/// (crates/calm-core/tests/derived_artifact_versions.rs) -- bump this AND
8694
/// that test's expected hash together, in the same commit, never one alone.
87-
pub const SOURCE_EXTRACTION_VERSION: i64 = 1;
95+
pub const SOURCE_EXTRACTION_VERSION: i64 = 2;
8896

8997
pub fn extract_type_relations_from_tree(
9098
tree: &Tree,
@@ -356,11 +364,28 @@ fn walk_effects(
356364
if let Some((enclosing_name, enclosing_line)) = &current
357365
&& let Some((kind, text)) = detect_effect(node, source, language)
358366
{
367+
// P3 (docs/plans/2026-08-08-derived-artifact-hardening-execution-plan.md):
368+
// classified here (language + kind aware), not inside each
369+
// per-language `detect_*` function -- only Python's `explicit_throw`
370+
// detection is textual-heuristic (`looks_like_exception_reference`);
371+
// Java/TS/JS throw detection requires a real constructor node
372+
// structurally, so it's always exact when it fires at all, and
373+
// `write_field`'s target (the field name) is always exact once
374+
// detected, in every language.
375+
let target_confidence = if language == "python"
376+
&& kind == "explicit_throw"
377+
&& (text.is_empty() || !looks_like_exception_reference(&text))
378+
{
379+
"none"
380+
} else {
381+
"exact"
382+
};
359383
out.push(RawEffect {
360384
enclosing_name: enclosing_name.clone(),
361385
enclosing_line: *enclosing_line,
362386
effect_kind: kind,
363387
target_text: text,
388+
target_confidence,
364389
line: node.start_position().row + 1,
365390
});
366391
}
@@ -441,17 +466,29 @@ fn detect_python_throw(node: Node, source: &str) -> Option<(&'static str, String
441466
return None;
442467
}
443468
let mut cursor = node.walk();
444-
let first = node.named_children(&mut cursor).next()?;
469+
let Some(first) = node.named_children(&mut cursor).next() else {
470+
// Bare `raise` (re-raise the currently-handled exception) -- a
471+
// real throw EVENT with structurally no target text to capture.
472+
// `walk_effects` classifies this `target_confidence: "none"`.
473+
return Some(("explicit_throw", String::new()));
474+
};
475+
// Casing is no longer a FILTER here (P3) -- `raise e`/`raise factory()`
476+
// are still real throw events, just ones whose target isn't a resolved
477+
// exception TYPE. `walk_effects` runs `looks_like_exception_reference`
478+
// on the returned text to classify `target_confidence`, so both the
479+
// confident and uncertain cases are recorded, never dropped.
445480
match first.kind() {
446481
"call" => {
447482
let func = first.child_by_field_name("function")?;
448-
let text = source[func.byte_range()].trim().to_string();
449-
looks_like_exception_reference(&text).then_some(("explicit_throw", text))
450-
}
451-
"identifier" => {
452-
let text = source[first.byte_range()].trim().to_string();
453-
looks_like_exception_reference(&text).then_some(("explicit_throw", text))
483+
Some((
484+
"explicit_throw",
485+
source[func.byte_range()].trim().to_string(),
486+
))
454487
}
488+
"identifier" => Some((
489+
"explicit_throw",
490+
source[first.byte_range()].trim().to_string(),
491+
)),
455492
_ => None,
456493
}
457494
}
@@ -569,6 +606,29 @@ mod tests {
569606
.collect()
570607
}
571608

609+
/// P3 (docs/plans/2026-08-08-derived-artifact-hardening-execution-plan.md):
610+
/// same as `effects` but also returns `target_confidence`, for the
611+
/// tests that specifically assert on the confidence split. A separate
612+
/// helper (rather than widening `effects` itself) so the other tests
613+
/// that don't care about confidence stay untouched.
614+
fn effects_with_confidence(
615+
lang: &str,
616+
src: &str,
617+
) -> Vec<(String, &'static str, String, &'static str)> {
618+
let tree = parse_tree(src, lang).expect("parse");
619+
extract_effects_from_tree(&tree, src, lang)
620+
.into_iter()
621+
.map(|e| {
622+
(
623+
e.enclosing_name,
624+
e.effect_kind,
625+
e.target_text,
626+
e.target_confidence,
627+
)
628+
})
629+
.collect()
630+
}
631+
572632
#[test]
573633
fn java_extends_and_implements() {
574634
let rs = relations(
@@ -700,34 +760,40 @@ mod tests {
700760

701761
#[test]
702762
fn python_reraise_of_bound_variable_is_not_captured() {
703-
// `raise e` -- `e` is a caught-exception variable, not an
704-
// exception TYPE reference. Capturing it would mislabel the
705-
// variable's name as if it were a resolved exception class.
706-
assert!(
707-
effects(
763+
// `raise e` -- P3: still a real throw EVENT, target uncertain.
764+
assert_eq!(
765+
effects_with_confidence(
708766
"python",
709767
"def f():\n try:\n pass\n except Exception as e:\n raise e\n"
710-
)
711-
.is_empty()
768+
),
769+
vec![("f".into(), "explicit_throw", "e".into(), "none")]
712770
);
713771
}
714772

715773
#[test]
716774
fn python_raise_of_lowercase_factory_call_is_not_captured() {
717775
// `raise factory()` -- syntactically identical to
718776
// `raise SomeException()`; only the PEP 8 casing convention lets
719-
// us tell them apart without full symbol resolution.
720-
assert!(effects("python", "def f():\n raise factory()\n").is_empty());
777+
// us tell them apart without full symbol resolution. P3: recorded
778+
// now (target_confidence=none), not dropped -- function name kept
779+
// for stability even though the assertion below changed.
780+
assert_eq!(
781+
effects_with_confidence("python", "def f():\n raise factory()\n"),
782+
vec![("f".into(), "explicit_throw", "factory".into(), "none")]
783+
);
721784
}
722785

723786
#[test]
724787
fn python_bare_reraise_is_skipped() {
725-
assert!(
726-
effects(
788+
// P3: bare `raise` is now recorded too (empty target_text,
789+
// target_confidence=none) instead of dropped -- function name kept
790+
// for stability even though the assertion below changed.
791+
assert_eq!(
792+
effects_with_confidence(
727793
"python",
728794
"def f():\n try:\n pass\n except Exception:\n raise\n"
729-
)
730-
.is_empty()
795+
),
796+
vec![("f".into(), "explicit_throw", "".into(), "none")]
731797
);
732798
}
733799

crates/calm-core/tests/derived_artifact_versions.rs

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -71,17 +71,18 @@ fn source_extraction_snapshot(conn: &Connection) -> String {
7171
.unwrap();
7272
let mut effects: Vec<String> = conn
7373
.prepare(
74-
"SELECT symbol_qn, effect_kind, target_text, confidence \
74+
"SELECT symbol_qn, effect_kind, target_text, event_confidence, target_confidence \
7575
FROM symbol_effects ORDER BY symbol_qn, effect_kind, target_text",
7676
)
7777
.unwrap()
7878
.query_map([], |r| {
7979
Ok(format!(
80-
"{}|{}|{}|{}",
80+
"{}|{}|{}|{}|{}",
8181
r.get::<_, String>(0)?,
8282
r.get::<_, String>(1)?,
8383
r.get::<_, String>(2)?,
8484
r.get::<_, String>(3)?,
85+
r.get::<_, String>(4)?,
8586
))
8687
})
8788
.unwrap()
@@ -98,7 +99,7 @@ fn source_extraction_snapshot(conn: &Connection) -> String {
9899

99100
/// Regenerate via: read the assertion failure's "actual hash" and paste it
100101
/// in below alongside a `SOURCE_EXTRACTION_VERSION` bump.
101-
const EXPECTED_SOURCE_EXTRACTION_HASH: &str = "1473f45c2586b01e";
102+
const EXPECTED_SOURCE_EXTRACTION_HASH: &str = "cd8a8816b5c84b00";
102103

103104
#[test]
104105
fn source_extraction_fixture_is_pinned_to_its_version() {

crates/calm-server/src/__toolsnaps__/locate.snap

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -347,12 +347,22 @@
347347
"line": {
348348
"type": "integer",
349349
"format": "int64"
350+
},
351+
"event_confidence": {
352+
"description": "P3 (docs/plans/2026-08-08-derived-artifact-hardening-execution-plan.md):\ncertainty THAT the effect happened -- currently always `\"exact\"`\n(every extraction site fires only on a real syntactic raise/throw/\nwrite node); kept as its own field for forward compatibility rather\nthan folded into `target_confidence`, which is a DIFFERENT question\n(certainty about WHAT the target is).",
353+
"type": "string"
354+
},
355+
"target_confidence": {
356+
"description": "`\"exact\"` | `\"none\"` -- certainty about what `target_text` names.\n`\"none\"` means the effect definitely happened but the exact target\ncouldn't be determined syntactically (e.g. Python `raise e` where\n`e` is a caught-exception variable, not a resolvable type) -- the\ntext is still the real syntax that was there, just not resolved.",
357+
"type": "string"
350358
}
351359
},
352360
"required": [
353361
"effect_kind",
354362
"target_text",
355-
"line"
363+
"line",
364+
"event_confidence",
365+
"target_confidence"
356366
]
357367
},
358368
"FileOverviewOutput": {

crates/calm-server/src/__toolsnaps__/symbol_info.snap

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -231,12 +231,22 @@
231231
"line": {
232232
"type": "integer",
233233
"format": "int64"
234+
},
235+
"event_confidence": {
236+
"description": "P3 (docs/plans/2026-08-08-derived-artifact-hardening-execution-plan.md):\ncertainty THAT the effect happened -- currently always `\"exact\"`\n(every extraction site fires only on a real syntactic raise/throw/\nwrite node); kept as its own field for forward compatibility rather\nthan folded into `target_confidence`, which is a DIFFERENT question\n(certainty about WHAT the target is).",
237+
"type": "string"
238+
},
239+
"target_confidence": {
240+
"description": "`\"exact\"` | `\"none\"` -- certainty about what `target_text` names.\n`\"none\"` means the effect definitely happened but the exact target\ncouldn't be determined syntactically (e.g. Python `raise e` where\n`e` is a caught-exception variable, not a resolvable type) -- the\ntext is still the real syntax that was there, just not resolved.",
241+
"type": "string"
234242
}
235243
},
236244
"required": [
237245
"effect_kind",
238246
"target_text",
239-
"line"
247+
"line",
248+
"event_confidence",
249+
"target_confidence"
240250
]
241251
},
242252
"Caveat": {

crates/calm-server/src/__toolsnaps__/understand.snap

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -277,12 +277,22 @@
277277
"line": {
278278
"type": "integer",
279279
"format": "int64"
280+
},
281+
"event_confidence": {
282+
"description": "P3 (docs/plans/2026-08-08-derived-artifact-hardening-execution-plan.md):\ncertainty THAT the effect happened -- currently always `\"exact\"`\n(every extraction site fires only on a real syntactic raise/throw/\nwrite node); kept as its own field for forward compatibility rather\nthan folded into `target_confidence`, which is a DIFFERENT question\n(certainty about WHAT the target is).",
283+
"type": "string"
284+
},
285+
"target_confidence": {
286+
"description": "`\"exact\"` | `\"none\"` -- certainty about what `target_text` names.\n`\"none\"` means the effect definitely happened but the exact target\ncouldn't be determined syntactically (e.g. Python `raise e` where\n`e` is a caught-exception variable, not a resolvable type) -- the\ntext is still the real syntax that was there, just not resolved.",
287+
"type": "string"
280288
}
281289
},
282290
"required": [
283291
"effect_kind",
284292
"target_text",
285-
"line"
293+
"line",
294+
"event_confidence",
295+
"target_confidence"
286296
]
287297
},
288298
"SourceOutput": {

0 commit comments

Comments
 (0)