Problem
The current classifier emits one dominant a_class through a priority-ordered decision tree and assigns fixed scores such as 0.70, 0.85, and 0.95 under the field name confidence. These values are explicitly heuristic, not posterior probabilities, but the numeric presentation and tier thresholds can be misread as calibrated certainty.
Priority branches also shadow simultaneous mechanisms: a system may show Mpemba overlap, symmetrised-gap reduction, skin/transient amplification and phantom/pseudospectral evidence at once, while only one class/family is returned. Reserved A6/A7/A9 classes further make the taxonomy only partially reachable.
claim_status: pending; this issue is an architecture/calibration roadmap, not authorization for an immediate classifier rewrite.
Correct target
Preserve a summary, add a hypothesis-wise evidence matrix
For each A1..A12 and F1..F5, record separately:
- supporting measurements;
- counterevidence/vetoes;
- missing required evidence;
- numerical uncertainty;
- robustness under admissible perturbations and rate rescaling;
- claim floor (
UNDEFINED, candidate, confirmed) derived from explicit rules.
A dominant class may remain as a convenience projection, but it must not erase concurrent supported mechanisms.
Rename or calibrate confidence
Choose one:
- Rename the current value to an ordinal
support_score with documented non-probabilistic semantics; or
- Calibrate it against held-out labelled reference families and report reliability curves / uncertainty intervals.
Do not retain probability-like numbers without calibration evidence.
Validation design
- group splits by physical model family and system-size family to prevent leakage;
- calibration set, locked holdout set and adversarial negatives;
- class-conditional precision/recall, abstention rate and uncertainty intervals;
- threshold sensitivity and bootstrap confidence intervals;
- explicit cost matrix for false scientific confirmation versus false abstention;
- no synthetic data point from the same parameter sweep in both calibration and holdout;
- assess branch shadowing by reporting all triggered hypotheses before priority resolution.
Reachability and ontology gate
For A6/A7/A9 either:
- define code-backed necessary evidence and reference systems, or
- mark them as schema-reserved and exclude them from claims/coverage denominators.
Release policy
Until calibration exists, documentation and reports should describe confidence as a deterministic heuristic support score, not a probability or empirical confidence level. Publication-grade tiers require positive certificates and independent review, not merely crossing a fixed numeric threshold.
Problem
The current classifier emits one dominant
a_classthrough a priority-ordered decision tree and assigns fixed scores such as0.70,0.85, and0.95under the field nameconfidence. These values are explicitly heuristic, not posterior probabilities, but the numeric presentation and tier thresholds can be misread as calibrated certainty.Priority branches also shadow simultaneous mechanisms: a system may show Mpemba overlap, symmetrised-gap reduction, skin/transient amplification and phantom/pseudospectral evidence at once, while only one class/family is returned. Reserved A6/A7/A9 classes further make the taxonomy only partially reachable.
claim_status: pending; this issue is an architecture/calibration roadmap, not authorization for an immediate classifier rewrite.Correct target
Preserve a summary, add a hypothesis-wise evidence matrix
For each
A1..A12andF1..F5, record separately:UNDEFINED, candidate, confirmed) derived from explicit rules.A dominant class may remain as a convenience projection, but it must not erase concurrent supported mechanisms.
Rename or calibrate
confidenceChoose one:
support_scorewith documented non-probabilistic semantics; orDo not retain probability-like numbers without calibration evidence.
Validation design
Reachability and ontology gate
For A6/A7/A9 either:
Release policy
Until calibration exists, documentation and reports should describe
confidenceas a deterministic heuristic support score, not a probability or empirical confidence level. Publication-grade tiers require positive certificates and independent review, not merely crossing a fixed numeric threshold.