Skip to content

Commit 64c19e7

Browse files
TheTomclaude
andcommitted
fix(refract): null skipped axis score in JSON + 'skip' in compare table
Audit follow-up to the earlier skip-axis fix (728cd84). The stub axis result dataclasses still carry score=100, so without these changes: - json_report still wrote 'axes.gtm.score: 100.0' for a skipped axis, visible to anyone parsing the JSON or running 'refract compare' - 'refract compare' showed 100.0 in the Traj/KLD column for skipped reports, which made cross-report comparison falsely flatter the skipped run Now: when CompositeScore.gtm_score is None we write axes.gtm.score = null in the JSON. 'refract compare' reads 'skipped' or score=None and prints 'skip' in that column instead of the stub 100. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-Authored-By: tturney1@gmail.com <tturney1@gmail.com>
1 parent 728cd84 commit 64c19e7

2 files changed

Lines changed: 13 additions & 2 deletions

File tree

refract/cli.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -808,7 +808,10 @@ def _run_compare(args) -> int:
808808
a = r["axes"]
809809
def fmt(d, k):
810810
try:
811-
return f"{a[k]['score']:.2f}"
811+
ax = a[k]
812+
if ax.get("skipped") or ax.get("score") is None:
813+
return "skip"
814+
return f"{ax['score']:.2f}"
812815
except Exception:
813816
return "—"
814817
comp_val = r["composite"]

refract/report.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -338,6 +338,14 @@ def json_report(
338338
composite_band = composite_dict.pop("band")
339339
def _band_or_skipped(s):
340340
return band(s) if s is not None else "skipped"
341+
# When an axis was skipped, the underlying result dataclass still has a
342+
# stub score=100. Null out the JSON 'score' field so downstream readers
343+
# (compare, leaderboards, paper tables) don't pick up the stub as real.
344+
if composite.gtm_score is None:
345+
gtm_dict["score"] = None
346+
kld_dict = asdict(kld)
347+
if composite.kld_score is None:
348+
kld_dict["score"] = None
341349
axes_block: dict = {
342350
"gtm": {
343351
**gtm_dict,
@@ -346,7 +354,7 @@ def _band_or_skipped(s):
346354
"description": _AXIS_PROSE["gtm"],
347355
},
348356
"kld": {
349-
**asdict(kld),
357+
**kld_dict,
350358
"band": _band_or_skipped(composite.kld_score),
351359
"skipped": composite.kld_score is None,
352360
"description": _AXIS_PROSE["kld"],

0 commit comments

Comments
 (0)