Context
Follow-up from review of #154 (rubric-review branch).
Problem
#154 added conditional applicability ("Always report results… but only score if identified elsewhere") only to d4d-rubric20-semantic.md. d4d-rubric10-semantic.md was left with strict binary present/absent scoring.
This asymmetry directly hurts D4D-core evaluation. D4D_Core.yaml intentionally excludes fields:
"Fields not in the exchange layer mapping (e.g., file_collections, relationships, splits, consent workflow) are intentionally excluded; use the full data_sheets_schema.yaml for those."
So rubric10-semantic will penalize claudecode_agent_core datasheets (the new *_core_evaluation.json runs) for absences that are by design — while rubric20-semantic now degrades gracefully for the same datasheets.
Proposed fix
Either:
- (a) Add the same conditional-applicability treatment to rubric10-semantic sub-elements (report but only score when the relevant context is present elsewhere in the datasheet), or
- (b) Add an explicit "absent-by-design ≠ fail when evaluating a D4D-core datasheet" stance, with a documented list of core-excluded fields that should not be penalized.
Depends on / coordinates with the N/A-encoding convention (companion issue) so rubric10 scores stay comparable.
Scope
.claude/agents/d4d-rubric10-semantic.md
Context
Follow-up from review of #154 (
rubric-reviewbranch).Problem
#154 added conditional applicability ("Always report results… but only score if identified elsewhere") only to
d4d-rubric20-semantic.md.d4d-rubric10-semantic.mdwas left with strict binary present/absent scoring.This asymmetry directly hurts D4D-core evaluation.
D4D_Core.yamlintentionally excludes fields:So rubric10-semantic will penalize
claudecode_agent_coredatasheets (the new*_core_evaluation.jsonruns) for absences that are by design — while rubric20-semantic now degrades gracefully for the same datasheets.Proposed fix
Either:
Depends on / coordinates with the N/A-encoding convention (companion issue) so rubric10 scores stay comparable.
Scope
.claude/agents/d4d-rubric10-semantic.md