Field naming: ADR + non-technical guide + enforcement (source-standard names vs LIF/tech constraints)

[bjagg]

Describe the issue

LIF ingests field names from external standards and source systems (CEDS, IMS, …) into the MDR, and those names don't always satisfy LIF's naming convention or the constraints of the technologies that consume the model. The iSO639-2LangCode incident (#1011) is a concrete failure: a hyphenated, non-camelCase CEDS attribute reached the MDR and crashed GraphQL schema generation in dev and demo.

Three pressures conflict, and there's no recorded decision reconciling them:

• Source standards deliver names like iSO639-2LangCode (hyphens, irregular caps).
• LIF convention (docs/specs/data-model-rules.md → Naming Styles): PascalCase entities, camelCase scalar leaves, PascalCase enums.
• Technology constraints: GraphQL names must match /[_A-Za-z][_0-9A-Za-z]*/ (no hyphen, no leading digit); Python attrs are snake_cased; JSON/Mongo store verbatim; the translator relies on case-insensitive lookups.

data-model-rules.md documents the rule but not the conflict or a decision, and explicitly notes the convention is "enforced by readers, not tooling" — which is how the bad names slipped in. (Analogous to the classic Clojure kebab-case vs JSON/DynamoDB tension.)

Deliverables

1. ADR (docs/design/adr/data_model/0001-…) capturing the conflict, options, and a ratified decision (e.g. normalize-on-ingest + preserve source name as metadata, and/or reject-illegal-at-MDR-write-time). Draft PR incoming (status: Proposed) for the team to react to.
2. Non-technical naming guide for schema designers / CEDS contributors — plain-language "name fields like this, not that, and why."
3. Enforcement — lint for the checked-in convention files and/or MDR write-time validation, so the convention can't silently drift again. (Relates to the unenforced-convention gap and the uniqueness work in Ensure the MDR API enforces unique names of _anything_ that can be exportable (schemas or mappings) #746.)

Related: #1011 / #1012 (crash + codegen hardening — defense-in-depth, not a substitute for a naming policy), #1013 (rename the three iSO639-* fields).

[bjagg]

Audit evidence — the permissive-input problem is not GraphQL-specific

Scouted the codebase for other places where permissive, source-defined input collides with a strict component (the iSO639-2LangCode crash was one instance). Framing it by the two entry points:

Entry point 1 — MDR (schema & mapping definitions) → the MDR can warn/restrict

The same illegal name independently threatens at least five consumers, each sanitizing differently or not at all:

#	Component	Trigger	Severity	Status
1	GraphQL codegen	invalid char / leading digit / leading __ / post-sanitize collision	startup crash	hardened in #1012 (incl. __/collision edge cases)
2	Semantic-search / MCP	raw schema name → invalid Pydantic field	startup crash, uncovered	#1016 (verified; parallel to #1011)
3	MongoDB writes	name starts with $ or contains .	write rejected	covered once boundary-validated
4	Composer / fragment paths	name contains . (paths split on .)	silent mis-nesting	covered once boundary-validated
5	Translator / JSONata	unquoted name with hyphen/space parses as an operator	transform fails	covered once boundary-validated

These are verified in code for #1 and #2; #3–#5 are code-pointed and strongly plausible (worth confirming during implementation). The takeaway: each consumer has its own (or no) sanitizer, and they drift — which is the core argument for validating once at the MDR write boundary (the ADR's recommended decision) rather than N inconsistent downstream guards.

Mapping/transformation definitions belong here too: the MDR should reject a mapping whose JSONata is invalid, or that references unknown/ill-named elements.

Entry point 2 — data (learner records) → validate against the assigned schema + sanity checks

We can't dictate source data shape, but the system shouldn't silently corrupt it:

• GraphQL resolver: no int/bool/float coercion — typed fields silently null (or mismatch) for string source data #1017 (verified): a value typed boolean/integer/number/date in the schema but delivered as a string ("Yes", "N/A", "50", "2024") isn't coerced by the GraphQL resolver — it's silently turned into null (the code even carries a TODO: should really be more strict). Lever: schema-conformance validation + sanity checks at ingestion, and a deliberate policy for non-conforming values (preserve-as-string vs. null-with-signal) rather than silent loss.

What's already in motion

• Issue #1011: harden GraphQL schema build against invalid source names #1012 — GraphQL hardening + the __/collision/reserved edge cases (follow-up pushed).
• docs: ADR 0001 (Proposed) — field naming & source-standard normalization #1015 — ADR 0001 (Proposed) updated with this two-entry-points framing + the audit evidence.
• Semantic-search/MCP: dynamic filter & mutation models crash on invalid source field names (parallel to #1011) #1016 / GraphQL resolver: no int/bool/float coercion — typed fields silently null (or mismatch) for string source data #1017 — discrete issues for the semantic-search crash and the resolver coercion gap.
• Rename invalid CEDS language-code fields (iSO639-2/3/5LangCode) to valid LIF names #1013 — the concrete data rename (first application of the boundary policy).

Recommended enforcement deliverables for this issue: (1) MDR write-time name/structure validation (rejects or normalizes), (2) MDR mapping/JSONata validation, (3) lint for the checked-in convention files, (4) the non-technical naming guide.

[bjagg]

Verification of the code-pointed findings (resolving the earlier "confirm during implementation" caveat)

Read the actual code paths:

• MongoDB — CONFIRMED. build_mongo_update_ops (query_cache_service/core.py:96-110) builds keys as f"{root_prefix}.{k}" from field names, and insert_one(lif_record.model_dump(by_alias=True)) uses field names as keys. A $-leading name is rejected by Mongo; a .-containing name is misread as nesting → wrong/failed write.
• Composer dot-path — CONFIRMED. add_fragment_to_lif_record (composer/core.py:60) does keys = dot_map_path.split(".") and navigates; a field name containing . splits into spurious segments, hits the "unexpected type" branch, and the fragment is silently dropped.
• JSONata — RECLASSIFIED (was overstated). There is no code that auto-builds a JSONata expression from field names — expressions are author-supplied (Transformation.Expression, evaluated at translator/core.py:36). So this is an authoring hazard, not an automatic code bug: a human-written mapping referencing an unquoted hyphenated/spaced name mis-parses. Right mitigation = MDR validates the expression compiles (and references known elements) on write — folded into the enforcement scope and the ADR Decision.

Net: of the five name-consumers, GraphQL (#1011/#1012) and semantic-search (#1016) are verified startup crashes, MongoDB and Composer are verified write/compose corruption, and JSONata is a verified authoring hazard (→ mapping validation). All five collapse to the same fix: validate/normalize at the MDR write boundary. ADR 0001 (#1015) updated to mark these verified and to add mapping-expression validation to the Decision.

(I did not open separate issues for the Mongo/Composer cases — they're subsumed by the boundary-validation work here. Say the word if you'd prefer discrete tickets.)

[bjagg]

Opened discrete tickets for the two verified consumer-side failures: #1018 (MongoDB $/. key constraint) and #1019 (Composer dot-path silently drops fragments). Both remain subsumed by the MDR write-time validation here, but are now individually trackable / fixable as defense-in-depth. ADR 0001 references updated.