`$populate` silently drops open-choice answers that are not in `answerOption`

[not-in-stock]

$populate silently drops open-choice answers that are not in answerOption

Describe the bug

During pre-population, @aehrc/sdc-populate drops valueCoding answers produced by an initialExpression when they do not match one of the item's answerOption codings - including on open-choice items, where FHIR allows answers outside the listed options.

What makes this hard to deal with is that nothing surfaces. The expression resolves correctly and the answer is present in the intermediate response, then it disappears before the QuestionnaireResponse is returned, no error, no warning, no entry in the OperationOutcome. An answer that was produced and then discarded looks exactly like one that was never produced, so there is nothing to triage on and each occurrence has to be traced by hand.

To Reproduce

An open-choice item that lists common diagnoses as options but has to accept any ICD-10 code recorded on the patient:

{
  "linkId": "dx-primary",
  "type": "open-choice",
  "answerOption": [
    {
      "valueCoding": {
        "system": "http://hl7.org/fhir/sid/icd-10-cm",
        "code": "E11.9",
        "display": "Type 2 diabetes mellitus without complications"
      }
    }
  ],
  "extension": [
    {
      "url": "http://hl7.org/fhir/uv/sdc/StructureDefinition/sdc-questionnaire-initialExpression",
      "valueExpression": {
        "language": "text/fhirpath",
        "expression": "%condition.code.coding.first()"
      }
    }
  ]
}

1. Populate it for a patient whose Condition.code.coding.first() is any ICD-10 coding other than E11.9, for example J44.9.
2. dx-primary comes back with no answer.

Expected behavior

dx-primary is answered with the populated coding. open-choice exists so that answers outside the listed options are allowed, and the same function already preserves non-matching valueString answers for exactly that reason - the coding path just misses the same consideration.

Plain choice items should keep filtering as they do today.

Additional context

There appear to be two separable defects behind this:

1. filterAndNormaliseAnswers (utils/processValueSets.ts) filters regardless of item type. valueCoding answers absent from the item's options - answerOption, expanded valueSet, or contained valueSet are dropped, and the item type is not available at that point, so open-choice cannot be distinguished from choice. The neighbouring valueString branch already carries the comment "to support open-choice questions where arbitrary strings are allowed".

2. findInAnswerOptions (utils/answerOption.ts) never matches Coding values. parseValueToAnswer passes raw FHIRPath results to it, but the signature is (options, str: string) and it compares with === against option codes and displays. Coding objects, anything from %condition.code.coding.first() and similar, never match, so a Coding that is among the options is not normalised to that option's coding. It survives today only because the later options filter happens to compare by code.

• Reproduced on @aehrc/sdc-populate 4.7.1.
• A fix for the first defect is behaviour-changing only for open-choice items; choice items keep their current filtering, so existing forms are unaffected.
• Found in a Da Vinci DTR prior-authorization form (CMS-0057 tooling) on an ICD-10 primary-diagnosis field: the Condition was passed correctly, the FHIRPath resolved correctly, and the answer vanished between the two.
• I have a working implementation for both defects with regression tests, and would be happy to submit a PR if this approach is acceptable.

[not-in-stock]

While reviewing the fix branch I found two more pre-existing gaps in the same population path. Both are older than the defects described in this issue and neither is fixed by the PR: the first would grow it well beyond its scope, so I'm noting them here instead.

1. Items with child items skip answer filtering entirely. filterValueSetAnswersRecursive() returns early into recursion whenever qrItem.item is non-empty. An item that carries both an answer and nested items (normal SDC nesting for a question with sub-questions) never gets its own answer checked against answerOption or valueSet options. Out-of-options codings survive even on type: 'choice', and matching codings are not normalised to the option coding. No later pass compensates: addDisplayToQuestionnaireResponseCodings and removeEmptyAnswersFromResponse filter nothing relevant. A fix would need to filter the item's own answers before recursing, which changes the recursion structure rather than the matchers.

2. Number-typed FHIRPath results never match valueInteger answerOptions. findInAnswerOptions() compares value === option.valueInteger.toString(), a string comparison, and the answer side is never converted to a string. An integer-typed FHIRPath result (as opposed to the string '5') falls through to the numeric branches in parseValueToAnswer instead of returning the matching option. The PR fixes the zero-valued variant of this ({valueInteger: 0} was skipped by a falsy guard) because that line was already being rewritten; the string-only comparison itself is untouched.