Align schemas with those provided by PDBP

[philerooski]

All fields and their values will need to be mapped from the clinical data and the fox insight survey data to the fields and values provided by PDBP.

1. Create separate mapping files of fields and values for both clinical and fox insight surveys.

• The clinical mapping file will be a JSON file with the following structure:

{
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "target_field": {
                "type": "object",
                "properties": {
                    "source_fields": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "source_field": {
                                    "type": "object",
                                    "properties": {
                                        "name": {
                                            "type": "string"
                                        },
                                        "cohort": {
                                            "type": "string"
                                        }
                                    }
                                }
                            }
                        }
                    },
                    "source_values": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "source_value": {
                                    "type": "object",
                                    "properties": {
                                        "name": {
                                            "type": "string"
                                        },
                                        "target_value": {
                                            "type": "string"
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

That is, a target (PDBP) field will have a name and map to multiple source (clinical) fields. Each source field will have a name, a cohort, and one or more visits at which the field was collected. The cohort will not only be unique to the source field name, but will also identify which GUIDs we expect to have provided this source/target field information. The visits field will be an array of pre-defined strings which identify which visits are expected to provide that value. Each visit corresponds to a distinct record in the target schema. Although the source field names will vary depending on the cohort we are mapping from, the source fields will share the same value mapping. Each value mapping is a JSON object with a name (the literal of the source value) and target value (the literal of the target value). In summary, we map a target field to its potential source fields, then for each potential source field we map its values to the target values.

This looks like this as a list in R:

[[1]]
[[1]]$name
[1] "target_field"

[[1]]$source_fields
[[1]]$source_fields[[1]]
[[1]]$source_fields[[1]]$name
[1] "source_field_one"

[[1]]$source_fields[[1]]$cohort
[1] "at-home-pd"


[[1]]$source_fields[[2]]
[[1]]$source_fields[[2]]$name
[1] "source_field_two"

[[1]]$source_fields[[2]]$cohort
[1] "super-pd"


[[1]]$source_values
[[1]]$source_values[[1]]
[[1]]$source_values[[1]]$name
[1] "source_value_one"

[[1]]$source_values[[1]]$target_value
[1] "target_value_one"


[[1]]$source_values[[2]]
[[1]]$source_values[[2]]$name
[1] "source_value_two"

[[1]]$source_values[[2]]$target_value
[1] "target_value_two"

• The Fox Insight mapping file will be a JSON file with a simpler structure:

{
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "target_field": {
                "type": "object",
                "properties": {
                    "name": {
                        "type": "string"
                    },
                    "source_field": {
                        "type": "string"
                    },
                    "source_values": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "source_value": {
                                    "type": "object",
                                    "properties": {
                                        "name": {
                                            "type": "string"
                                        },
                                        "target_value": {
                                            "type": "string"
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

In this case, each target field will map to a single source field. That source field will have a name and a variable number of possible source values, each of which maps to a single target value.

This looks like this as a list in R:

[[1]]
[[1]]$name
[1] "target_field_one"

[[1]]$source_field
[1] "source_field"

[[1]]$source_values
[[1]]$source_values[[1]]
[[1]]$source_values[[1]]$name
[1] "source_value_one"

[[1]]$source_values[[1]]$target_value
[1] "target_value_one"


[[1]]$source_values[[2]]
[[1]]$source_values[[2]]$name
[1] "source_value_two"

[[1]]$source_values[[2]]$target_value
[1] "target_value_two"

2. Implement the logic for reading in clinical and fox surveys as well as their mappings, then produce a PDBP compliant dataset by mapping the appropriate fields to their new names and values.

[philerooski]

The field mapping can be parsed from https://www.synapse.org/#!Synapse:syn23593083

TODO: modify the file above so that there are separate columns for AHPD cohort fields, SUPER cohort fields, and Fox fields

Target fields are found in column dmr_variable
Source fields are found in columns ctcc_variable and notes

[philerooski]

The clinical data is not well organized and there is at least one special case that needs to be handled separately.

SUPER-PD participants underwent a physician administered MDS-UPDRS exam twice -- once while on medication and once while off. The datetime of these events come from fields time_mdsupdrs and time_mdsupdrs_off, respectively. The scores come from fields mdsupdrsgaitscore_spd and mdsupdrsgaitscore_2, respectively. But both events are stored under the same redcap_event_name = "SUPER-PD Data (Arm 2: Sub-study)".

This is problematic because the target fields (PDBP fields) could map to either the ON fields or the OFF fields, even given the cohort and redcap_event_name (the source field used to group records from the same visit).

To handle this case, we will introduce another field visits which will identify the visits that this source field is identified with. For example, the target field MDSUPDRSGaitScore will map to mdsupdrsgaitscore_spd for visit on-med and mdsupdrsgaitscore_2 for visit off-med. All on-med visits will consist of a single record distinct from those from the off-med visit.

This solution will require yet another mapping file which maps each visit in visits to redcap_event_names in the source data. Normally, the redcap_event_name is sufficient to delineate records from different visits, but since this dataset has multiple fields belonging to the same redcap_event_name but which ultimately correspond to different visits, this relationship needs to be made more explicit.

We will handle this as a special case of a single form (mdsupdrs_physician_exam) that should have its fields split across two different visits -- one visit for the ON medication exam, and another for the OFF medication exam. Because we think of this information as coming from two different visits, it will naturally be disjoint by record.

[philerooski]

Universal Fields

The following fields are required for each PDBP form, but the field from which they are derived can be dependent on the form.

SiteName

The site name is always AT-HOME-PD_University of Rochester (as per Andrea Lutz)

AgeVal

This is the age in months and is conditional on the visit date (see VisitDate below) and the participant date of birth. AT-HOME PD participants have date of birth recorded under dob whereas SUPER-PD participants have their date of birth censored but their age is recorded as a decimal and can be reverse engineered into an approximate date of birth using demo_age in conjunction with visitdate. Reverse engineering is necessary in this case to later on populate this field for SUPER-PD participants MDS-UPDRS physician exam records. We will dynamically produce an object during runtime mapping guid to date of birth, rather than storing this information on Synapse.

GUID

This is the GUID of the participant

VisitTypPDBP

This is one of Baseline, 12 months, or 24 months. SUPER-PD data is always Baseline.

VisitDate

The clinical field from which this data comes from depends on the clinical form where that field originates. So we will need an additional mapping which maps clinical forms to clinical datetime fields. The clinical form is likewise conditional on the cohort and visit (for example, AT-HOME PD participants complete the mdsupdrs form whereas SUPER-PD participants complete the substudy_mdsupdrs_part_iii, participant_mdsupdrs_survey, or mdsupdrs_physician_exam) and each of these forms potentially derives their VisitDate from a different field. So we will need a way of determining which clinical form our PDBP form data is coming from for this specific record. To do this, we will create yet another mapping file mapping PDBP forms to cohorts, and within each cohort we will map each clinical form to the redcap_event_name(s) which it applies to. Formatted like this:

{
    "type": "object",
    "properties": {
        "target_form": {
            "type": "object",
            "properties": {
                "name": {
                    "type": "string"
                },
                "cohort": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "cohort": {
                                "type": "object",
                                "properties": {
                                    "name": {
                                        "type": "string"
                                    },
                                    "source_form": {
                                        "type": "array",
                                        "items": {
                                            "type": "string"
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

Additionally, this will require special rules for the form mdsupdrs_physician_exam because this form should have been organized as two separate visits, one for the ON exam, and another for the OFF exam.

AgeYrs

This is derived from AgeVal (above).

AgeRemaindrMonths

This is also derived from AgeVal.

[philerooski]

PDBP Form Population Logic

Psuedo-code describing how we will use the mapping files to populate PDBP forms with clinical data:

for each PDBP form: // our top level output
  for each GUID: // our top level, record-wise clinical entity
    for each visit: // repeat longitudinal measurements (redcap_event_name)
      populate_universal_fields() // fill in universal fields according to logic outlined in this issue
      for each form-specific PDBP form field:
        if cohort == "at-home-pd":
          populate_conditional_on_cohort_and_visit()
        else if cohort == "super-pd" and PDBP form == "mdsupdrs_physician_exam":
          // produce two records
          populate_conditional_on_off_meds()
        else if cohort == "super-pd":
          populate_conditional_on_cohort_and_visit()

def populate_conditional_on_cohort_and_visit():
  // reference file mapping PDBP fields, visits, and cohorts to clinical fields
  // TODO

def populate_conditional_on_off_meds():
 // TODO

[philerooski]

Confusingly, the time_mdsupdrs_off field corresponds to when SUPER-PD participants had their ON medication exam, whereas time_mdsupdrs field corresponds to when participants had their OFF medication exam.