Explore and establish procedures and expectations for monitoring / alignment with HTAN2 data model

[aclayton555]

Orion: My primary interest here is to establish a protocol (or at least some focal points) to guide MC2-HTAN2 model alignment. I think this can primarily be done based on common data element identifiers, but it would be good to define what we actually want this to look like.

Ashley: The first few RFC are being incorporated into HTAN2 data model in late summer / fall 2025. Suggest that we pick this up in late 2025 to explore how we can monitor alignment between these two data models, which also share goals of aligning with the latest standards in use by the CRDC.

[aclayton555]

Potential consider with: #201

[aclayton555]

For reference, tentative prioritization of HTAN2 data model development and RFCs can be found in ncihtan/htan2-data-model#46

[Bankso]

Suggested approach:

MC2 should integrate HTAN2 models once an RFC has been finalized. Since the integration approach will be slightly different between HTAN2 and MC2 (LinkML vs schematic, respectively), I think we can work off of the RFC and focus on capturing Common Data Elements.

For each data type defined by an HTAN2 RFC:

• the MC2 model should have a corresponding Component, OR;

• all attributes defined in the HTAN2 RFC have an analog in the MC2 model, designated by either a CDE Public Id or HTAN2 attribute Id listed in the Properties column

• When reviewing an HTAN2 RFC for integration in the MC2 model:

  • determine if the RFC corresponds to an existing Component in the MC2 model
    • If YES, document any new attributes or mappings using the existing annotationProperty.csv in modules/{Component}
    • If NO, determine if this is a record model or a file-based model
      • If file-based, create the directory modules/{data type}, copy an existing annotationProperty.csv, and use this to document any new attributes or mappings
        • Define the new data model by listing the relevant attributes in column DependsOn
      • If record-based, determine if the attributes can be integrated into an existing Component
        • If YES, document any new attributes or mappings using the existing annotationProperty.csv in modules/{Component}
        • If NO, create the directory modules/{data type}, copy an existing annotationProperty.csv, and use this to document any new attributes or mappings

• Attribute matching approach

  • If a matching attribute already exists in the MC2 model, using the MC2 attribute is preferred
    • If no CDE is listed for the MC2 attribute, assign the HTAN2 CDE in column Properties as CDE:{cde public id}
    • If the HTAN2 attribute does not have a CDE Public Id, assign a mapping to the HTAN2 attribute as HTN:{attribute id}
  • If no matching or similar attribute exists in the MC2 model, add the HTAN2 attribute to the selected annotationProperty.csv and incorporate it into the relevant Component by listing it alongside other attributes in column DependsOn
  • If updating an existing model, address any table schema mismatches by replacing table schemas or regenerating tables
  • If we find MC2 experimental metadata attributes without a clear match to a CDE or HTAN2 attribute, we should consider removing them