UNC Charlotte COVID-19 R19 submission

[lunarmouse]

With the requested result file and metadata.

[github-actions]

Model Metadata

--- UNCC-Hierbin.yaml ---

✅: [metadata_schema_exists]: File exists at path 'hub-config/model-metadata-schema.json'.

✅: [metadata_file_exists]: File exists at path 'model-metadata/UNCC-Hierbin.yaml'.

✅: [metadata_file_ext]: Metadata file extension is "yml" or "yaml".

✅: [metadata_file_location]: Metadata file directory name matches "model-metadata".

❌: [metadata_matches_schema]: Metadata file contents must be consistent with schema specifications.

• /license must be equal to one of the allowed values .

[github-actions]

Model Metadata

--- UNCC-Hierbin.yaml ---

✅: [metadata_schema_exists]: File exists at path 'hub-config/model-metadata-schema.json'.

✅: [metadata_file_exists]: File exists at path 'model-metadata/UNCC-Hierbin.yaml'.

✅: [metadata_file_ext]: Metadata file extension is "yml" or "yaml".

✅: [metadata_file_location]: Metadata file directory name matches "model-metadata".

✅: [metadata_matches_schema]: Metadata file contents are consistent with schema specifications.

✅: [metadata_file_name]: Metadata file name matches the model_id specified within the metadata file.

[LucieContamin]

Good morning @lunarmouse ,

Thank you for your R19 submissions!
It seems that the validation encounter some issues:

• For the metadata file: it seems that you first version included a cc-by-nc-4.0 license, I added it in the accepted list of license if you want to use it, or let me know if you need another license to be added to the list.

• For the abstract file: the content is all good, but the file needs to be moved in the model-abstract folder and needs to be named: model-abstract/UNCC-Hierbin/2025-04-27-UNCC-Hierbin.md

• For the model output file: I ran the validation manually. To summarize:

  • the filename should be updated to model-output/UNCC-Hierbin/2025-04-27-UNCC-Hierbin.gz.parquet (with a H for the model name, Hierbin)
  • the content of the file is all good, just could you please validate the pairing information and that he flat projection are expected?

To have the latest version of the information correctly used in the validation (for example the added license), please update your forked repository, before updating your files.

Please let me know if any issues or questions,
Best,
Lucie

Model Output

--- UNCC-Hierbin/2025-04-27-UNCC-hierbin.gz.parquet ---

✅: [file_exists]: File exists at path model-output/UNCC-Hierbin/2025-04-27-UNCC-hierbin.gz.parquet.

✅: [file_name]: File name "2025-04-27-UNCC-hierbin.gz.parquet" is valid.

❗: [file_location]: File directory name must match model_id
metadata in file name.
File should be submitted to directory
"UNCC-hierbin" not "UNCC-Hierbin"
✅: [round_id_valid]: round_id is valid.

✅: [file_format]: File is accepted hub format.

✅: [file_n]: Number of accepted model output files per round met.

✅: [metadata_exists]: Metadata file exists at path model-metadata/UNCC-hierbin.yaml.

---

Run validation on files: 2025-04-27-UNCC-hierbin.gz.parquet

ℹ: [pairing_info]: Run grouping pairing: location, horizon, scenario_id, target, age_group; stochastic run pairing: location, horizon, scenario_id, target, age_group. Number of Samples: 100
✅: [valid_round_id_col]: round_id_col name is valid.

✅: [unique_round_id]: round_id column "origin_date" contains a single, unique round ID value.

✅: [match_round_id]: All round_id_col "origin_date" values match submission round_id from file name.

✅: [colnames]: Column names are consistent with expected round task IDs and std column names.

✅: [col_types]: Column data types match hub schema.

✅: [valid_vals]: tbl contains valid values/value combinations.

✅: [rows_unique]: All combinations of task ID column/output_type/output_type_id values are unique.

✅: [req_vals]: Task ID/output type/output_type_id combinations all present.

✅: [value_col_valid]: Values in column value all valid with respect to modeling task config.

ℹ: [value_col_non_desc]: No quantile or cdf output types to check for non-descending values.
Check skipped.
✅: [spl_compound_taskid_set]: All samples in a model task conform to single, unique compound task ID set that matches or is
coarser than the configured compound_taksid_set.
Pairing information: location, horizon, scenario_id, target, age_group
✅: [spl_compound_tid]: Each sample compound task ID contains single, unique value.

ℹ: [spl_non_compound_tid]: Task ID combinations of non compound task id values not tested
✅: [spl_n]: Required unique number of samples per compound idx task present.
Only one number of samples from: 100 to 300 are accepted. Submission contains: 100
✅: [n_decimal]: value associated with sample is in a valid format.
A maximum of 1 decimal place is expected.
✅: [na_value]: value does not contain NA value.

❗: [flat_projection]: Some projections have a unique value for the whole projection period.
Please verify, for example: 2025-04-27, 02, sample, A-2025-04-01, inc death, 44, 65-130; 2025-04-27, 02, sample, A-2025-04-01, inc death, 65, 65-130; 2025-04-27, 02, sample, A-2025-04-01, inc death, 66, 65-130; 2025-04-27, 02, sample, A-2025-04-01, inc death, 69, 65-130; 2025-04-27, 02, sample, A-2025-04-01, inc death, 73, 65-130
✅: [cumul_proj]: The cumulative values are not decreasing.

[github-actions]

Model Metadata

--- UNCC-Hierbin.yaml ---

✅: [metadata_schema_exists]: File exists at path 'hub-config/model-metadata-schema.json'.

✅: [metadata_file_exists]: File exists at path 'model-metadata/UNCC-Hierbin.yaml'.

✅: [metadata_file_ext]: Metadata file extension is "yml" or "yaml".

✅: [metadata_file_location]: Metadata file directory name matches "model-metadata".

✅: [metadata_matches_schema]: Metadata file contents are consistent with schema specifications.

✅: [metadata_file_name]: Metadata file name matches the model_id specified within the metadata file.

[github-actions]

Model Output

--- UNCC-Hierbin/2025-04-27-UNCC-Hierbin.gz.parquet ---

✅: [file_exists]: File exists at path 'model-output/UNCC-Hierbin/2025-04-27-UNCC-Hierbin.gz.parquet'.

✅: [file_name]: File name "2025-04-27-UNCC-Hierbin.gz.parquet" is valid.

✅: [file_location]: File directory name matches model_id
metadata in file name.

✅: [round_id_valid]: round_id is valid.

✅: [file_format]: File is accepted hub format.

✅: [file_n]: Number of accepted model output files per round met.

✅: [metadata_exists]: Metadata file exists at path 'model-metadata/UNCC-Hierbin.yaml'.

---

Run validation on files: 2025-04-27-UNCC-Hierbin.gz.parquet

ℹ: [pairing_info]: Run grouping pairing: location, horizon, scenario_id, target, age_group; stochastic run pairing: location, horizon, scenario_id, target, age_group. Number of Samples: 100
✅: [valid_round_id_col]: round_id_col name is valid.

✅: [unique_round_id]: round_id column "origin_date" contains a single, unique round ID value.

✅: [match_round_id]: All round_id_col "origin_date" values match submission round_id from file name.

✅: [colnames]: Column names are consistent with expected round task IDs and std column names.

✅: [col_types]: Column data types match hub schema.

✅: [valid_vals]: tbl contains valid values/value combinations.

✅: [rows_unique]: All combinations of task ID column/output_type/output_type_id values are unique.

✅: [req_vals]: Task ID/output type/output_type_id combinations all present.

✅: [value_col_valid]: Values in column value all valid with respect to modeling task config.

ℹ: [value_col_non_desc]: No quantile or cdf output types to check for non-descending values.
Check skipped.
✅: [spl_compound_taskid_set]: All samples in a model task conform to single, unique compound task ID set that matches or is
coarser than the configured compound_taksid_set.
Pairing information: location, horizon, scenario_id, target, age_group
✅: [spl_compound_tid]: Each sample compound task ID contains single, unique value.

ℹ: [spl_non_compound_tid]: Task ID combinations of non compound task id values not tested
✅: [spl_n]: Required unique number of samples per compound idx task present.
Only one number of samples from: 100 to 300 are accepted. Submission contains: 100
✅: [n_decimal]: value associated with sample is in a valid format.
A maximum of 1 decimal place is expected.
✅: [na_value]: value does not contain NA value.

❗: [flat_projection]: Some projections have a unique value for the whole projection period.
Please verify, for example: 2025-04-27, 02, sample, A-2025-04-01, inc death, 44, 65-130; 2025-04-27, 02, sample, A-2025-04-01, inc death, 65, 65-130; 2025-04-27, 02, sample, A-2025-04-01, inc death, 66, 65-130; 2025-04-27, 02, sample, A-2025-04-01, inc death, 69, 65-130; 2025-04-27, 02, sample, A-2025-04-01, inc death, 73, 65-130
✅: [cumul_proj]: The cumulative values are not decreasing.

[LucieContamin]

Good morning @lunarmouse ,

Thank you for the update! The validation has run, however the abstract still have some issues.

• Abstract: the content is all good, but the file needs to be moved into the model-abstract folder and needs to be named like the model output file with the round id date and your team-model name, so here: model-abstract/UNCC-Hierbin/2025-04-27-UNCC-Hierbin.md

• For the model output file: the content of the file is all good, just could you please let me know if the pairing information is correct and if the "flat" projection are expected?

Please let me know if any issues or questions,
Best,
Lucie

[lunarmouse]

Thanks Lucie! I have moved the abstract to the correct folder, and renamed it. The pairing information in the main result parquet file is correct. As for the flat predictions of deaths in some states, I double checked the historical data, it could be caused by extended period of 0 deaths in these states. Therefore, the model just tried to "mimic" the pattern by generating a flat prediction as well. Since the historical and predicted death numbers are extremely low, I would not add some random noise around the predictions. Best regards, Shi

[LucieContamin]

Just for information, I updated the format of the output_type_id column to avoid issue. Please let me know if any issues, or questions.

Best, Lucie

[lunarmouse]

Thank you for your help, Lucie. Appreciate it. Have a great weekend, Shi