Merge pull request #1910 from ihmeuw/no_19

Run request for model 19.0 nutrition optimization / Ethiopian targeted SQ-LNS reruns

Author: alibow

docs/source/models/concept_models/vivarium_nutrition_optimization/kids/concept_model.rst

@@ -1526,7 +1526,7 @@ Wave II
       * `MMS effect size update and implementation confirmed to be functioning in interactive sim <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/blob/data_prep/verification_and_validation/child_model/model_12.1.1_interactive_MMS_effect.ipynb>`_
   * - 12.2
     - Check that results for this run approximate the mean of the results from run 12.1
-    - 
+    - [TODO: CONFIRM V&V CONCLUSIONS FROM THIS RUN]
 
 .. list-table:: Outstanding V&V issues
   :header-rows: 1
@@ -1625,6 +1625,19 @@ Wave III
     - All
     - National runs
     - Standard SQ-LNS effects only
+  * - ``vivarium_v4.0_vph_v5.0_update``
+    - Runs that integrate vivarium and vivarium_public_health framework updates to confirm we still meet V&V criteria following these updates
+    - Baseline, MMS (run using maternal simulation outputs from the ``vivarium_v4.0_vph_v5.0_update`` model version)
+    - Baseline, 8 (all)
+    - National runs (informed from subnational artifact). 10 draws (not the mean draw). Nigeria and Pakistan only. 
+    - Standard SQ-LNS effects only
+  * - 19.0
+    - Production runs for re-run of model 17 for Ethiopia to resolve subnational scrambling issue
+    - Same as model 17.0
+    - Same as model 17.0
+    - Same as model 17.0
+    - This run is to resolve an issue for Ethiopian results in model 17.0 in which data were scrambled across subnational locations (`see details here <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/blob/fea37d5913dca2a65c814decbe3a457d383913b6/emulator/sqlns_targeting/artifact_data_checks.ipynb>`__). New custom data was generated based on an updated Ethiopian artifact without this issue (`see PR here <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/pull/207>`__ and `here <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/pull/212>`__) and updated in the simulation repo (`see PR here <https://github.com/ihmeuw/vivarium_gates_nutrition_optimization_child/pull/197>`__ and `here <https://github.com/ihmeuw/vivarium_gates_nutrition_optimization_child/pull/198>`__). Notably, these runs will be performed in 2026 almost two years after model 17.0 and the exact environments used to run model 19.0 is not expected to exactly match that used to run model 17.0. This model will include Standard and modified SQ-LNS effects.
+
 
 
 .. list-table:: Output specifications
@@ -1723,13 +1736,21 @@ Wave III
       5. Stunting state person time stratified by SQ-LNS coverage
     - * Age group
       * Subnational location (need to determine how this will be "stratified")
-  * - 18.0
+  * - 18.0, ``vivarium_v4.0_vph_v5.0_update``
+    - 1. Deaths and YLLs 
+      2. YLDs 
+      3. Count of incident SAM cases stratified by SAM treatment coverage
+      4. Count of incident MAM cases stratified by MAM treatment coverage
+      5. Stunting state person time stratified by SQ-LNS coverage
+    - Age strata of 0-6 months, 6-18 months, 18-60 months
+  * - 19.0
     - 1. Deaths and YLLs (non-cause-specific)
       2. YLDs (all-cause observer only)
       3. Count of incident SAM cases stratified by SAM treatment coverage
       4. Count of incident MAM cases stratified by MAM treatment coverage
       5. Stunting state person time stratified by SQ-LNS coverage
-    - Age strata of 0-6 months, 6-18 months, 18-60 months
+    - * Age strata of 0-6 months, 6-18 months, 18-60 months
+      * Results specific to subnational location (in the same manner as model 17.0)
 
 
 .. list-table:: Verification and validation tracking
@@ -1798,7 +1819,18 @@ Wave III
   * - 17.0
     - * Confirm that results for production runs match expected outputs
     - * As in model 16, we checked the SQ-LNS effects to ensure these matched for both standard, and modifed effects. In general, all subnational locations were within the confidence intervals. The cases where this is untrue and the rationale for approval are noted in the Jupyter notebook here. `Model 17 SQLNS effects <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/blob/0858f98bd9e19f2621873c83a20efef055a0a5d8/verification_and_validation/child_model/model_17.0_sqlns_effects.ipynb>`_. 
+      * An issue with artifact data used for subnational locations in Ethiopia was identified that necessitates a rerun (`see details here <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/blob/fea37d5913dca2a65c814decbe3a457d383913b6/emulator/sqlns_targeting/artifact_data_checks.ipynb>`__)
+      * An issue with the application of modified SQ-LNS effects was identified and `summarized in this PR <https://github.com/ihmeuw/vivarium_research/pull/1902>`__, but deemed an acceptable limitation.
   * - 18.0
     - * Confirm that results for production runs match expected outputs
+    - * An issue with artifact data used for subnational locations in Ethiopia was identified that necessitates a rerun (`see details here <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/blob/fea37d5913dca2a65c814decbe3a457d383913b6/emulator/sqlns_targeting/artifact_data_checks.ipynb>`__)
+      * Potential issue with mean draw systematically overestimating baseline parameters is expected (TODO: update with notebook demonstrating this issue)
+      * TODO: update with links to general V&V notebooks for this run
+  * - ``viv4.0_vph_v5.0_update``
+    - * Confirm baseline scenario cause burden and CGF risk exposures meets GBD expectation
+      * Compare total scenario-specifc (maternal MMS, child scenario 8) DALYs to results from model 18.0
+    - 
+  * - 19.0
+    - Re-run model 17.0 V&V for updated Ethiopian locations
     - 
 

docs/source/models/concept_models/vivarium_nutrition_optimization/pregnancies/concept_model.rst

@@ -607,7 +607,13 @@ Specific outputs for specific models are specified in the following section.
     - Same as 9.2 but including count of live births and still births 
     - Same as 9.2
     - 
-
+  * - ``vivarium_v4.0_vph_v5.0_update``
+    - Runs that integrate vivarium and vivarium_public_health framework updates to confirm we still meet V&V criteria following these updates
+    - All
+    - 10 draws, 10 seeds with 20,000 population (200,000 pregnancies per draw). Nigeria and Pakistan only
+    - Default
+    - Default
+    - 
 
 .. note::
 
@@ -737,6 +743,13 @@ Specific outputs for specific models are specified in the following section.
     - `V&V notebooks for model 11.2 can be found here <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/pull/146>`_
       * Maternal disorders YLDs now look as expected -- ready to move on to production runs
       * Also confirmed that the birth outcome observer is capturing outcomes among all simulated individuals, including those who die during the simulation
+  * - ``vivarium_v4.0_vph_v5.0_update``
+    - All V&V checks
+    - All models are meeting V&V criteria. Notebooks linked below have been updated to support the results management system format as of March 2026, but currently only support a single location at once.
+
+      * `Intervention coverage notebook <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/blob/data_prep/verification_and_validation/pregnancy_model/model_viv4.0_intervention_coverage.ipynb>`__
+      * `Maternal disorders and anemia burden notebook <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/blob/data_prep/verification_and_validation/pregnancy_model/model_viv4.0_maternal_disorders_anemia.ipynb>`__
+      * `Pregnancy model notebook <https://github.com/ihmeuw/vivarium_research_nutrition_optimization/blob/data_prep/verification_and_validation/pregnancy_model/model_viv4.0_preg_states.ipynb>`__
 
 .. list-table:: Outstanding V&V issues
   :header-rows: 1