Optimization debugging

[tylerdy]

Update 9/8 - changes in the last couple weeks:

• Age group 80 to 84, sex F, still US
• Fix equilibrium % diff checks
  • 25 to 35% equilibrium differences
• Use % diff instead of raw diff for error functions
  • Severe % difference really high, others 0%
• Set emr_mild initial guess = 0
  • emrs and csmrs not increasing
  • Theres a divide by zero at some point during optimization for percent diff calcc
• Added some tolerance (within 10^-5 returns 0% diff)
  • This failed
• Added constraints so that emrs and csmrs were increasing
  • State equilibriums are even now! Wow
  • Predicted Mod and sev mortalities are lower than lit by .14 and .28 resp. (raw)
  • Predicted Emr weighted mortality is .08, gbd oveall is .037
• Removed csmr stage variables since csmr = emr * prevalence
  • Stage mortalities are very similar to one another
  • Sev equilibrium is off by 66% while the others are even
• Better initial guess
  • Emr_mild = 0, emr_mod = emr and emr_sev = emr*2 did not work
• Emr_mild near 0 error function
  • Fails even at .01, minimum seems to be .02
• Changing weighting (5x on severe equil.) to have higher mortality at more severe stages, and better match severe equilibrium
  • Worse match on literature mortality, and GBD mortality
• Prioritized GBD mortality higher than lit mortality
  • Seems better, but mild and mod are similar mortality
• Instead of matching lit mortality, just match ratios between lit mortalities (mild to mod, mod to sev)
  • This looks pretty good I think!
• Tried removing the 5 * severe equilibrium error
  • Back to being off by 60% for severe equilibrium

---

8/15: Work off Syl's branch trying to debug optimization.

Steps I've taken since Syl's work:

• Follow the "next steps" listed beginining with fixing function errors
• Add constraints to keep predicted mortality positive
• Removed the multiplication by 1000 bc it allowed my contraints to give the semi-reasonable outputs which are currently shown in the notebook (equilibrium errors of 5%, 30% and 59%) and because the way the functions are written I couldn't be sure the scaling by 1000 was working correctly
• Experiment a bit with age groups (tried to use 60+ as the GDB age group which resulted in the optimization failing due to impossible constraints, eventually reverted from 70-74 I was using in "next steps", back to all age groups)
• Experimented a bit with re-removing error functions 4-6 (resulted in the optimization being solved trivially using the initial guesses)

Current state as mentioned in third bullet above might suggest that the optimization is working but that there is significant disagreement bt input values. (Hard to be sure its working properly though given that it breaks easily when I change things like age groups or the 1000x scaling).

My current theory is that it is working ok considering the value disagreements, but we have too many error functions which is putting too much weight on making those values agree compared to the equilibrium functions, which I'm assuming are quite important to be balanced for the sake of population stability. As mentioned below maybe it would help to do some interpretation as to what proper inputs should be including the disagreeing CSMRs and GBD age group, and to make the error function be more dominated by the equilibrium functions rather than forcing inherently different values to agree, if possible.

Next steps I want to ty:

• Based on Syl's suggestion at check-in on Thurs 8/14, due to disagreement bt GDB values, remove the error function (7) that makes csmr from gbd match individual sequelae values
• Decide on proper GBD age group to use to best match the study environment that the literature values are from as well as our simulation environment, and make it work without the optimization failing
• Interpret why values disagree and resolve it by making an assumption on correct values (and removing more non equilibrium error functions)
• Redefine the non-parameter, fixed values used by the error function (prev_sev, prev_mod, prev_mild alz_prev, acmr, csmr, and emr) to be passed as args rather than global variables to make it easier to debug/ be sure that scaling things by 1000 is working
• And/or, is there a way to tune the optimization function that would have a similar effect as multiplying by 1000 without actually having to do that?
• Double check math on non-equilibrium error functions? Mostly for my own understanding

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB