Conversation
- Add "Calibrating Consistent Rates" section describing the NumPyro/JAX Bayesian calibration approach from consistent_rates.py - Update state data tables to use calibrated conditional prevalences (δ_BBBM, δ_MCI) instead of duration-based ratios - Update GBD references from 2021 to 2023 - Simplify Vivarium Modeling Strategy section, removing Model 4/5 references and explaining that BBBM-AD and MCI-AD require calibration - Fix LaTeX rendering (use arrow notation instead of escaped underscores) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Rename "presymptomatic" to "pre-clinical" throughout - Clarify that excess mortality comes from dementia envelope but only incidence and prevalence are multiplied by AD proportion - Remove outdated MCI-AD age restriction notes - Update BBBM-AD state definition to note absence of cognitive impairment - Reorganize calibration reference for clarity 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
docs/source/models/causes/alzheimers/presymptomatic_and_mci_gbd_2021/index.rst
Outdated
Show resolved
Hide resolved
…d_2021/index.rst Co-authored-by: SylLutze <108029462+SylLutze@users.noreply.github.com>
| * In practice, the age start for MCI-AD will be age_group_id = 10 | ||
| (25 to 29) because we will be adding simulants at most 10.2 | ||
| years before AD incidence (so 40 -- 10.2 = 29.8, in the 25-29 | ||
| age group) |
There was a problem hiding this comment.
What's the reason for deleting this? I think this is the only place where we documented how we computed 25-30 for the youngest age group we need to simulate.
There was a problem hiding this comment.
Instead of approximating this with an algebraic expression based on the average durations of MCI, it is now being generated through the calibration optimization, so "10.2 years" years is no longer relevant. The optimization starts at age 30, though, so I could specify that here. (I could also include age group 25-30 in the optimization!)
There was a problem hiding this comment.
Ok, I was confused because I thought the population model was still doing what the documentation says, which implies the 10.2-year duration. If the BBBM incidence rate is now 0 for age < 30, we shouldn't need the 25-30 age group anymore, so we should drop that from the sim. And it might be worth still noting here that the youngest possible age for MCI is 30 because that's now the youngest possible age for BBBM incidence.
docs/source/models/causes/alzheimers/presymptomatic_and_mci_gbd_2021/index.rst
Show resolved
Hide resolved
| see :math:`h_\text{MCI}(t)` in `Data Values and Sources`_ | ||
| * - :math:`h_{\text{MCI} \to D}` | ||
| - Hazard rate of transitioning from MCI-AD to AD-dementia | ||
| - Constant hazard: :math:`1 / \Delta_\text{MCI}` |
There was a problem hiding this comment.
| many simulants to add into the BBBM-AD state on each time step. | ||
| * - i_MCI | ||
| - From :ref:`rate calibration <cause_alzheimers_rate_calibration>`. | ||
| When modeling only AD simulants, this is used implicitly by the |
There was a problem hiding this comment.
Is the hazard rate you calculated actually used by the population model, or do we essentially recompute it using a different method? Should the population model just be using the rate you calculated instead of doing all the convoluted calculations we're doing to get
There was a problem hiding this comment.
The population model uses this rate, it is not actually used in the disease model. I will try to make that clearer here.
There was a problem hiding this comment.
Ok, it sounds like you're saying that this calibrated rate is in fact being used by the population model. I was confused because this contradicts the documentation for the population model, which still says that we use approximations to back-calculate the simulant entrance rate from the Alzheimer's incidence rate. So how exactly are you using this new incidence rate instead? Did you just replace the artifact key that gets multiplied by the model scale and timestep? Presumably with
There was a problem hiding this comment.
As discussed on 1/7, I will work on a PR to edit the population model documentation to reflect this change.
docs/source/models/causes/alzheimers/presymptomatic_and_mci_gbd_2021/index.rst
Outdated
Show resolved
Hide resolved
|
|
||
| ================================================================== | ||
| Alzheimer's disease with presymptomatic and MCI stages (GBD 2021) | ||
| Alzheimer's disease with pre-clinical and MCI stages (GBD 2023) |
There was a problem hiding this comment.
I think "preclinical" is usually written without a hyphen, but it's probably fine to leave it.
| above formula applies to age groups 40+ since this is where | ||
| prevalence_AD and :math:`\Delta_\text{AD}` are nonzero. For ages | ||
| 30--39, use the value of :math:`p_\text{(all AD states)}` for age | ||
| group 40--44; for ages <30, set :math:`p_\text{(all AD states)} = 0`. |
There was a problem hiding this comment.
Presumably your consistent rates code will save zeros in the appropriate age groups, so we don't need to explicitly specify this anymore?
There was a problem hiding this comment.
Oh, I wonder what is happening outside of the age groups I have saved; I bet that they are extrapolated from the age groups I do have, so they might be non-zero. I will add a TODO about that to my consistent rates with severity branch.
There was a problem hiding this comment.
Could that explain the weird bump in incidence and prevalence that we were seeing in the younger ages?
docs/source/models/causes/alzheimers/presymptomatic_and_mci_gbd_2021/index.rst
Outdated
Show resolved
Hide resolved
| * - :math:`\Delta_\text{AD}` | ||
| - Average duration of AD-dementia | ||
| - * prevalence_AD / incidence_AD for ages 40+ | ||
| * 0 for ages under 40 | ||
| - Follows from the steady-state equation (prevalent cases) = (incident | ||
| cases) x (average duration). Note that the denominator is the **raw | ||
| total-population incidence rate from GBD**, not the | ||
| susceptible-population incidence rate usually returned by Vivarium | ||
| Inputs. This is because we want the total-population person-time in the | ||
| denominators of prevalence and incidence to cancel out, leaving a ratio | ||
| of counts. | ||
| * - :math:`\Delta_\text{(all AD states)}` | ||
| - Average duration of all stages of AD combined if there is no | ||
| mortality in the BBBM-AD and MCI-AD stages | ||
| - :math:`\Delta_\text{BBBM} + \Delta_\text{MCI} + \Delta_\text{AD}` | ||
| - |
There was a problem hiding this comment.
Are
| m_\text{all}(a) = m(a) + f(a) \cdot p_\text{dementia}(a) | ||
|
|
||
| **ODE Consistency Constraints.** The calibration enforces consistency by | ||
| solving a 5-compartment ODE system from age :math:`a` to :math:`a + 5` and |
There was a problem hiding this comment.
I don't know what it means to solve an ODE system "from age
There was a problem hiding this comment.
Yeah, I really have no idea how you're dealing with different age groups, so it would be good to explain that somewhere.
There was a problem hiding this comment.
Does my modest edit help? I think we can probably make it clearer through discussion and then figure out how to write it.
|
|
||
| **ODE Consistency Constraints.** The calibration enforces consistency by | ||
| solving a 5-compartment ODE system from age :math:`a` to :math:`a + 5` and | ||
| penalizing deviations between ODE-predicted values and calibrated parameters. |
There was a problem hiding this comment.
What's the difference between an "ODE-predicted value" and a "calibrated parameter"? Are "calibrated parameters" the given data that you are trying to match, or did you have to use some additional process to calibrate the target values first?
There was a problem hiding this comment.
It is basically the "method of multiple shootings", which is a cool hack I learned from Brad Bell. I will elaborate in the docs.
| - :math:`\delta_\text{BBBM}`, :math:`\delta_\text{MCI}`: Conditional | ||
| prevalences of BBBM and MCI states among AD cases | ||
| - :math:`h_{S \to \text{BBBM}}`: Transition rate from susceptible to BBBM | ||
| - :math:`i`: Population incidence rate of dementia |
There was a problem hiding this comment.
I don't see
There was a problem hiding this comment.
It is different than h_MCI->D, but related. But I made it a parameter and used consistency constraints to get the relationship instead of enforcing it deterministically (h_MCI->D has a denominator of MCI person years, while i has a denominator of any person years).
There was a problem hiding this comment.
Ok, that makes sense. It might be better to call it the "total-population incidence rate" for extra clarity, since that's what we call it elsewhere in the docs.
| **Numerical Methods.** We solve the ODE using diffrax (Dopri5) and sample using | ||
| NUTS with 500 warmup and 500 sample iterations. | ||
|
|
||
| .. list-table:: Calibration Input Data |
There was a problem hiding this comment.
tl;dr: If these are the same as some of the values in the Data Values and Sources table above, you should rename things to be consistent (or at least explicitly specify a mapping from one to the other). And rename "f" so that the input EMR is not called the same thing as the output EMR.
...
Are these all values we get directly from GBD? It would be good to specify the sources more explicitly.
Is p_dementia the same as prevalence_AD in the Data Values and Sources table above? I.e., the total prevalence of the dementia envelope multiplied by the proportion Alzheimer's that we got from the dementia modelers?
Similarly, is i_dementia the same as incidence_AD from the Data Values and Sources table?
If so, we probably shouldn't have two different notations for the same thing -- if you like your notation better, you could rename the variables in the Data Values and Sources table. Except in that table it seems more important to distinguish AD from other types of dementia...
Is f the excess mortality rate from the dementia envelope? If so, it might be better to call it emr_m24351, following our usual naming convention for data. Especially because you are already using
Presumably m_all is the mortality data from FHS? I.e., the same as acmr in the Data Values and Sources table above? If so, it seems like you can just rename "acmr" in that table to
There was a problem hiding this comment.
Also, how exactly is the input data used? E.g., are the values treated as constraints that must stay fixed throughout the calibration, or are they treated as initial values that then get updated to make things consistent?
It seems like maybe p_dementia, i_dementia, and the EMR are used as initial values, and m_all is a constraint that remains fixed, but I'm not sure.
There was a problem hiding this comment.
There is probably a way to clean this up, I think I have some confusing redundancy here. These are the values to calibrate the ODE to (these are in addition to the consistency constraints described above)
| :widths: 15 35 30 20 | ||
| :header-rows: 1 | ||
|
|
||
| * - Variable |
There was a problem hiding this comment.
If these are the same variables you're using in your equations above, you should use the same font. E.g.,
There was a problem hiding this comment.
Let's discuss before I make the change, because maybe something else will be clearer.
| enforcing ODE-based consistency constraints. | ||
|
|
||
| The calibration is implemented in ``consistent_rates.py`` in the | ||
| vivarium_csu_alzheimers repository. We fit separate models for males and |
There was a problem hiding this comment.
Might as well link to the repository (or the specific file) here.
| - :math:`f`: Excess mortality rate | ||
| - :math:`m`: Background (non-AD) mortality rate | ||
|
|
||
| **Key Relationships.** Dementia prevalence derives from total AD prevalence: |
There was a problem hiding this comment.
What exactly is "Key" about these relationships? How are they being used?
I'm struggling to figure out exactly what's going on. Are
There was a problem hiding this comment.
I guess it would be more accurate to call these definitions, rather than key relationships.
| **ODE Consistency Constraints.** The calibration enforces consistency by | ||
| solving a 5-compartment ODE system from age :math:`a` to :math:`a + 5` and | ||
| penalizing deviations between ODE-predicted values and calibrated parameters. | ||
| The state variables are S (susceptible), BBBM, MCI, D (dementia), and |
There was a problem hiding this comment.
Aha! I couldn't figure out how the
You should probably explicitly write (some subset of) these somewhere to clarify how everything is related.
| \frac{dD}{dt} &= h_{\text{MCI} \to D} \cdot \text{MCI} - (m + f) \cdot D \\ | ||
| \frac{dD_\text{new}}{dt} &= h_{\text{MCI} \to D} \cdot \text{MCI} | ||
|
|
||
| The transition rates :math:`h_{\text{BBBM} \to \text{MCI}} = 1 / \Delta_\text{BBBM}` |
There was a problem hiding this comment.
As I noted above, this constant hazard differs from our definition of the non-constant hazard function above, so you should probably note that this is an approximation of our Weibull hazard function.
| solving a 5-compartment ODE system from age :math:`a` to :math:`a + 5` and | ||
| penalizing deviations between ODE-predicted values and calibrated parameters. | ||
| The state variables are S (susceptible), BBBM, MCI, D (dementia), and | ||
| :math:`D_\text{new}` (cumulative incident dementia). The ODE system is: |
There was a problem hiding this comment.
Why do you need
There was a problem hiding this comment.
To calculate the dementia incidence rate for calibration; I will elaborate.
| \left(\log \hat\delta_\text{BBBM} - \log \delta_\text{BBBM}(a+5)\right)^2 + | ||
| \left(\log \hat\delta_\text{MCI} - \log \delta_\text{MCI}(a+5)\right)^2 + | ||
| \left(\log \hat p_\text{dementia} - \log p_\text{dementia}(a+5)\right)^2 + | ||
| \left(\log \hat\imath - \log i(a+2.5)\right)^2 |
There was a problem hiding this comment.
Why does
Also, as noted above, how does
There was a problem hiding this comment.
I don't remember! I think there is something different about incidence and prevalence that made me think incidence at the midpoint of the interval would be better than at the endpoint.
| .. math:: | ||
|
|
||
| \epsilon(a) = \sqrt{ | ||
| \left(\log \hat\delta_\text{BBBM} - \log \delta_\text{BBBM}(a+5)\right)^2 + |
There was a problem hiding this comment.
Are the "hat" variables supposed to be evaluated at
There was a problem hiding this comment.
hat is what the ODE implies, no-hat is the parameters being optimized
|
|
||
| where :math:`\hat\delta_\text{BBBM}`, :math:`\hat\delta_\text{MCI}`, | ||
| :math:`\hat p_\text{dementia}`, and :math:`\hat\imath` are derived from the ODE | ||
| solution. The calibration applies a normal penalty :math:`\epsilon(a) \sim |
There was a problem hiding this comment.
I'm not sure what a "normal penalty" means. Does this mean that you are including the error term
| females. | ||
|
|
||
| **Model Parameters.** The NumPyro model defines 7 age-varying parameters, each | ||
| with truncated normal priors on :math:`[0, 1]`: |
There was a problem hiding this comment.
Why do the rates
There was a problem hiding this comment.
I did not try other priors, and [0, inf] would be a fine alternative for rates.
| includes presymptomatic states (BBBM-AD and MCI-AD) not directly measured by | ||
| GBD. To derive internally consistent rates for these states, we use Bayesian | ||
| inference with NumPyro/JAX to fit disease progression rates to GBD data while | ||
| enforcing ODE-based consistency constraints. |
There was a problem hiding this comment.
I think the biggest thing missing from this section is an explanation of your overall strategy for finding the consistent rates. This one-sentence summary is good if you want to give someone an idea of what you did without actually explaining it, but it's not sufficient to actually understand how your approach works. If I had to guess, here's my best attempt to reconstruct what you did:
- You're doing an optimization, presumably to find the best possible values of the seven parameters listed in the "model parameters" section, subject to the constraints imposed by the system of ODEs. However, as noted elsewhere, I'm still not sure what data is being used as a fixed constraint (maybe all-cause mortality?) vs. as an initial value for one of the parameters in the optimization (maybe some of the dementia envelope data from DisMod?). And I also don't know whether you're treating different age groups completely independently, or whether solutions for different age groups are somehow coupled together, or what.
- You're using Bayesian inference with MCMC, which means you must be sampling from some joint distribution. My best guess at what distribution you're sampling is the 8-dimensional distribution of the seven model parameters together with the variable
$\epsilon(a)$ representing the value of the loss function for the optimization. Is that right? And if you're doing Bayesian inference, what are the likelihood function and the evidence/data that go into the calculation of the posterior distribution? Presumably these are somehow related to the system of ODEs, but I'm unclear exactly how to formulate the inference problem.
This seems like a clever idea to use Bayesian inference to do an optimization, but I'm not sure if I'm understanding correctly -- I could be way off with my attempt to fill in the details. My point is that there are a lot of details that are left to the imagination, and it would be helpful if you filled more of them in.
There was a problem hiding this comment.
I've tried to adapt your thoughts to give future readers a head start in figuring this out.
|
|
||
| **Loss Function.** After solving the ODE from age :math:`a` to :math:`a + 5`, | ||
| the calibration computes the root-sum-squared log-difference between ODE | ||
| predictions and parameter values: |
There was a problem hiding this comment.
Maybe this is a small thing, but in what sense are the values from the ODEs "predictions"?
There was a problem hiding this comment.
Perhaps it is more precise to say "After solving the ODE with initial values for age :math:a to find the ODE-implied values at :math:a + 5, the calibration computes the root-sum-squared log-difference between ODE-impled values and parameter values"
| - ``cause.all_causes.cause_specific_mortality_rate`` | ||
| - 2025 | ||
|
|
||
| .. list-table:: Calibrated Outputs (written to artifact for year 2025) |
There was a problem hiding this comment.
It looks like the values in this table correspond to the 7 variables in the "model parameters" section, except that you've added the ODE errors (
There was a problem hiding this comment.
I did not use m in the sim (it is calculated by the mortality component as m_all - CSMR); I did save CSMR, though, which I did not include in this table.
There was a problem hiding this comment.
Then why does the data table list
Also, how did you calculate CSMR in your calibration? I don't see that explained anywhere.
|
There are a few references to data values in the Data Sources table on the Alzheimer's Population page that will need to get updated based on the changes in this PR (e.g., |
…d_2021/index.rst Co-authored-by: Nathaniel Blair-Stahn <ndbs@uw.edu>
…d_2021/index.rst Co-authored-by: Nathaniel Blair-Stahn <ndbs@uw.edu>
|
|
||
| The transition rates :math:`h_{\text{BBBM} \to \text{MCI}} = 1 / \Delta_\text{BBBM}` | ||
| and :math:`h_{\text{MCI} \to D} = 1 / \Delta_\text{MCI}` are fixed based on | ||
| literature values in the data values table above and the assumption that the Weibull distribution is approximated acceptably by a constant for calibration purposes. |
There was a problem hiding this comment.
It's confusing to say the distribution can be approximated by a constant...
| literature values in the data values table above and the assumption that the Weibull distribution is approximated acceptably by a constant for calibration purposes. | |
| literature values in the data values table above and the assumption that the Weibull distribution is approximated acceptably by a constant hazard for calibration purposes. |
| - Background mortality rate (non-AD mortality) | ||
| - From :ref:`rate calibration <cause_alzheimers_rate_calibration>`, | ||
| artifact key ``cause.alzheimers_consistent.background_mortality_rate`` | ||
| - Applies to all states. Derived from forecasts of all-cause mortality. |
There was a problem hiding this comment.
Is it possible to specify how this is derived from all-cause mortality together with the calibration? Currently, this data table lists ACMR as one of the needed data inputs for the model, but doesn't show it being used anywhere.
| p\_\text{dementia} = \frac{D}{S + \text{BBBM + MCI} + D} | ||
|
|
||
|
|
||
| **ODE Consistency Constraints.** The calibration produces consistent parameters by |
There was a problem hiding this comment.
I think this section should probably go before the non-ODE constraints, in order to introduce all the notation.
| solving a 5-compartment ODE system starting with initial conditions at from age :math:`a` to find the implied values at :math:`a + 5`; we include the squared difference between these implied values and the parameter values in the MCMC objective. | ||
|
|
||
| The state variables are S (susceptible), BBBM, MCI, D (dementia), and | ||
| :math:`D_\text{new}` (cumulative incident dementia, which is used to calibrate the incidence rate of AD dementia). The ODE system is: |
There was a problem hiding this comment.
I still don't understand why
If so, you should explicitly write that in the equation for
I guess I'm confused because if you have
| with additional evidence on state durations. See :ref:`below for details <cause_alzheimers_rate_calibration>`. | ||
|
|
||
| To obtain Alzheimer's-specific estimates (excluding "other dementias"), we | ||
| multiply the GBD 2023 dementia envelope by the proportion due to Alzheimer's |
There was a problem hiding this comment.
We'll need to explain that we also include 94% of mixed dementias, as specified in #1883
Update the Alz cause model docs, and add a section about using a system of ODEs to calibrate the model.