Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
e777a80
update testing coverage function
NathanielBlairStahn Jan 23, 2026
6109f70
add randomness to testing interval
NathanielBlairStahn Jan 23, 2026
9f5b82d
update BBBM testing history from a 3-year interval to a 5-year interval
NathanielBlairStahn Jan 23, 2026
5fbef6e
adjust algorithm to distribute next test date uniformly between 3 and…
NathanielBlairStahn Jan 24, 2026
144321a
update initialization strategy for BBBM testing and describe it befor…
NathanielBlairStahn Jan 26, 2026
7df9a20
delete old information and add a to-do to explain probability formula
NathanielBlairStahn Jan 26, 2026
346849b
clarifications for how to count time steps
NathanielBlairStahn Jan 26, 2026
695a008
remove testing function knot at 2050
NathanielBlairStahn Jan 26, 2026
f25cfdf
edit BBBM testing algorithm to assign a testing history just in time …
NathanielBlairStahn Jan 27, 2026
875613b
rewrite initialization strategy to assign NaT appropriately
NathanielBlairStahn Jan 27, 2026
4d28ff2
comment out alternate BBBM testing algorithm
NathanielBlairStahn Jan 27, 2026
2bc0abf
Math explanation, adding model run request, adjusting limitations
Jan 27, 2026
bde4322
Merge branch 'main' into alzheimers/model-12
SylLutze Jan 27, 2026
d421514
changing to test immediately upon eligibility
Jan 27, 2026
cce45a0
clarify testing algorithm and clean up assumptions
NathanielBlairStahn Jan 28, 2026
28bc036
Update docs/source/models/intervention_models/alzheimers/testing_diag…
SylLutze Jan 30, 2026
febaa79
Updating age, sensitivity value, and reorganizing the match engineer …
Feb 3, 2026
d6a46ca
Updating to test future date not test history date
Feb 11, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -681,6 +681,13 @@ scenario, and input draw.
- * Locations: USA, China, Brazil
- Stratify disease state transitions and person-time by treatment
- Add observer for months on treatment
* - 12.0
- Updates to testing model
- Baseline, Alternative Scenario 1, Alternative Scenario 2
- * Locations: USA, China, Brazil
- Stratify disease state transitions and person-time by treatment
- Default


5.2 V & V Tracking
------------------------
Expand Down
185 changes: 132 additions & 53 deletions docs/source/models/intervention_models/alzheimers/testing_diagnosis.rst
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ should be tested at the first time step.

To accomplish this, simulant eligibility should be checked at simulation initialization,
and simulants who satisfy all eligibility requirements at that time should be marked as having
previously recieved a CSF/PET test. These simulants will be ineligible for future
previously received a CSF/PET test. These simulants will be ineligible for future
CSF/PET testing.

Assumptions and Limitations
Expand Down Expand Up @@ -208,59 +208,143 @@ assign positive/negative diagnosis which will inform treatment in :ref:`Alternat
Time-specific testing rates
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Testing rates do not vary by location, age or sex.
In 2020, 0% of eligible simulants are tested annually. This increases (instantly) to 10% at year 2030,
then increases linearly over time in each six-month period to reach 20% in 2035, to 40% in 2040
and then maxes out at 60% in 2045.
In 2020, 0% of eligible simulants are tested annually. This becomes
nonzero in 2027, increasing to 10% at year 2030,
then increases linearly for awhile, then levels off and eventually maxes
out at 60% after 2045. We will model this as a piecewise linear function
with knots at the following (year, coverage) values:

* (2020.0, 0%)
* (2027.0, 0%) -- Note that this is 0% at the *beginning* of 2027, but
coverage will become positive on the second time step that year
* (2030.5, 10%) -- Note that this is 10% at **mid**-year
* (2045.0, 50%)
* (2055.0, 60%)
* (2100.0, 60%)

.. _bbbm_requirements:

Eligibility for BBBM testing
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A simulant is eligible for a BBBM test if they meet the following
requirements:

- Simulant is not in MCI or AD dementia state (they can only be in
susceptible or preclinical)
- Simulant age is :math:`\ge 65` and :math:`< 80`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note this change: >= 65 now, at CSU client's request. (It was different before)

- Simulant has not received a BBBM test in the last three years (more
precisely, they have not had a BBBM test on any of the previous five
time steps)
- Simulant has never received a positive BBBM test

.. _bbbm_propensity:

Implementation
^^^^^^^^^^^^^^
The simulant's existing testing propensity will also be used as their BBBM testing propensity.
The simulant's existing CSF/PET testing propensity will also be used as
their BBBM testing propensity. At the client's request, we will retest
simulants every 3-5 years, rather than having all simulants be retested
at a fixed interval of 3 years (which can cause unrealistic oscillations
in the number of tests over time). In the implementation below, we
use a formula for testing that creates uniform testing in the interval :math:`[3, 5]`
years.

On initialization
'''''''''''''''''
In order to avoid having an unreasonably large fraction of eligible
simulants be tested immediately upon entering the simulation, we will
assign a future BBBM test date to each initialized simulant who otherwise would
have an opportunity for BBBM testing on their first time step.

On timestep
'''''''''''
On each timestep, use the following steps to assign BBBM tests:
For this future BBBM test date assignment, we must meet the following requirements:

.. _bbbm_requirements:
#. Test date is assigned to simulants who meet
the :ref:`eligibility requirements for BBBM testing
<bbbm_requirements>` and has a testing propensity is less than the
current BBBM testing rate.
#. The BBBM test date should mirror the future testing scheme of uniformly random
retesting between 3-5 years.

1. Assess eligibility based on the following requirements:
.. note::

- Simulant is not in MCI or AD dementia state (only susceptible, or pre-clinical)
- Simulant age is >=60 and <80
- Simulant has not received a BBBM test in the last three years (or
six time steps)
- Simulant has never received a positive BBBM test
Implementation: we achieve the above criteria by having two random draws. First,
a random time between 3 and 5 years is selected. This value is the time the simulant
was assigned to wait to retest. Second, a duration between zero and the time from the prior draw is picked.
This is the amount of time into the waiting period the simulant is. The time to testing
is then calculated from these two draws. For example, if in the first draw we select
4 years, and the second draw we select 2 years, the simulant would be assigned to
retesting 4-2=2 years in the future.

2. If eligible (meets all requirements), check propensity.
If the propensity value is less than the time-specific testing rate, give the test. If not, do not give the test.
3. Assign a positive diagnosis to 90% of people and a negative diagnosis to 10% of people. This 90% draw should be independent of any previous draws, e.g., people who test negative still have a 90% chance of being positive on a re-test.
4. Record time of last test, yes/no diagnosis for future testing eligibility.
We assume for simplicity that there were no prior false positive tests
among simulants entering the simulation, so all previous BBBM tests are
negative. For simulants who are assigned a previous test date, the first
time they could become eligible for testing again is 6 time steps after
the chosen previous test date.

On initialization
'''''''''''''''''
On timestep
'''''''''''
On each timestep, simulants will have a chance to receive a BBBM test.
This process must meet the following requirements:

#. Only simulants who are eligible based on the :ref:`eligibility requirements for
BBBM testing <bbbm_requirements>` and whose propensity is within the time specific
value will receive testing.
#. If a simulant meets these criteria and has NaT assigned, they will be tested
immediately.
#. For those who get tested, assign a positive diagnosis to 50% of people and a
negative diagnosis to 50% of people. This 50% draw should be independent of
any previous draws, e.g., people who test negative still have a 50% chance
of being positive on a re-test.
#. If a simulant tests negative, the time of their next test is uniformly
distributed between 3 and 5 years since their prior test.
#. Record time of last test and yes/no diagnosis for determining future testing eligibility.

.. note::

Implementation: there are multiple possible ways to implement a uniform
distribution of retesting. The current model implementation assigns a
future retest date between 3 and 5 years in the future.

Another mathematically equivalent option is to use a non-constant hazard
function to that increases the test liklihood each timestep. The formula
for for the non-constant hazard function for this would be :math:`1/(11 - k)`.
This results in a uniformly distributed population between 6 and 10 time steps,
or 3 to 5 years. To conceptualize why this works, please see the table below
outlining the time step value :math:`k`, the resulting probability of testing
and how a hypothetical population of 100 simulants is distributed over the time steps.

.. list-table:: Simulation Components
:header-rows: 1

* - Time Step :math:`k`
- Testing Probability
- People Tested
- Remaining Untested Population
* - 0-5
- 0% (ineligible)
- 100 * 0% = 0
- 100 - 0 = 100
* - 6
- 1/(11-6) = 20%
- 100 * 20% = 20
- 100 - 20 = 80
* - 7
- 1/(11-7) = 25%
- 80 * 25% = 20
- 80 - 20 = 60
* - 8
- 1/(11-8) = 33%
- 60 * 33% = 20
- 60 - 20 = 40
* - 9
- 1/(11-9) = 50%
- 40 * 50% = 20
- 40 - 20 = 20
* - 10
- 1/(11-10) = 100%
- 20 * 100% = 20
- 20 - 20 = 0

In order to avoid having all eligible simulants be tested immediately
upon entering the simulation, we will assign a BBBM testing history to
each initialized simulant who is eligible for a BBBM test. Since
simulants are only eligible for testing every three years (more
precisely, every 6 time steps), we will assign a random test date within
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aflaxman I wrote this section post hoc to describe what the engineers actually did, which involves counting time steps explicitly. I think they do it that way to ensure uniformity in sampling since the time steps are not exactly half a year (though really they should be if we wanted to do things better...), so maybe if we just rounded from a continuous time, it would tend to be biased in one direction or the other (though I'm not totally sure). Now that we're re-doing this piece, do you think I should rewrite this in terms of continuous time to avoid locking us into a particular time step, or is it fine to leave it in terms of the number of time steps? Rewriting it will require an explicit strategy for mapping continuous time to a discrete time step -- I'm not sure what the best strategy is or what strategies are compatible with how the engineers think about time steps. I think we are slated to have a wider team meeting about time-step-related issues like this at some point, but I'm not sure when.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine to keep to minimal change and not rewrite

the last three years before entering the simulation, as follows.

On initialization of each eligible simulant, choose uniformly at random
from one of the last 6 time steps when they could have been tested,
omitting any time steps before 2030 when testing is not yet available.
If there are no such time steps (i.e., all 6 are before 2030), assign
"not a time" (NaT) for the simulant's previous test date. Otherwise, the
first time the simulant could be eligible for testing again is 6 time
steps after the chosen previous test date. We assume for simplicity that
there were no prior false positive tests among simulants entering the
simulation, so all previous BBBM tests are negative.

Even with prior BBBM testing history in place, due to test coverage
jumping from 0% to 10% in 2030, we expect a large group to be
immediately tested and then a drop-off in testing counts.

.. note::

Expand Down Expand Up @@ -289,18 +373,13 @@ Assumptions and Limitations
the simulation;
- The strategy for assigning BBBM test history does not account for the
fact that simulants may not have been eligible for BBBM testing on all
of the previous 6 time steps prior to entering the simulation; for
example, we will assign a previous BBBM test date to a 60-year-old
entering the simulation after 2030 even though they wouldn't have been
eligible; the effects of this are likely small because improper
testing can only happen during the first 3 years of the 20 years of
eligible ages;
- The strategy of choosing the prior BBBM testing date uniformly over
the last 3 years is a simplification that doesn't align perfectly with
our assumption that there will be a cyclical pattern in the number of
people getting tested each year (with the first peak in 2030 when the
test first becomes available); the uniformity assumption will likely
smooth out this cyclical pattern somewhat;
of the previous 10 time steps prior to entering the simulation; for
example, we will assign a previous BBBM test date to a 65-year-old
entering the simulation in, say, 2035 even though they wouldn't have
been eligible; the effects of this are hopefully small because
improper testing can only happen during the first 5 years of the 20
years of eligible ages;



References
Expand Down