Allow sequential testing

[Gabrielcidral1]

Currently we can use the class NormalPowerAnalysis.mde_time_line to see the evolution of MDE by the time. But I understand that this help to decide the experiment duration as a fixed period. I would like to see this line as a sequential testing, where you decide a max duration of a test, but it can stop if you pass some threshold level in terms of alpha.

2 references:

• https://www.youtube.com/watch?v=4rWOx5fOJbg
• https://engineering.atspotify.com/2023/03/choosing-sequential-testing-framework-comparisons-and-discussions/

[david26694]

Would the input and output be the same? I give some dataframe with an outcome and date column and it returns a mapping of experiment length to MDE, right?

The only difference is that, when calculating this mapping time -> MDE, we use sequential testing and not normal testing, right?

[Gabrielcidral1]

exactly! We can also set as an input how "aggressive" this increase of alpha can be.So it can follow a function like this (@matheusbuniotto's idea):

[david26694]

very cool, do you have an idea of how should we implement it?