Allow running non inferiority tests

[Gabrielcidral1]

I would like to be able to run a non inferiority power analysis,
Ho: B - A ≤ -Δ
H1: B - A > -Δ

From what I understand, it's the same as the superiority test, but with the addition of a non inferiority margin (delta).

So I am thinking of achieving it by:

alpha = 0.05
power = 0.8
delta = -3

pw_normal = NormalPowerAnalysis.from_dict(
    {
        "splitter": "clustered",
        "analysis": "clustered_ols",
        "cluster_cols": ["customer_id"],
        "n_simulations": 5,
        "hypothesis": "greater",
        "seed": 3,
        "time_col": "margin_date",
        "target_col": "revenue_value"
    }
)

z_alpha = norm.ppf(1 - alpha)
z_beta = norm.ppf(power)

generator_se = pw_normal.run_average_standard_error(
    df=df,
    experiment_length=range(1,31),
)

mde_results = []
for se, days in generator_se:
    mde = se * (z_alpha + z_beta) + delta
    mde_results.append((days, se, mde))

mde_df = pd.DataFrame(mde_results, columns=["days", "standard_error", "mde"])

I am not sure if this should be implemented as a new capability of the NormalPowerAnalysis, or it we create a new NonInferiorityNormalPowerAnalysis

[david26694]

I would keep it in NormalPowerAnalysis (and ideally add it in PowerAnalysis too). The API should look like:

# Normal approximation (faster)
npw = NormalPowerAnalysis.from_dict({
    "analysis": "ols",
    "splitter": "non_clustered",
    "n_simulations": 5,
    "time_col": "date",
    "hypothesis": "greater",
})
power_normal = npw.power_analysis(df, average_effect=0.1, margin_threshold=0.05)

mde_timeline = npw.mde_time_line(
    df,
    powers=[0.8],
    experiment_length=[7, 14, 21],
    margin_threshold=0.01
)

What I'm wondering is if it's worth adding this, since these two should give the same result:

power_normal = npw.power_analysis(df, average_effect=0.1, margin_threshold=0.03)
power_normal = npw.power_analysis(df, average_effect=0.1 - 0.03)