Deferred from PR #229 review (CodeRabbit, hvantk/algorithms/burden/fet.py).
_driver_af picks the reported driver with max(drivers, key=lambda d: d["cc"]), which has no
secondary key. When two driver variants tie on cc (max case-carrier count), the chosen driver --
and therefore the reported driver_af (its control-carrier frequency) -- depends on the order the
drivers happen to appear in the collected Hail array. pick_min_p already sorts deterministically,
so this is the one place a tie can leak nondeterminism into an output column.
Why it wasn't fixed inline: _driver_af's carrier-vs-allele semantics are explicitly pinned by
the CHD reproduction gate (see the comment directly above the function). Adding a ctrl_freq
tie-break changes which driver's frequency is reported on cc-ties, so it must be validated
against a gate re-run (a Slurm job) to confirm it doesn't perturb the pinned numbers.
Plan:
- Add a deterministic secondary key, e.g.
key=lambda d: (d["cc"], -d["ctrl_freq"]).
- Re-run
local/sbatch/chd_burden_gate.{py,sbatch} and confirm the reduction columns are unchanged
(expected: no cc-ties with differing ctrl_freq in the CHD cohort, so the gate is identical).
- Optionally add a unit test with a constructed
cc-tie asserting a stable pick.
Deferred from PR #229 review (CodeRabbit,
hvantk/algorithms/burden/fet.py)._driver_afpicks the reported driver withmax(drivers, key=lambda d: d["cc"]), which has nosecondary key. When two driver variants tie on
cc(max case-carrier count), the chosen driver --and therefore the reported
driver_af(its control-carrier frequency) -- depends on the order thedrivers happen to appear in the collected Hail array.
pick_min_palready sorts deterministically,so this is the one place a tie can leak nondeterminism into an output column.
Why it wasn't fixed inline:
_driver_af's carrier-vs-allele semantics are explicitly pinned bythe CHD reproduction gate (see the comment directly above the function). Adding a
ctrl_freqtie-break changes which driver's frequency is reported on
cc-ties, so it must be validatedagainst a gate re-run (a Slurm job) to confirm it doesn't perturb the pinned numbers.
Plan:
key=lambda d: (d["cc"], -d["ctrl_freq"]).local/sbatch/chd_burden_gate.{py,sbatch}and confirm the reduction columns are unchanged(expected: no
cc-ties with differingctrl_freqin the CHD cohort, so the gate is identical).cc-tie asserting a stable pick.