You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<textx="150"y="270"textAnchor="middle"fill="#50c878"fontSize="8">promoted track becomes a normal arm</text>
878
878
</svg>
879
879
880
-
<pstyle={{marginTop: "0.35rem"}}>A <strong>challenger</strong> is one new proxy configuration — a single region × datacenter × protocol — provisioned as a tiny track. It's measured against the <strong>control</strong>: the established track free users in that region already get. Experiments only run in <strong>censored markets</strong> (default CN, RU, IR, MM), where a better config matters most.</p>
880
+
<pstyle={{marginTop: "0.35rem"}}>A <strong>challenger</strong> is one new proxy configuration — a single target market × datacenter × protocol — provisioned as a tiny track on one VPS. It's measured against the <strong>control</strong>: the established track free users in that market already get. Concurrent challengers in the same market are pinned to <strong>disjoint device-hash slices</strong> (≤ 2% each) so they don't cannibalize each other's samples. Experiments only run in <strong>censored markets</strong> (default CN, RU, IR, MM), where a better config matters most.</p>
881
881
882
-
<pstyle={{marginTop: "0.35rem"}}>Three background workers drive the loop. The <strong>proposer</strong>enumerates untried combinations and launches a budgeted batch (capped by max-concurrent and per-region limits). The <strong>realizer</strong> turns a proposal into a live challenger track — one VPS serving at most ~2% of free clients, so a bad config can't hurt many people. The <strong>evaluator</strong>, after at least <strong>24 hours</strong> of gathering, compares the median download <strong>goodput</strong>of challenger vs control within each client country.</p>
882
+
<pstyle={{marginTop: "0.35rem"}}>Three background workers drive the loop. The <strong>proposer</strong>scores untried combinations with a <strong>learning value model</strong> — protocol-family and provider effects, novelty, and cross-market seeding of proven winners, all recency-decayed — and launches a budgeted batch by ε-greedy selection (defaults: 20 concurrent, 4 per market, 4 per batch). A losing combo goes on a <strong>cooldown</strong> (default 30 days) rather than being blacklisted forever, so it can be re-explored as censorship shifts. The <strong>realizer</strong> turns a proposal into a live challenger track — one VPS on a disjoint ≤ 2% slice. The <strong>evaluator</strong>, after at least <strong>24 hours</strong> of gathering, compares challenger vs control on the <strong>target market</strong>across two co-primary axes: median download <strong>goodput</strong> and connect/<strong>success rate</strong>.</p>
883
883
884
-
<pstyle={{marginTop: "0.35rem"}}>The decision rule optimizes for<strong>big wins</strong>: a challenger is promoted only if it beats the control by <strong>≥ 40%</strong>in a majority of qualifying country strata (each needs ≥ 6 strata with ≥ 50 sessions per arm). A symmetric loss — or an inconclusive result after<strong>14 days</strong>— retires it; anything in between holds for more data.</p>
884
+
<pstyle={{marginTop: "0.35rem"}}>The verdict is <strong>co-primary</strong>. On<strong>goodput</strong>, a challenger is promoted when it beats the control by an <strong>adaptive margin</strong> — starting near <strong>40%</strong>at the sample floor and relaxing toward a <strong>15%</strong> floor as sessions accrue — judged on the target market's own stratum (other markets act only as a no-regression guardrail). Independently, a Wald <strong>sequential test (SPRT)</strong> on the target-market<strong>success rate</strong>promotes a challenger that is <em>materially more reachable</em> even when goodput is inconclusive — and retires one that is materially less reachable — often deciding in hours rather than days. A challenger is also retired if it <strong>starves</strong> (too few samples by 48h; zero samples aborts and frees the combo immediately) or stays inconclusive past the <strong>5-day</strong> gathering cap; anything in between holds for more data.</p>
885
885
886
886
<pstyle={{marginTop: "0.35rem"}}>Two <strong>guardrails</strong> can veto a goodput win: a challenger whose routes are getting <strong>blocked</strong> (≥ 50%) or whose connect/<strong>success rate</strong> trails the control by more than 10 percentage points is downgraded to hold — a fast-but-flaky or already-detected config never wins on speed alone. A promotion then <strong>ramps in stages</strong> (10% → 50% → 100% of free users), re-checking the blocking guardrail at each step and rolling back if it regresses. Once at 100%, the challenger becomes an ordinary track in the catalog and the bandit takes over.</p>
887
887
888
-
<pstyle={{marginTop: "0.35rem",color: "#667080"}}>Every stage is gated by settings and starts disabled. With<em>auto-act</em> off the evaluator only logs the recommendation it would make (recommend-only); turning it on lets it actually promote and retire. Watch the live pipeline, per-experiment stats, and tune these knobs on the <strong>Experiments</strong> tab.</p>
888
+
<pstyle={{marginTop: "0.35rem",color: "#667080"}}>Every stage is gated by settings and starts conservative. The learning <strong>proposer runs in shadow</strong> by default — it logs the pick it <em>would</em> make while launches still follow enumeration order — until <codestyle={{background: "rgba(255,255,255,0.06)",padding: "1px 4px",borderRadius: "3px",fontSize: "0.65rem"}}>experiment_scorer_enabled</code> is turned on. And with<em>auto-act</em> off the evaluator only logs the recommendation it would make (recommend-only); turning it on lets it actually promote and retire. Watch the live pipeline, per-experiment stats, and tune these knobs on the <strong>Experiments</strong> tab.</p>
0 commit comments