Skip to content

Activate edge + spread gate values (forward-test) — companion to #999#1001

Merged
jmoreira-valory merged 5 commits into
mainfrom
feat/polymarket-gate-activation-values
Jul 1, 2026
Merged

Activate edge + spread gate values (forward-test) — companion to #999#1001
jmoreira-valory merged 5 commits into
mainfrom
feat/polymarket-gate-activation-values

Conversation

@jmoreira-valory

@jmoreira-valory jmoreira-valory commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Ships the Polymarket edge + spread caps as committed defaults and wires them in at runtime (companion to #999, which added the gate machinery default-OFF). The backing analysis was re-run on the live execution measure (edge_ask = p_side − best_ask, 100% coverage) and independently re-verified by 5 agents; the rationale is summarized inline below. Decision: ship as a monitored production trial (reversible, KPI-safe) — see Rollout below.

Changes

(a) The caps — polymarket_trader/service.yaml, pure caps, no floors raised

The analysis is explicit that the min_edge / spread_min floors are dead knobs; the ROI lever is the upper cap. So this PR only adds caps:

param value change?
strategies_kwargs.min_edge 0.01 unchanged (floor stays low — lowering it is a no-op)
strategies_kwargs.max_edge 0.15 added (cap on overconfident high-edge bets)
POLYMARKET_SPREAD_MIN 0.0 unchanged
POLYMARKET_SPREAD_MAX 0.10 added (cap — reject the widest/most illiquid books)

(b) Runtime activation — repoint the kelly_criterion strategy CID (resolves the P1 from review)

The max_edge cap is enforced inside kelly_criterion, which the agent loads at runtime by the CID in file_hash_to_strategiesnot from the repo file. That map still pointed kelly_criterion at the pre-999 CID bafybeididlci… (whose strategy ignores max_edge), so the new cap was inert in deployment — a CLOB bet with edge > 0.15 would still pass. Repointed kelly_criterion → the #999 CID bafybeidme2u…:

Re-locked agent + both services; autonomy packages lock --check passes. The spread gate itself lives in the agent-package behaviour (decision_receive.py, runs from the repo), so it was already live — only the kelly strategy needed repointing. Caps remain env-overridable (STRATEGIES_KWARGS, POLYMARKET_SPREAD_MAX) for instant rollback.

Why this wasn't caught in #999 or the pre-merge smoke tests

Honest postmortem, because the gap is subtle and worth recording:

#999. It added the max_edge machinery to kelly_criterion.py and re-locked (new CID), but shipped it default-OFF (max_edge=1.0, inert) and never repointed file_hash_to_strategies. The strategy runs at runtime from the configured CID, so #999's new code — correct and fully unit-tested — was never wired into any deployment. Because the default was inert there was zero observable behaviour change, so nothing (CI, unit tests, reviewers) had a signal to expose the dangling wiring: by design, #999 changed nothing at runtime.

The local smoke tests all ran blind to it, for compounding reasons:

  1. Strategy code runs from an IPFS CID, not the repo. Even make run-agent loads whatever CID file_hash_to_strategies names (the old kelly) — the new code never executed in any run.
  2. The edge-reject test used the wrong knob. The forced reject used min_edge=0.99; min_edge is read by both the old and new kelly, so it rejected regardless — proving nothing about the new path. max_edge (the only discriminating knob) was never exercised with a gating value.
  3. Default max_edge=1.0 is inert — a defaults-based smoke run can't engage the cap regardless of which kelly loads.
  4. The Polymarket smoke run never reached the strategy — it stalled at the unrelated mech_information marketplace-enumeration step.
  5. The Omen smoke "confirmation" was false. It reached a decision, but Omen is FPMM (max_edge is CLOB-only), and the max_edge=1.0 line in the BET-DECISION log is printed by the decision_receive behaviour (which does run from the repo) — not the kelly strategy (which runs from IPFS). It looked like confirmation; it wasn't.
  6. 100% unit coverage tests the repo kelly_criterion.py by direct import — orthogonal to which CID the runtime loads.

Root cause / takeaway: strategy customs are deployment-decoupled from the repo — loaded by hash. A strategy-code change isn't live until its CID is (i) published to IPFS and (ii) referenced by file_hash_to_strategies; and it must be validated against the runtime strategy with a value that actually gates, not just unit-tested. This PR adds the repoint; the outstanding validation is the forward-test below.

Outstanding validation

A live forward-test (run with max_edge below the live edge and confirm the strategy rejects, reading kelly's info/rejection lines — not the behaviour's print) is still pending. It needs the new CID published to IPFS (on release) and a Polymarket run that reaches the decision (the mech_information enumeration was intermittently empty during testing). The wiring is verified by the lock; runtime gating is unit-tested.

What the analysis found

  1. Edge holds on the execution measure. The gate filters p_oracle − best_ask; re-deriving on exactly that (edge_ask = p_side − fill, 100% coverage) reproduces the band — [0.05,0.10] is the best, only positive-ROI bucket (+0.6%).
  2. The lever is the CAP, not the floor. Raising min_edge does nothing (ROI stays −5.4%). Capping high edge helps because high-edge bets are overconfident: the gap between p_side and realized win-rate rises monotonically to +0.52 (adverse selection, not a price artifact — independently verified).
  3. Ceiling is break-even, not profit. Best gate ≈ +0.6% ROI; bootstrap CI straddles 0; one of five whole-market folds reverses −9.85pp. The mechanism is robust; the dollar effect is not statistically established.
  4. Selection caveat. ~10% of bets carry negative execution edge, placed only under old configs — a forward gate never faces those.

Volume / supply (on LIVE Polymarket data)

Checked the live candidate universe (Gamma /events/keyset, ≤4-day window, + per-market CLOB book — not fleet bets): 675 markets, 294 with a 2-sided book, median live CLOB spread 0.02, ~215 pass spread ≤ 0.10 (~193 at ≤ 0.05). The spread gate is not supply-constraining. The edge cap is forecast-dependent (p_side per market), so its bet-volume effect is forward-test only. Staking KPI is unaffected — edge/spread are post-mech gates; the agent still makes every prediction, only bet volume changes.

Rollout — monitored production trial (not a validated profit claim)

This is not a "fixes ROI" change; it caps bets where the forecaster is provably overconfident, with an expected loss-reducing-to-break-even effect. Ship it as a live forward-test with a safety net:

  • Baseline: trailing Polymarket capital-weighted ROI + bets/day before activation (≈ −5.4%).
  • Monitor weekly: ROI, bets/day, win-rate vs baseline.
  • Rollback: depends on channel — an operated deployment reverts in seconds via env (POLYMARKET_SPREAD_MAX / STRATEGIES_KWARGS), but a Pearl release reaches all end-user instances, so its rollback is a follow-up release (days). Ship to Pearl knowing the safety net is a release cycle. (Defensive change → bounded downside, so all-users is still acceptable.)
  • Power note: weeks of bets catch a regression (rollback safety); confirming a positive effect needs months given the small, noisy edge — so read interim results as "not worse," not "it works."

The durable ROI fix remains upstream — calibrate the forecaster so a high p_side is actually right.

🤖 Generated with Claude Code

jmoreira-valory and others added 3 commits June 30, 2026 10:35
Companion to the edge/spread bet-selection gates (#999). Sets the
Polymarket-only activation values found most consistent in the maxedge
analysis, on top of the default-OFF gate machinery:

  strategies_kwargs.min_edge : 0.01 -> 0.05
  strategies_kwargs.max_edge : (1.0) -> 0.10   (new band upper bound)
  POLYMARKET_SPREAD_MIN      : 0.0  -> 0.02
  POLYMARKET_SPREAD_MAX      : 1.0  -> 0.05

These are FORWARD-TEST LEADS, not validated production settings:
- Neither knob is statistically significant (edge p~0.10, spread p~0.14).
- The analysis measured edge as |p_yes - market_prob| and spread from the
  Gamma snapshot; the live gates filter `p_oracle - best_ask` and live-CLOB
  `best_ask - best_bid` respectively, so the lifts may not transfer.
- min_edge 0.01 -> 0.05 sharply tightens bet selection (lower volume).

Polymarket-only (polymarket_trader/service.yaml); Omen untouched. Open as
DRAFT pending a paper/live forward-test before merge.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Correct the activation values to match the analysis: the min_edge FLOOR
is a dead knob (raising it does nothing for ROI); the lever is the CAP.

  min_edge            : 0.05 -> 0.01  (revert — keep the floor low, as in prod)
  max_edge            : 0.10           (cap — the actual ROI lever, kept)
  POLYMARKET_SPREAD_MIN : 0.02 -> 0.0  (revert — no spread floor)
  POLYMARKET_SPREAD_MAX : 0.05         (cap — reject wide/illiquid spreads, kept)

Net vs #999: add max_edge=0.1 and cap spread at 0.05. No floors raised.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…0.10

Supply check on the data lake showed the tight caps (max_edge 0.10 +
spread_max 0.05) cut bet volume from ~142/day fleet to ~2-3/day — the
spread cap especially (median Polymarket spread is 0.15, so <=0.05 keeps
only 27%). Loosen to keep enough markets while still capping the tails:

  min_edge   : 0.01  (unchanged — floor is not the constraint)
  max_edge   : 0.10 -> 0.15   (~38 bets/day fleet vs ~24)
  spread_min : 0.0   (unchanged)
  spread_max : 0.05 -> 0.10   (keeps ~43% vs 27%; still rejects widest books)

Staking KPI unaffected (edge/spread are post-mech; predictions unchanged).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jmoreira-valory jmoreira-valory marked this pull request as ready for review June 30, 2026 10:38
Base automatically changed from feat/polymarket-bet-selection-gates to main June 30, 2026 10:39
@jmoreira-valory jmoreira-valory marked this pull request as draft June 30, 2026 11:17
@jmoreira-valory jmoreira-valory marked this pull request as ready for review June 30, 2026 11:46
OjusWiZard
OjusWiZard previously approved these changes Jun 30, 2026

@dagacha dagacha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found one issue that prevents the new edge cap from taking effect.

Comment thread packages/valory/services/polymarket_trader/service.yaml
…gy CID

dagacha's P1: file_hash_to_strategies still mapped kelly_criterion to the
pre-999 CID (bafybeididlci...), whose strategy ignores max_edge — so the
max_edge=0.15 cap added by this PR was inert at runtime (the deployed
agent downloads the strategy by hash, not the repo file). Repoint to the
#999 CID (bafybeidme2u...) and re-lock.

- polymarket_trader/service.yaml: activates the max_edge cap (functional)
- trader_pearl/service.yaml: consistency only — Omen is FPMM, max_edge is
  CLOB-only and unset there, so behaviour is unchanged; this just stops
  Omen deploying the stale pre-999 strategy
- aea-config.yaml: agent default (the merged #999 left this stale too)

autonomy packages lock --check passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dagacha

dagacha commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Re-reviewed the latest diff at 036297a. The previous max_edge wiring issue is resolved by repointing kelly_criterion to the CID that consumes max_edge, and I found no new actionable issues. LGTM.

@bennyjo bennyjo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — edge cap is now correctly wired and the spread cap was already live. One non-blocking note inline on the strategy repoint.

Comment thread packages/valory/services/polymarket_trader/service.yaml
@jmoreira-valory jmoreira-valory merged commit ed39a1e into main Jul 1, 2026
28 checks passed
@jmoreira-valory jmoreira-valory deleted the feat/polymarket-gate-activation-values branch July 1, 2026 08:37
jmoreira-valory added a commit that referenced this pull request Jul 1, 2026
The "Update branch" merge combined #1001's kelly re-lock with this branch's
fixed_bet re-lock without recomputing the resulting package hashes, so
`autonomy packages lock --check` failed on the merge commit. Re-locked; check
now passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants