Skip to content

Commit 4766232

Browse files
committed
Fix research fill sizing assumptions
1 parent 13e9be3 commit 4766232

8 files changed

Lines changed: 358 additions & 103 deletions

reports/maker-simulation-tradetape-2026-05-13.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Maker Simulation v2 — Trade Tape (2026-05-13T03:50:03.951692+00:00)
22

3+
> **Post-review correction (2026-05-13)**: this report was generated before the simulator capped PnL by the thinnest at-or-below-target leg trade size and before maker quotes were forced to stay strictly below bestAsk. Treat the dollar figures below as a stale upper bound. Re-run `scripts/simulate_maker_basket_v2.py` with the corrected code before making any trade/no-trade decision.
4+
35
**Method**: real Polymarket trade tape. For each (group, day, markup), check if any SELL Yes trade at price <= target occurred on each leg that day. If ALL legs had a qualifying trade, basket fills.
46

57
**Window**: 14 days (2026-04-29 -> 2026-05-13)
@@ -64,4 +66,4 @@ v1 mid-touch results from earlier today (see `maker-simulation-2026-05-13.md`):
6466
- v2 vs v1 mismatch: v2 < v1 means mid-touch over-counts (less real trade activity at target); v2 > v1 means mid-touch under-counts (trades happened that mid-snapshot didn't capture).
6567

6668
---
67-
*Snapshot: 2026-05-13T03:50:03.951692+00:00*
69+
*Snapshot: 2026-05-13T03:50:03.951692+00:00*

reports/research-summary-2026-05-13.md

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44
**面向读者**:同学 WW
55
**与昨日报告 (`research-summary-2026-05-12.md`) 的关系**:昨日是一次 06:13 UTC snapshot 的普查结果("有 1 个 strict 候选 James Bond, edge +8.93%")。今日把数据量扩到 14 天 × 15 分钟 granularity,并且对 strict 候选做了真实 orderbook 深度检查。**结论从「thesis 待验证」推到「thesis 在当前深度下商业死亡」**
66

7+
> **Post-review correction(2026-05-13)**:§3.9 的 maker v2 dollar 结论是旧 simulator 产物;旧公式按目标 basket size 计收益,没有按每条腿真实 at-or-below-target SELL-Yes 成交量封顶,也没有强制 maker quote 严格低于 bestAsk。代码已修正,下面涉及 `$918/yr``$200-500/yr``$2-5k/yr` 的数字只能视为 stale upper bound,必须重跑 `scripts/simulate_maker_basket_v2.py` 后再决策。
8+
79
---
810

911
## TL;DR —— 30 秒结论(vs 昨日)
@@ -20,10 +22,10 @@
2022
1. mid-price 给出的"持续 edge"和 bestAsk 实际可成交 edge 是两个东西(§3.4)
2123
2. TAKER 一次性吃光 bestAsk 在 2 个测试组(James Bond + SC Gov)下死亡(§2 + §3.7)
2224
3. 我据此说"thesis 死了" —— 用户当场质疑(§3.9)
23-
4. 补做 MAKER 模拟:v1 mid-touch 给 $15k/yr 假象,v2 trade tape $918/yr 真值
24-
5. 修正后预期:**$200-500/yr @ $100 basket,或 $2-5k/yr @ $1000 basket**(capital 占用 $144k)
25+
4. 补做 MAKER 模拟:v1 mid-touch 给 $15k/yr 假象,v2 trade tape 旧公式给 $918/yr 上界
26+
5. Post-review 修正:maker v2 需要按真实成交量封顶后重跑;**`$200-500/yr @ $100 basket` 不再作为最终结论**
2527

26-
**今天 Claude 的判决(修正后)****Taker 死,Maker 活在 hobby 规模。最严重的教训是 §3.9:我用 1 个角度的测试做了全局结论,错了。用户的"不信邪" 把这份报告从错误里拉了回来**
28+
**Post-review 后的判决****Taker 基本死;Maker 不能再按旧数字下结论,必须用成交量封顶版本重跑。最严重的教训是 §3.9:我用 1 个角度的测试做了全局结论,错了;但旧 maker v2 又犯了 size 上限错误。**
2729

2830
---
2931

@@ -241,14 +243,16 @@ scripts/verify_group_book.py --group-id 0xa8574c0caacc --basket-sizes "50,200,50
241243

242244
**警告写在脚本里:mid-touch 不等于 trade-at-target。真实 fill 率会低很多**
243245

244-
#### v2 (trade tape) 修正
246+
#### v2 (trade tape) 旧公式结果(post-review 后需重跑)
245247

246248
`scripts/simulate_maker_basket_v2.py`:从 `data-api.polymarket.com/trades` 拉了真实成交记录。48,030 raw trades → 1,602 个 SELL Yes 在窗口内(只有 **3.3%** 的成交是 SELL-Yes,即"会触发我们 maker bid 的那种")。
247249

248-
| Metric | v1 (mid-touch) | v2 (trade tape) |
250+
Post-review 发现旧公式仍把每次 fill 乘以目标 basket size,没有按最薄腿真实 at-or-below-target 成交量封顶;因此本节数字只能作为旧版上界。
251+
252+
| Metric | v1 (mid-touch) | v2 (trade tape, pre-fix upper bound) |
249253
|---|---:|---:|
250254
| 总日 $ | $42.59 | **$2.51** |
251-
| 年化 | $15,546 | **$918** |
255+
| 年化 | $15,546 | **$918(旧上界)** |
252256
| 正期望组 | 49/72 | **17/72** |
253257
| 平均 fill rate | 23-69% | **5-6%** |
254258

@@ -262,21 +266,21 @@ scripts/verify_group_book.py --group-id 0xa8574c0caacc --basket-sizes "50,200,50
262266
| D/R 相关动(联合 fill 比独立 fill 难) | ×0.7 |
263267
| Partial fill 风险(一腿成 一腿没成 → 持仓不对冲) | -10% |
264268
| Polygon gas / 多笔交易成本 | -20% |
265-
| **现实估计** | **$200-500/yr @ $100 basket** |
269+
| **旧现实估计** | **无效,需按成交量封顶后重跑** |
266270

267-
如果放大到 $1000 basket~$2-5k/yr,但资金占用 $144k(72 组 × 2 腿 × $1000)
271+
旧版 "$1000 basket~$2-5k/yr" 线性外推同样无效,因为真实成交量通常远低于目标 basket size
268272

269273
#### 修正后的两层 verdict
270274

271275
| 策略 | 现实预期 \$/yr | 备注 |
272276
|---|---:|---|
273277
| Taker basket arb | \$0-200 | 被深度杀死,verified |
274278
| Maker basket arb(mid-sim 错估) | \$15k 假象 | 方法错 |
275-
| **Maker basket arb(trade tape)** | **\$200-500 @ \$100 / \$2-5k @ \$1000** | 方法学上可辩护 |
279+
| **Maker basket arb(trade tape)** | **需重跑** | 代码已改成成交量封顶 + 非 crossing maker quote |
276280

277281
#### 我学到的最严肃的教训
278282

279-
我前两天说 "thesis 已死" 是**过度推论**。我只测了 1 个视角(TAKER 一次性吃光 bestAsk),用 2 个组的单次 snapshot 就下了"整条 thesis 死亡"的判决。**用户当面质疑后做的真测试(trade tape v2)证明 thesis 活着,只是商业规模上接近 hobby**
283+
我前两天说 "thesis 已死" 是**过度推论**。我只测了 1 个视角(TAKER 一次性吃光 bestAsk),用 2 个组的单次 snapshot 就下了"整条 thesis 死亡"的判决。**用户当面质疑后补做的 trade tape v2 提示 maker 方向仍值得验证,但 post-review 后必须用成交量封顶版本重跑,不能再把旧收益数当结论**
280284

281285
更广义的教训:**"测一个角度 → 推全局"** 是科研里最廉价的错误之一。Robust 测试需要至少:
282286
- 多种策略视角(taker / maker / hold-to-resolution)
Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
"""Shared helpers for research-only simulation scripts."""
2+
from __future__ import annotations
3+
4+
import statistics
5+
from typing import Any
6+
7+
DEFAULT_TICK_SIZE = 0.001
8+
EPSILON = 1e-9
9+
10+
11+
def simulate_buy_cost(asks: list[tuple[float, float]], target_units: float) -> tuple[float, float, float]:
12+
"""Walk an ask ladder and return (units_filled, total_cost, avg_price_paid)."""
13+
filled = 0.0
14+
cost = 0.0
15+
for price, size in asks:
16+
if filled >= target_units:
17+
break
18+
take = min(float(size), target_units - filled)
19+
if take <= 0:
20+
continue
21+
cost += take * float(price)
22+
filled += take
23+
avg_price = cost / filled if filled > 0 else 0.0
24+
return filled, cost, avg_price
25+
26+
27+
def fee_rate_from_book(book: dict[str, Any]) -> float:
28+
member = book.get("member") or {}
29+
return float(book.get("fee_rate", member.get("fee_rate", 0.0)) or 0.0)
30+
31+
32+
def simulate_basket_fill(book_data: list[dict[str, Any]], requested_size: float) -> dict[str, Any]:
33+
"""Simulate a mutually-exclusive YES basket at the common executable size.
34+
35+
A basket only pays out for the minimum size completed across all legs. If
36+
one leg has 2 units of ask depth and the requested basket is 10 units, the
37+
executable basket is 2 units, not 10.
38+
"""
39+
requested_size = float(requested_size)
40+
requested_fills: list[float] = []
41+
for book in book_data:
42+
filled, _, _ = simulate_buy_cost(book.get("asks") or [], requested_size)
43+
requested_fills.append(filled)
44+
45+
effective_size = min(requested_fills) if requested_fills else 0.0
46+
effective_size = max(0.0, min(effective_size, requested_size))
47+
48+
total_cost = 0.0
49+
total_fee = 0.0
50+
per_member: list[dict[str, Any]] = []
51+
if effective_size > EPSILON:
52+
for book in book_data:
53+
filled, cost, avg_px = simulate_buy_cost(book.get("asks") or [], effective_size)
54+
fee_rate = fee_rate_from_book(book)
55+
fee = fee_rate * avg_px * (1.0 - avg_px) * filled
56+
total_cost += cost
57+
total_fee += fee
58+
member = book.get("member") or {}
59+
per_member.append(
60+
{
61+
"member": str(member.get("question") or "")[:40],
62+
"filled": filled,
63+
"avg_px": avg_px,
64+
"cost": cost,
65+
"fee": fee,
66+
}
67+
)
68+
69+
edge_dollars = effective_size - total_cost - total_fee
70+
edge_pct = edge_dollars / effective_size if effective_size > EPSILON else 0.0
71+
return {
72+
"size": requested_size,
73+
"requested_size": requested_size,
74+
"effective_size": effective_size,
75+
"max_fillable_units": effective_size,
76+
"is_full_size_fillable": effective_size + EPSILON >= requested_size,
77+
"total_cost": total_cost,
78+
"total_fee": total_fee,
79+
"edge_dollars": edge_dollars if effective_size > EPSILON else 0.0,
80+
"edge_pct": edge_pct,
81+
"per_member": per_member,
82+
}
83+
84+
85+
def maker_target_price(
86+
best_bid: float,
87+
best_ask: float,
88+
markup: float,
89+
tick_size: float = DEFAULT_TICK_SIZE,
90+
) -> float | None:
91+
"""Return a non-crossing maker bid target or None if the spread is too tight."""
92+
best_bid = float(best_bid)
93+
best_ask = float(best_ask)
94+
markup = float(markup)
95+
tick_size = float(tick_size)
96+
if best_ask <= 0 or best_bid < 0 or tick_size <= 0:
97+
return None
98+
lower = best_bid + tick_size
99+
upper = best_ask - tick_size
100+
if upper + EPSILON < lower:
101+
return None
102+
target = max(best_ask - markup, lower)
103+
target = min(target, upper)
104+
if target <= 0 or target + EPSILON >= best_ask:
105+
return None
106+
return round(target, 6)
107+
108+
109+
def zero_maker_stats(n_total_days: int, reason: str) -> dict[str, Any]:
110+
return {
111+
"targets": [],
112+
"n_filled_days": 0,
113+
"n_total_days": n_total_days,
114+
"fill_rate": 0.0,
115+
"avg_edge_given_fill": 0.0,
116+
"median_edge_given_fill": 0.0,
117+
"expected_daily_edge_dollars": 0.0,
118+
"avg_min_leg_sell_size": 0.0,
119+
"avg_effective_basket_size": 0.0,
120+
"max_effective_basket_size": 0.0,
121+
"n_positive_edge_days": 0,
122+
"n_negative_edge_days": 0,
123+
"skipped_reason": reason,
124+
}
125+
126+
127+
def capped_expected_daily_edge(
128+
filled_days: list[dict[str, Any]],
129+
n_total_days: int,
130+
basket_size: float,
131+
) -> dict[str, float]:
132+
"""Compute daily maker PnL capped by observed trade size on the thinnest leg."""
133+
if n_total_days <= 0 or not filled_days:
134+
return {
135+
"expected_daily_edge_dollars": 0.0,
136+
"avg_effective_basket_size": 0.0,
137+
"max_effective_basket_size": 0.0,
138+
}
139+
140+
basket_size = float(basket_size)
141+
effective_sizes = [
142+
min(basket_size, max(0.0, float(day.get("min_leg_sell_size") or 0.0)))
143+
for day in filled_days
144+
]
145+
pnl = [
146+
float(day.get("edge") or 0.0) * size
147+
for day, size in zip(filled_days, effective_sizes)
148+
]
149+
return {
150+
"expected_daily_edge_dollars": sum(pnl) / n_total_days,
151+
"avg_effective_basket_size": statistics.mean(effective_sizes) if effective_sizes else 0.0,
152+
"max_effective_basket_size": max(effective_sizes) if effective_sizes else 0.0,
153+
}
154+
155+
156+
def qualifying_trade_size(trades: list[dict[str, Any]], target_price: float) -> float:
157+
"""Return total sell size that could have hit a resting bid at target_price."""
158+
target_price = float(target_price)
159+
total = 0.0
160+
for trade in trades:
161+
try:
162+
price = float(trade.get("price") or 0.0)
163+
size = float(trade.get("size") or 0.0)
164+
except (TypeError, ValueError):
165+
continue
166+
if price <= target_price and size > 0:
167+
total += size
168+
return total

scripts/simulate_maker_basket.py

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,8 @@
5353
from urllib.parse import urlencode
5454
from urllib.request import Request, urlopen
5555

56+
from research_simulation_utils import maker_target_price, zero_maker_stats
57+
5658
REPO_ROOT = Path(__file__).resolve().parent.parent
5759
GAMMA_MARKETS_URL = "https://gamma-api.polymarket.com/markets"
5860
PRICES_HISTORY_URL = "https://clob.polymarket.com/prices-history"
@@ -232,13 +234,17 @@ def main() -> int:
232234
# For each markup level, compute basket fill rate + avg basket edge
233235
markup_stats: dict[float, dict] = {}
234236
for markup in markups:
235-
# Maker target price per leg = today's bestAsk - markup (clamped >= bestBid)
237+
# Maker target price per leg must stay inside the spread and below bestAsk.
236238
targets: list[float] = []
237239
for m in members:
238-
t = m["best_ask"] - markup
239-
# Don't go below today's bestBid (would never realistically fill)
240-
t = max(t, m["best_bid"] + 0.001)
240+
t = maker_target_price(m["best_bid"], m["best_ask"], markup)
241+
if t is None:
242+
targets = []
243+
break
241244
targets.append(t)
245+
if not targets:
246+
markup_stats[markup] = zero_maker_stats(len(all_days), "no_non_crossing_maker_quote")
247+
continue
242248

243249
filled_days: list[dict] = []
244250
for d in all_days:
@@ -314,7 +320,8 @@ def best_markup_income(r: dict) -> float:
314320
f"# Maker-strategy Basket Simulation ({iso})",
315321
"",
316322
f"**Method**: for each (group, UTC-day, markup-level), check if every leg's "
317-
f"mid-price touched (today's bestAsk - markup) at some point during the day. "
323+
f"mid-price touched a non-crossing maker target derived from today's bestAsk - markup "
324+
f"at some point during the day. "
318325
f"If ALL legs filled, compute basket cost at maker target prices + fee. "
319326
f"Aggregate fill_rate * avg_edge as proxy for expected daily $income.",
320327
"",

0 commit comments

Comments
 (0)