Skip to content

Commit 49d1064

Browse files
xjxu21claude
andcommitted
Restructure into generation + evaluation; ship arena inputs and adapter skill
The repo covered only scoring; the paper's other contribution — driving any model from one action vocabulary — had no artifact. Now both halves ship. Generation (new): - arena_inputs/: the read-only benchmark inputs (100 images across view x domain, per-image action assignments, intrinsics, captions) - .claude/skills/world-model-adapter/ + AGENTS.md: an agent skill that writes the adapter and batch-renders, given a model repo and a (view, domain); works for Claude Code and Codex from the same Markdown - generation/check_delivery.py: the acceptance gate. Decodes the mp4s and checks naming, completeness, duration and consistency, so a malformed delivery is caught before it silently produces wrong scores. Verified end to end against a synthesized 125-video delivery. Evaluation: moved unchanged to evaluation/ (run_eval.sh resolves its package via $HERE, so nothing internal changed). Docs: the root README is now only "what this is / how to generate / how to evaluate". Metric definitions moved to docs/METRICS.md, scoring setup to evaluation/README.md, the I/O contract to the skill's output_spec.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 25732bb commit 49d1064

239 files changed

Lines changed: 1573 additions & 102 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
---
2+
name: world-model-adapter
3+
description: Adapt a new world-model / camera-controlled video-generation repo to the WorldMark arena inputs, and batch-render the videos. Use when the user points at an inference repo (Matrix-Game, LingBot, GameCraft, Sana-WM, Lyra, YUME, AlayaWorld, DreamX, WorldPlay, or any new one) and wants to generate the benchmark videos, or asks how to convert the benchmark's WSADLR action protocol into that model's native control format.
4+
---
5+
6+
# World-model adapter
7+
8+
Take `arena_inputs/` (starting images + per-image intrinsics + action assignments) and produce
9+
one mp4 per (image × action) in that model's native control format.
10+
11+
The investigation is yours to figure out. The **I/O contract** and the **verification gates**
12+
are not negotiable.
13+
14+
## Required from the user — exactly two things
15+
16+
1. **The model repo** — a local path or a URL to clone. Everything about the control format is
17+
derived from it by reading the source; nothing is assumed from the model's name.
18+
2. **Which view and which domain** — the unit of work is one **(view, domain)** pair
19+
= 125 videos. `view``first_view` / `third_view`, `domain``real` / `style`.
20+
21+
**If either is missing, ask before doing anything else.** Do not start probing without the
22+
repo, and do not start G4 without knowing the scope.
23+
24+
Interpreting a partial answer:
25+
26+
| They said | Render |
27+
|---|---|
28+
| a specific pair ("first-person real", "第一视角 real") | that pair only |
29+
| a view only ("first_view") | `real` + `style` for that view |
30+
| a domain only ("style") | both views for that domain |
31+
| "all" / "全部" | all four = **500 videos**, hours of multi-GPU — say the cost, then confirm |
32+
33+
Gates G0–G3 only ever need **one** pair. Run them on the user's first-choice pair and report
34+
before expanding.
35+
36+
## Language
37+
38+
Converse in the user's language. Artifacts follow fixed rules regardless:
39+
40+
- `manifest.jsonl` field **names** and enum values: English, exactly as in `output_spec.md §3`
41+
— this is a machine contract, never localise the keys.
42+
- Filenames, paths, `model_id`, `MODEL` tags: ASCII, as specified.
43+
- `PROBE.md` / `README.md` prose: the user's language is fine, but keep every `file:line`
44+
citation, identifier, and code snippet verbatim from the source.
45+
46+
## Hard rules
47+
48+
1. **Every claim about the repo needs `file:line` evidence.** Never infer behaviour from a
49+
variable name, a README sentence, or "how other repos do it". Read the consumer of the
50+
value. Past defects came almost entirely from plausible-looking assumptions.
51+
2. **Do not guess key semantics.** `a`/`d` mean yaw in some repos and strafe in others. Find
52+
and quote the mapping table.
53+
3. **No GPU until gates G0–G1 pass.** A full run is hours of multi-GPU time.
54+
4. **Never edit `arena_inputs/`.** Repo edits go in `patches/` as patch files, and may change
55+
only *how control input gets in* — never the conditioning semantics, magnitudes, or
56+
constants (see `output_spec.md §6`). Changing those needs the user's approval.
57+
5. **Run `generation/check_delivery.py` yourself and paste its output before reporting done.**
58+
A delivery that does not exit 0 is not a delivery. Never report success on the strength of
59+
your own bookkeeping — the gate decodes the videos, you do not get to vouch for them.
60+
6. If something cannot be determined from source, write `UNVERIFIED` in `PROBE.md` and tell
61+
the user. Do not fill the gap with a guess.
62+
63+
## Workflow
64+
65+
### 1. Read the inputs first
66+
67+
`arena_inputs/action_protocol.txt` is authoritative for the action semantics. For the file
68+
layout, **list the directories yourself**`README.txt` has been stale before (it still
69+
understates which prompt files ship).
70+
71+
**[references/output_spec.md](references/output_spec.md) §1** records the layout and the
72+
verified facts about the data — including three things that catch people out:
73+
74+
- intrinsics are **per-image, in original-image pixels** — but only a genuinely
75+
camera-conditioned model consumes them; classify the model into one of the four cases in §4
76+
first (real FOV spans 22°–90°, so a hardcoded-FOV model's mismatch is worth recording);
77+
- all four `prompt_*.txt` exist and **differ by view** (captions are prefixed
78+
`First-person view.` / `Third-person view.`) — never cross views or domains (§5);
79+
- image aspect ratios are not uniform (1.00 for `style`, 1.33–1.79 for `real`).
80+
81+
### 2. Probe
82+
83+
Work through **[references/probe_checklist.md](references/probe_checklist.md)** and write
84+
`adapters/<model_id>/PROBE.md`. Every item gets evidence or `UNVERIFIED`.
85+
86+
Opening sweep (adjust to the repo):
87+
88+
```bash
89+
# control surface
90+
grep -rn "add_argument" --include="*.py" . | grep -iE "action|camera|pose|traj|motion|control|prompt"
91+
# native primitives
92+
grep -rln "plucker\|Plucker\|ray_condition\|rays_d\|prope\|PRoPE" --include="*.py" .
93+
grep -rn "c2w\|w2c\|intrinsic\|extrinsic" --include="*.py" . | head -30
94+
grep -rniE "keyboard|mouse|wasd|action_dict|KEY_TO|VALUE_MAP" --include="*.py" .
95+
# time / scale constants
96+
grep -rniE "fps|move_speed|rotate_speed|_SPEED|OFFSET|SENSITIVITY|duration" --include="*.py" .
97+
# batch blockers
98+
grep -rn "sys.exit\|exit()\|destroy_process_group" --include="*.py" .
99+
```
100+
101+
Read **[references/known_traps.md](references/known_traps.md)** before concluding. Every entry
102+
is a mistake that has actually been made; each maps to a checklist item.
103+
104+
### 3. Build
105+
106+
Produce the deliverables in **output_spec §6**: `PROBE.md`, `adapt.py` (CPU-only),
107+
`run_batch.py` (resumable), `test_golden.py`, `patches/`, `README.md`.
108+
109+
First decide the **integration level** (output_spec §6) — many repos cannot be driven purely
110+
from outside:
111+
112+
- **A** a batchable file/CLI control surface already exists → no repo changes;
113+
- **B** control input is interactive-only or otherwise unbatchable → patch in a file/flag path
114+
that feeds the *same* internal call;
115+
- **C** per-sample inference is not separable from loading → patch, and have `run_batch.py`
116+
import the pipeline class directly instead of shelling out to the repo's CLI.
117+
118+
B and C are normal. State the level in `PROBE.md` and `README.md`.
119+
120+
### 4. Verify — gates in order
121+
122+
| Gate | Cost | Catches |
123+
|------|------|---------|
124+
| **G0** `test_golden.py` | none | mapping / timing / intrinsics-math regressions |
125+
| **G1** `adapt.py --dry-run` + trajectory assertions | none | sign errors, wrong axis, wrong units |
126+
| **G2** 1 sample, shortest action (id 1–5) | minutes | crashes, env, arg errors, length mismatch |
127+
| **G3** watch that 1 video | minutes | **sign errors that pass every numeric test** |
128+
| **G4** full 125 per (view, domain), resumable | hours | — (scope: see top of file) |
129+
| **G5** `generation/check_delivery.py` | seconds | naming, missing/extra files, undecodable video, out-of-tolerance duration |
130+
131+
G3 is not optional. A flipped yaw sign satisfies G0–G2 and yields a video turning the wrong
132+
way. Watch it, or have the user watch it, before G4.
133+
134+
For G1 assert properties, not eyeballed numbers: during `W` the camera position advances
135+
monotonically along the forward axis; during `R` accumulated yaw is monotone and translation
136+
~0; if the world is y-up the y component stays ~0. Record sampled poses in `PROBE.md`.
137+
138+
**G5 is the acceptance gate and it decides.** It decodes the delivered mp4s and ignores any
139+
manifest, so it cannot be talked around. Run it after G2 on the single sample, and again after
140+
G4 on the full set; a delivery that does not exit 0 is not delivered.
141+
142+
```bash
143+
python generation/check_delivery.py \
144+
--arena arena_inputs/ --delivery real/first/<MODEL> \
145+
--view first_view --domain real
146+
```
147+
148+
If a duration genuinely cannot land inside the default ±10 % because of the model's rollout
149+
granularity, do **not** silently widen `--tolerance`: report the measured deviation to the user
150+
and let them decide the tolerance for that model.
151+
152+
### 5. Report
153+
154+
Concisely tell the user:
155+
- native format, and the exact string/file the model receives for one example action
156+
- achieved resolution / fps / frames / seconds vs. the 20 s-per-segment target, with deviation
157+
- which of the four intrinsics cases (§4) this model is, and any FOV mismatch
158+
- prompt handling (used verbatim / truncated / ignored)
159+
- anything `UNVERIFIED`
160+
- anything that makes cross-model comparison unsafe (traps 5, 10, 14)
161+
162+
## Conventions
163+
164+
- One `model_id` = one adapter. Two code paths in the same repo with different control
165+
semantics (this happens) are two adapters, or one adapter with an explicit documented switch.
166+
- Resolution and fps are **deliberately not unified** across models — evaluation resizes and
167+
time-normalises. Your job is to record the actuals, not to match a target.
168+
- Long runs: progress lives in `jobs.jsonl` on disk, never in conversation context.
Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
# Known traps
2+
3+
Every entry below is a mistake that has actually been made, or a hazard found by reading
4+
source that contradicted a reasonable assumption. Each maps to a checklist item.
5+
Read this before concluding the probe.
6+
7+
---
8+
9+
## 1. Letter semantics are not portable → B2
10+
11+
The same letter means different things across repos. Observed: one repo maps `a`/`d` to
12+
**yaw rotation** and `j`/`l` to **strafe translation**; several others do the exact opposite.
13+
Another repo uses `a`/`d` for translation and spells rotation as `left_rot`/`right_rot`.
14+
15+
An adapter that copies a letter string from a sibling repo produces a video that moves when
16+
it should turn — and passes every numeric test.
17+
18+
**Always grep the key→action dict and paste it into `PROBE.md`.**
19+
20+
## 2. A parameter named "speed" may not be a speed → C1, D1
21+
22+
Observed: a repo whose JSON takes `action_speed_list: [4, 6]` alongside `action_seq`.
23+
Reading the consumer showed those numbers are **relative frame-count weights**
24+
(`weight / total_weight * num_frames`); the actual motion magnitude came from a default
25+
keyword argument elsewhere in the call chain.
26+
27+
**Trace every control number to the function that consumes it.** The same repo had a second
28+
entry point where a similarly-named argument *was* a magnitude — see trap 7.
29+
30+
## 3. Duration units differ, and the unit is often not "frame" → C1
31+
32+
Observed units for the same-looking `<action>-<number>` syntax:
33+
pixel frames · **latents** (×4 pixel frames) · fixed-length action blocks · 2-second text
34+
lines · autoregressive chunks.
35+
36+
`w-31` meaning 31 latents = 124 frames, not 31 frames, is a 4× duration error.
37+
38+
## 4. README fps ≠ config fps ≠ writer fps → C2
39+
40+
Observed in the same repo: README says 25 FPS, config constant says `sample_fps = 16`,
41+
and the actual `export_to_video(..., fps=17)` call writes 17. Wall-clock duration is set by
42+
the **writer**; the config value may only affect training-time conditioning.
43+
44+
**Confirmed empirically**: one model whose README documents 24 fps produced benchmark videos
45+
at **19 fps** (1145 frames ≈ 60.3 s). Had the plan trusted the README, the requested duration
46+
would have been wrong by 26%.
47+
48+
Record all values you find, state which governs duration, and **measure the produced file**
49+
rather than trusting any of them.
50+
51+
## 5. Absolute motion scale may be discarded before conditioning → D3
52+
53+
Observed in more than one repo: per-frame translations are divided by their max norm before
54+
being turned into the geometric condition. Consequence: a trajectory at speed 0.05/frame and
55+
one at 0.5/frame produce **identical** conditioning if the shape is the same.
56+
57+
Where this happens, **absolute speed is not controllable** through the geometry path — only
58+
trajectory shape and *relative* speed variation within a clip. If your benchmark claims
59+
matched speeds across models, this must be disclosed. Some repos feed the raw action vector
60+
through a second path, which partially restores magnitude information; check whether both
61+
paths exist.
62+
63+
## 6. First chunk length ≠ later chunk length → C4
64+
65+
Observed: first clip 57 frames, every subsequent clip 40. Total is
66+
`57 + (N-1)*40`, not `N*40`. Segments therefore cannot all be exactly equal; pick a policy
67+
and record it (see `output_spec.md`).
68+
69+
## 7. Two code paths, same-looking control, different semantics → A3
70+
71+
Observed: one repo ships an autoregressive-forcing entry point where the per-segment number
72+
is a frame-count weight, **and** a separate helper where the analogous number is a magnitude
73+
with a fixed 33-frame duration, **and** the two use different hardcoded intrinsics.
74+
75+
Determine which path the launch script actually calls. Never mix constants across paths.
76+
77+
## 8. Repo bugs that only fire in multi-GPU or looped use → F1, F5
78+
79+
Observed:
80+
- a keyword-argument name mismatch in the sequence-parallel attention wrapper — single-GPU
81+
path never touches it, any multi-GPU run crashes immediately;
82+
- `destroy_process_group()` + `exit()` at the end of the per-sample generate function — a
83+
batch loop renders exactly one sample and the process exits with no error;
84+
- a pinned dependency incompatible with the installed torch, which only surfaces at import.
85+
86+
Fix in `patches/`, and check the patch is applied (`git diff --stat`) at the start of every
87+
run rather than assuming.
88+
89+
## 9. Intrinsics: classify before you inject → E1–E3
90+
91+
Observed hardcoded values implying FOVs of ~63°, ~79°, ~89° and exactly 90° across different
92+
repos, some normalised and some in pixels, some rescaled internally against a **fixed
93+
reference resolution** rather than the actual working one.
94+
95+
Two symmetric mistakes:
96+
97+
- **Injecting where nothing is wanted.** Only a genuinely camera-conditioned model
98+
(case 4 in `output_spec.md §4`) consumes `arena_inputs/*_intrinsics/`. A model with no
99+
camera model, a hardcoded/derived virtual pinhole, or its own runtime estimator needs
100+
nothing from us — do not manufacture an injection point, and do not patch a hardcoded
101+
constant unless the user asks.
102+
- **Passing ours through raw.** Our values are in **original-image pixels**
103+
(`cx ≈ (W-1)/2`). Every model resizes first, and often changes aspect ratio. Scale and crop
104+
the intrinsics to match, and check whether the repo wants pixels or normalised values, and
105+
against which resolution.
106+
107+
Either way, record the source FOV and the model's effective FOV. Our data spans **22°–90°**
108+
horizontal, so a fixed-FOV model is mismatched for most samples; that is a disclosable caveat,
109+
not something to silently fix.
110+
111+
## 10. Motion may be eased, not step-constant → D4
112+
113+
Observed: target velocity approached by exponential smoothing with separate press/release
114+
time constants (~0.45 s / ~1.0 s). Even with an exactly correct frame count, the first
115+
fraction of a second of each segment is still accelerating and the previous action is still
116+
coasting.
117+
118+
Consequence for evaluation: segment boundaries are not sharp. Prefer discarding a fixed
119+
transition window at the start of each segment **for all models**, so the eased model is not
120+
penalised relative to step-constant ones.
121+
122+
## 11. "It uses X, so it must not use Y" → G1
123+
124+
A conclusion of the form "this model is action-conditioned, therefore it has no camera
125+
geometry" was asserted and turned out to be **wrong**: the repo integrated the action stream
126+
into camera poses and built a ray-map condition, *in addition* to a separate action module.
127+
128+
Injection mechanisms are not mutually exclusive. One repo used an additive ray-map embedding
129+
**and** a projection-style camera position encoding inside attention. Grep for all of them
130+
before characterising a model, and prefer `UNVERIFIED` over a tidy story.
131+
132+
## 12. Example files are ground truth; README prose is a hint
133+
134+
Where a repo ships an example input, load it and print shapes, dtypes and a few values.
135+
Observed disagreements between README description and shipped file, and one case where the
136+
example's stated image dimensions were only a reference for intrinsics, not the output
137+
resolution.
138+
139+
## 13. Package name ≠ import name; installing one thing un-installs another
140+
141+
Environment hazards seen repeatedly: a drop-in replacement package that provides the same
142+
import name but a different distribution name, so the dependency resolver pulls the original
143+
back in; a headless variant of a library being silently replaced by the GUI variant as a
144+
transitive dependency of an unrelated install.
145+
146+
After any install into a model env, re-check the packages your run depends on rather than
147+
assuming the env is still good.
148+
149+
## 14. Prompts are per (view, domain, image) and not interchangeable → H1
150+
151+
There are four caption files — `{first_view,third_view}/prompt_{real,style}.txt`, 25 rows each.
152+
Every `first_view` caption is prefixed `First-person view.` and every `third_view` one
153+
`Third-person view.`, so the caption itself encodes the camera setup.
154+
155+
Failure modes seen or easily reachable:
156+
157+
- loading one prompt file and reusing it across views → the model is told "third-person" while
158+
being handed a first-person image;
159+
- off-by-one row indexing → every video gets its neighbour's caption, which is nearly invisible
160+
in spot checks and poisons any text-alignment metric;
161+
- an adapter "improving" or regenerating captions → the text then carries scene information the
162+
benchmark never supplied, making that domain incomparable.
163+
164+
Use `{view}/prompt_{domain}.txt` row `i` for image `i`, verbatim, and record
165+
`prompt.source` as `file:row` in the manifest so it is auditable.
166+
167+
Note the zip's own `README.txt` is stale here (it claims only `third_view` has prompts) — see
168+
trap 12.
169+
170+
## 15. Inputs are not uniform in size or aspect ratio → E5, H3
171+
172+
Measured: `real` images span 1024×768 … 1920×1200 (AR 1.33–1.79); `style` is uniformly
173+
1024×1024 (AR 1.00). Target models generate at ~1.7–1.8 AR.
174+
175+
So every sample undergoes a non-trivial resize, and square `style` inputs need a real
176+
crop/pad decision. An adapter that hardcodes one scale factor, or assumes a fixed input size,
177+
will be wrong for most of the suite. Decide one policy per model, apply it uniformly, document
178+
it.

0 commit comments

Comments
 (0)