Skip to content

Commit 3ef8b7f

Browse files
xjxu21claude
andcommitted
Open with "What is WorldMark?"; add teaser to README
- Merge Why WorldMark into a single opening section alongside the teaser figure: short intro, the two obstacles, and the paper's three contributions verbatim - Drop the Qualitative Examples header text; the three group headings now carry the section - Reorder navs to document order - README: teaser figure with caption under the links Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 6068000 commit 3ef8b7f

2 files changed

Lines changed: 40 additions & 30 deletions

File tree

README.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,17 @@
44
**[Paper](https://arxiv.org/abs/2604.21686)** ·
55
**[World Model Arena](https://warena.ai/)**
66

7+
<p align="center">
8+
<img src="docs/static/images/teaser.png" width="620" alt="WorldMark overview: per-model adapters translate one shared action vocabulary into each model's native control format; a round-trip probe compares outbound and return views at equal accumulated motion.">
9+
</p>
10+
11+
<p align="center"><sub>
12+
Models take actions in incompatible formats — keyboard actions, text prompts, camera
13+
trajectories — so per-model adapters translate one shared vocabulary into each native
14+
format. For a frame on the outbound leg and its return-leg counterpart at equal
15+
accumulated motion, the two views should match but diverge to different degrees.
16+
</sub></p>
17+
718
A benchmark for **interactive world models** that scores generated videos along
819
three complementary axes — **Action Dynamics**, **World Memory**, and
920
**Visual Quality** — for a total of **9 metrics**. All metrics are

docs/index.html

Lines changed: 29 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -313,7 +313,8 @@
313313
<div class="topbar-inner">
314314
<a class="brand" href="#top"><img src="static/images/favicon.png" alt="">WorldMark</a>
315315
<div class="topnav">
316-
<a href="#comparison">Why WorldMark</a>
316+
<a href="#overview">What is WorldMark</a>
317+
<a href="#examples">Examples</a>
317318
<a href="#suite">Benchmark</a>
318319
<a href="#metrics">Metrics</a>
319320
<a href="#results">Results</a>
@@ -373,24 +374,42 @@ <h1><span class="wm">WorldMark</span>: A Unified Benchmark Suite for Interactive
373374

374375
<!-- ── Teaser ── -->
375376
<section class="wrap" id="overview" style="padding-top: 28px;">
377+
<p class="kicker">What is WorldMark?</p>
378+
<h2>The user acts, and the world responds</h2>
379+
<p>
380+
An interactive world model is not a video generator: it is an environment. You press a key, and the world
381+
is supposed to move that way, keep moving while you hold it, stop when you let go, and still be there when
382+
you turn back. <span class="wm">WorldMark</span> is a benchmark that measures exactly that.
383+
</p>
384+
<p>
385+
It drives <strong>ten heterogeneous models</strong>&mdash;caption-, pose-, keyboard-, and trajectory-controlled&mdash;from
386+
<strong>one shared WASD-style action vocabulary</strong> over <strong>500 standardized cases</strong>, and scores the
387+
result with <strong>nine deterministic metrics</strong> covering action dynamics, world memory, and visual quality.
388+
</p>
376389
<figure>
377390
<div class="figframe"><img src="static/images/teaser.png" alt="WorldMark overview: per-model adapters translate one shared action vocabulary into each model's native control format; a round-trip probe compares outbound and return views at equal accumulated motion."></div>
378391
<figcaption>
379392
Models take actions in incompatible formats&mdash;keyboard actions, text prompts, camera trajectories&mdash;so per-model adapters translate one shared vocabulary into each native format.
380393
For a frame on the outbound leg and its return-leg counterpart at equal accumulated motion, the two views should match but diverge to different degrees.
381394
</figcaption>
382395
</figure>
396+
397+
<p class="blist-head">Two obstacles to fair evaluation</p>
398+
<ul class="blist bad">
399+
<li><b>Incompatible action formats.</b> Captions, trajectories, pose strings, action functions&mdash;no shared protocol, so no two models get identical instructions.</li>
400+
<li><b>Action following collapsed into one number.</b> Trajectory or direction error shows neither how fast the world reacts to a command switch, nor how cleanly it moves along the commanded axis.</li>
401+
</ul>
402+
403+
<p class="blist-head">Our contributions</p>
404+
<ul class="blist good">
405+
<li><b>A unified action interface</b> that drives ten models with incompatible control formats from a single action vocabulary, together with a standardized suite of 500 cases spanning visual styles, viewpoints, and difficulty tiers; supporting a new model requires one adapter.</li>
406+
<li><b>An action-centric metric suite</b> that treats each command as a <em>step input</em>, after control-systems practice, and characterizes <em>how</em> a world responds&mdash;when motion begins, whether it stays on the commanded axis, and whether it holds&mdash;rather than only how far it strays, resolved per motion axis and complemented by memory metrics at three timescales and two visual quality scores.</li>
407+
<li><b>A systematic evaluation</b> that turns these measurements into actionable findings: a response&ndash;stability trade-off no single action metric can capture, command axes that fail independently, and a negative correlation between perceptual quality and responsiveness that contradicts appearance-based evaluation.</li>
408+
</ul>
383409
</section>
384410

385411
<!-- ── Qualitative video examples ── -->
386412
<section class="wrap" id="examples">
387-
<p class="kicker">Qualitative Examples</p>
388-
<h2>What each metric actually sees</h2>
389-
<p class="muted">
390-
A high- and a low-scoring model per metric, on the <strong>same scene under the same command</strong>&mdash;the only
391-
difference is the model. The on-screen keys show the command being issued.
392-
</p>
393-
394413
<h3 class="ex-h">Action Dynamics <span>Does the world go where it is told&mdash;and when?</span></h3>
395414
<div class="ex-tabs" data-group="action" role="tablist">
396415
<button class="ex-tab active" data-group="action" data-ex="1">Direction Accuracy</button>
@@ -486,26 +505,6 @@ <h3 class="ex-h">Visual Quality <span>How does each frame look?</span></h3>
486505

487506
</section>
488507

489-
<!-- ── Comparison ── -->
490-
<section class="wrap" id="comparison">
491-
<p class="kicker">Why WorldMark</p>
492-
<h2>The user acts, and the world responds</h2>
493-
494-
<p class="blist-head">Two obstacles to fair evaluation</p>
495-
<ul class="blist bad">
496-
<li><b>Incompatible action formats.</b> Captions, trajectories, pose strings, action functions&mdash;no shared protocol, so no two models get identical instructions.</li>
497-
<li><b>Action following collapsed into one number.</b> Trajectory or direction error shows neither how fast the world reacts to a command switch, nor how cleanly it moves along the commanded axis.</li>
498-
</ul>
499-
500-
<p class="blist-head">How <span class="wm">WorldMark</span> removes them</p>
501-
<ul class="blist good">
502-
<li><b>One vocabulary, ten native interfaces.</b> Per-model adapters deliver semantically identical instructions across 500 standardized cases. A new model costs one adapter.</li>
503-
<li><b>Action as a step response.</b> Direction accuracy, purity, latency, and stability&mdash;each resolved per motion axis&mdash;plus world memory and visual quality.</li>
504-
<li><b>Deterministic by construction.</b> Feed-forward estimators only: bit-identical across runs, no SLAM or sampled-VLM variance.</li>
505-
<li><b>Differences one score cannot see.</b> The fastest responders are often the least stable, and the best-looking model is among the least responsive.</li>
506-
</ul>
507-
508-
</section>
509508

510509
<!-- ── Benchmark suite ── -->
511510
<section class="wrap" id="suite">
@@ -830,11 +829,11 @@ <h2>BibTeX</h2>
830829

831830
<!-- ── Right side nav ── -->
832831
<ul class="side-nav">
833-
<li><a href="#comparison">Why WorldMark</a></li>
832+
<li><a href="#overview">What is WorldMark</a></li>
833+
<li><a href="#examples">Examples</a></li>
834834
<li><a href="#suite">Benchmark Suite</a></li>
835835
<li><a href="#metrics">Metrics</a></li>
836836
<li><a href="#results">Results</a></li>
837-
<li><a href="#examples">Qualitative Examples</a></li>
838837
<li><a href="#findings">Key Findings</a></li>
839838
<li><a href="#arena">Arena</a></li>
840839
<li><a href="#bibtex">BibTeX</a></li>

0 commit comments

Comments
 (0)