🎬 LAST FRAME

A playable film. Every shot is generated as you watch, and the video model is the dice: whatever happens on screen is what happened, whatever you intended. You asked to leap the chasm; the frame shows you dangling from the ledge by one hand? That's what happened. Take the damage.

Powered by fal + MiniMax H3 Max, a video model faster than realtime, which makes a game rendered by a video model possible: the next shots film while the current one is still playing, so picking one plays with no gap.

Two engines live in this repo:

• Shots (EVERYDOOR, THE LOST TAPE, the quest worlds): discrete 10-second clips chained frame to frame, three branches pre-filmed per beat, an Adjudicator ruling on every landed shot.
• Live (the Horror category): one continuous MiniMax H3 Max Director stream over WebRTC. No clips, no cuts, no freeze-frames. You steer it with text and the world keeps moving while you decide.

From a run of THE LOST TAPE, the found-footage horror world. Shot 19: the rocking chair starts moving on its own while the next choices film over the still-rolling clip.

Shot 26, 5 vitality left. It stopped keeping its distance.

The rules

• 100 vitality, one life. The Adjudicator (a vision LLM that watches every shot's frames) rules on what it sees and charges you for it. Every HP change carries an evidence still of the frame that justified it.
• Three choices per beat, or type your own. The options surface over the clip while it is still playing, and all three branches film in parallel behind it. Pick one mid-clip and it rolls the instant the shot ends. Press 1, 2 or 3, or click. Typed moves are refereed against the current frame and risky ones roll a visible d20: the roll decides whether the success or the failure shot gets filmed.
• Hesitate and the world moves. If the clip ends before you pick, the frame freezes and a timer runs. Let it run out and the world takes its own shot, never in your favor.
• What you see is what you take. Objects your hero visibly grabs become inventory cards (with the frame they came from). An item is a stored prompt fragment: arm one and it is written into your next shot, guaranteed.
• Win on screen. Quest worlds have a goal the Adjudicator must see accomplished. Open worlds (EVERYDOOR, the horror) have no goal: one life, see how far you get. Death gets a final generated cinematic; either way you get the full timeline of your run, shot by shot, with every roll and wound.

Live Horror

Five worlds on one continuous Director stream. Each is a different fear grammar, and each carries two hidden decks the Director GM draws from at random, so nobody can predict the beat:

World	The rule
THE LOST TAPE: LIVE	Night woods, a rotting farmhouse, a tall pale shape that is always somewhere it should not be.
BLACKOUT	An apartment tower losing power. The shadow exists only where there is no light, and every light is failing.
MIRRORS	A grand empty house of glass. The wrongness lives only in reflections, until it does not.
MILE MARKER	A night drive that will not end. The same figure on the shoulder, a mile later, closer to the road.
FOGBOUND	A fog-drowned coastal town that emptied mid-dinner and is slowly remembering its people.

How a live run works:

• The hero is always moving. The video model is good at walking, driving and opening doors, and bad at a person standing still while a monster performs at them.
• The unknown does the work. The entity is never seen clearly and never approachable; light only ever fails, never flares. For the first two thirds of the tape the GM can only draw subtle cards (a switch clicking off behind you, footprints appearing one step at a time). Spike cards, the real danger and the jumpscare, unlock in the final act.
• Every ~9 seconds, or the instant you make a move, the Director GM grabs the live frame, rules vitality from what is on screen, and sends the next segment as a prompt message, which the stream applies at its next chunk boundary. Vitality drains slowly by design; the tape is meant to be survivable.
• Director sessions run about two minutes. That cap is the premise: a REC clock counts down, and the tape running out means you survived it.

Quick start

npm install
cp .env.example .env.local   # put your fal API key in it
npm run dev

Get a key at fal.ai/dashboard/keys. Open http://localhost:3000 and pick a world. The live worlds use the alpha line of @fal-ai/client for fal.realtime.open.

How the shot engine works

opening ──► play ──► ADJUDICATOR rules mid-playback (zero wall-clock)
              │            │
              │            ▼
              │      options surface OVER the rolling clip
              │      3 branches pre-film in parallel (i2v from the last frame)
              │      each ready branch is pre-adjudicated too
              ▼
     pick mid-clip: rolls the instant the shot ends ──► loop
     no pick: freeze-frame + timer ──► the world's own shot

• Continuity is the frame, novelty is the text. Every shot chains from the previous shot's last frame (image_url), so the film never breaks; prompts only ever describe change. A rare hard cut (at most one option per beat, only when no exit is visible) films text-to-video so the story can actually leave a scene.
• The Adjudicator (Gemini Flash Lite via fal's vision router) returns strict JSON per shot: scene ground truth, narration, HP delta + reason, items gained/lost, a short synopsis of completed facts, death/victory flags, the next three options, and the world's move if you stall. It runs while the clip plays, and ready branches are ruled on before you pick them, so the next options usually appear the moment a shot starts.
• The Referee judges typed actions against the frame: risk tier (safe / risky 8+ / deadly 13+ on a d20) plus a success shot and a failure shot. Moderation (strict ALLOW/BLOCK, injection-hardened, fail-closed) runs in parallel server-side.
• Speech is scripted. The video model babbles unless voices are declared wordless, and pronounces a line cleanly only when it is quoted verbatim (measured with Whisper; see experiments/). Prompts with a quoted line get the scripted sound clause, everything else stays wordless.
• Your FAL_KEY stays server-side (@fal-ai/server-proxy), and clips stream through a same-origin media proxy so canvas frame grabs never hit CORS.

Cost note

A shot-engine run films ~3 branch clips per choice plus one shot that plays, at 480P / 10s each, plus a few small LLM calls per beat. A live horror run is a Director session, billed per second of stream with a 60-second minimum, so budget a couple of dollars per two-minute tape. Keep an eye on your fal usage; dying is the cheapest way to stop.