Feature: Frame-by-frame stepping & on-frame drawing annotations for body language review workflows

[VictorEffectiv]

Context

I'm a life coach who uses video to train people on body language, facial expressions, and non-verbal communication. My team and I would use lawn as our review platform. Our core workflow is:

1. A client uploads a video of themselves (a presentation, an interview rehearsal, a coaching session)
2. My team reviews the video together, focusing on micro-expressions and body language cues
3. We need to slow down and step through the video frame by frame to catch subtle facial expressions and gestures
4. We need to draw/annotate directly on a paused frame — circling a facial expression, pointing an arrow at a hand gesture, highlighting posture — so the client can see exactly what we're referring to

This is the kind of targeted, visual feedback that a plain text comment with a timestamp can't convey. "Your left eyebrow raises at 1:23" is helpful, but a circle drawn around the eyebrow on that exact frame is 10x clearer.

What lawn already does well for this

• Team-based video sharing and review is exactly right
• Timestamped comments with timeline markers are great for general feedback
• Playback speed control (0.5x, 0.75x) helps with slow-motion review
• Share links let us send annotated videos back to clients
• Workflow statuses (review → rework → done) map perfectly to our coaching flow
• Real-time presence shows who's watching

What's missing (two features)

1. Frame-by-frame navigation

The video player currently supports seeking (±5s with arrow keys, ±10s with buttons) and speed control (0.5x–2x), but there's no way to step one frame forward or backward. For body language review, we need to freeze on an exact frame to study a micro-expression.

Suggested behavior:

• , (comma) key steps back one frame while paused
• . (period) key steps forward one frame while paused
• These are the same keys Final Cut Pro and DaVinci Resolve use, which aligns with the existing todo.md note about Final Cut style keybindings
• Use video.currentTime += 1/fps approach (detect fps from the video or default to 30fps) or use requestVideoFrameCallback where supported
• Show the current frame number or precise timecode (e.g., 00:01:23:15) somewhere in the controls when paused, so users can reference exact frames in discussion
• Optionally, add a 0.25x speed option to the existing PLAYBACK_RATES array for even slower motion review

Implementation notes:

• The PLAYBACK_RATES array on line ~61 of VideoPlayer.tsx could be extended to include 0.25
• The onKeyDown handler (around line ~950) already handles keyboard shortcuts and could be extended with , and . handlers
• Frame stepping should only work when paused to avoid confusing behavior
• The seekTo ref method could be used internally for frame stepping

2. Drawing/annotation overlay on paused frames

When the video is paused (especially after frame-stepping to the right moment), we need a simple drawing tool overlay to annotate the frame. The annotation should be saved as a comment attachment so the whole team (and the client via share link) can see it.

Suggested behavior:

• When video is paused, a toolbar appears with basic drawing tools: freehand pen, circle/ellipse, arrow, and a color picker (red, yellow, green, white — a few high-contrast options)
• Drawing happens on a transparent canvas overlaid on the video at its native resolution
• When the user submits the drawing, it gets saved as an annotation attached to a comment at that timestamp
• In the comment list, annotated comments show a small thumbnail preview of the marked-up frame
• Clicking that thumbnail in the comment list seeks to that timestamp and re-renders the drawing overlay on the frame
• The drawing data could be stored as SVG paths or a simple JSON stroke format (array of {tool, color, points}) in the comment record — keep it lightweight, not a rasterized image
• On share links, annotations should be visible to external reviewers (read-only)

Implementation notes:

• A <canvas> element sized to match the <video> dimensions, positioned absolutely on top of it, with pointer-events: auto only when drawing mode is active
• The comment schema (convex/schema.ts around line ~108) would need an optional annotations field — something like v.optional(v.string()) holding serialized JSON of the drawing strokes
• The CommentInput component could gain an "Attach drawing" button that activates drawing mode on the player
• The CommentItem component would check for annotations and render a clickable thumbnail
• The VideoPlayer component would need a new annotations prop and an overlay render layer
• For the share page (-share.tsx) and watch page (-watch.tsx), annotations should be rendered read-only when a comment is clicked

Data model addition

In convex/schema.ts, the comments table could be extended:

comments: defineTable({
  // ...existing fields...
  annotations: v.optional(v.string()), // JSON: { tool, color, strokeWidth, points }[]
  frameNumber: v.optional(v.number()), // exact frame for annotation alignment
})

Why this matters

Body language coaching is inherently visual and frame-precise. Text comments at a timestamp get you 80% there, but the last 20% — the ability to freeze on a micro-expression and literally circle it — is what separates a useful tool from an indispensable one. These two features would make lawn viable for an entirely new category of users: coaches, trainers, sports analysts, acting instructors, and anyone who reviews human movement on video.

Both features also complement each other: frame-stepping lets you find the exact right frame, and drawing lets you mark it up. They're most powerful together.

Priority suggestion

Frame-by-frame stepping is the simpler of the two and independently useful — it aligns with the existing todo.md item about Final Cut style keybindings. Drawing annotations are more complex but far more impactful for visual feedback workflows. If these need to be phased, frame stepping first would be my recommendation.