Skip to content

SugarWOD import: narrow scope to catalog + single-modality, fix barbell-lift rep-scheme bug via set_details #1843

Description

@Yanchek99

SugarWOD Import: Scope Narrowing + Barbell-Lift Rep-Scheme Fix

Context

The original 11-task SugarWOD CSV import feature (implemented and reviewed on branch
worktree-sugarwod-import) was validated end-to-end against a real 1869-row SugarWOD export.
That run surfaced findings serious enough to change the feature's scope before merge:

  1. 96.1% of rows have zero delimiters between exercises in the raw CSV (no bullets, no
    newlines — exercises are literally concatenated, e.g.
    "5 Rounds for Time10 Benchpress 15520 GHD30 cal Ski"). This is a genuine SugarWOD export
    characteristic, not something preprocessing can fix.
  2. The general multi-exercise heuristic parser only resolves 5 of 442 currently-"imported"
    rows
    (93 are catalog matches via NameMatcher, 344 are barbell-lift rows). It does not
    carry its weight relative to the complexity it would take to extend (a tokenizer for
    undelimited text was explored and shelved — see "Out of scope" below).
  3. The LLM fallback (WorkoutExtraction::LlmParser) silently drops data. Tested directly
    against a real ambiguous row: it correctly resolved the exercise/rep-count structure but
    dropped a 155 lb Bench Press load entirely — no error, no low-confidence signal, just an
    incomplete Log that looks clean.
  4. A confirmed, separate correctness bug in already-reviewed code: BarbellLiftHeader's
    DEFAULT_REPS fallback (hardcoded to 1) produces actively wrong PRs for any
    pyramid/varying rep scheme it doesn't otherwise recognize — 17 of 358 real barbell-lift rows
    (4.75%). E.g. "Deadlift 21-15-15-9-9-9" was recorded as a false "265 lb 1-rep-max Deadlift"
    when the real final set was 9 reps.
  5. A previously-unused CSV column, set_details, contains ground-truth per-set data — a
    JSON array of {"success": bool, "load": number} per set, populated in 100% of the 358
    barbell-lift rows
    in the real export. Once failed attempts ("success": false) are
    excluded, best_result_raw equals max(successful set_details loads) in 358/358 rows
    (100%, verified)
    . This eliminates the need to infer which rep count a load belongs to —
    the data says so directly.

Goal

Narrow the importer to only what can be resolved with high confidence, fix the rep-scheme bug
using set_details as ground truth, and never silently guess, drop, or corrupt data. Lower
overall row coverage vs. attempting everything is an accepted tradeoff.

Scope — what this feature imports

  1. Catalog matches (benchmarks, hero WODs, the Open) — unchanged, via NameMatcher.
  2. Barbell-lift rows (barbell_lift CSV column populated) — redesigned per below, using
    set_details as the primary data source.
  3. New: monostructural distance/time trials — single monostructural movement (row/run/bike/
    ski), fixed-distance-for-time or fixed-time-for-max-distance/calories (e.g. "2k row," "5k
    run"). Narrow shape only — no intervals, no multi-piece pieces.
  4. New: bodyweight max-rep tests — single gymnastics/bodyweight movement, unbounded
    max-reps scoring (e.g. "max unbroken pull-ups").
  5. Everything else → skipped with a reason. No general multi-exercise heuristic parsing, no
    LLM fallback.

Out of scope (explicitly deferred)

  • General multi-exercise custom WOD parsing. A movement-name-anchored tokenizer for
    undelimited text was explored in depth (character-class-transition boundaries + Movement
    catalog anchoring + plausible-value-range disambiguation) and looks technically viable for a
    meaningful chunk of cases, but is shelved given it would only lift the general-heuristic path
    from 5 rows to some larger-but-still-partial number, at real implementation and maintenance
    cost. Revisit only if this feature's real-world coverage proves insufficient.
  • CrossFit Games workouts. Not currently seeded in the catalog (unlike benchmarks/hero/Open).
    Adding Games-workout matching would require building db/seeds/games_workouts.rb first — a
    separate, standalone data-entry effort.
  • A general cross-cutting "completeness gate" (movement-taxonomy-aware validation applied
    uniformly across every parse path) was considered as a response to the LLM's silent-data-loss
    finding, but since the LLM path is cut entirely in this scope, that motivating case no longer
    applies. Each new/fixed path validates its own completeness inline instead (see below).

Design: barbell-lift rows, redesigned around set_details

Data model split

  • Workout = the reusable prescription/template (e.g., "Front Squat, 7 sets: 3-1-3-1-3-1-12
    reps"). Shared/deduped across users doing the same rep-scheme, exactly like today's
    WorkoutFingerprint/absorb_duplicate! mechanism.
  • Log + MovementLog = this session's actual performance — the real weights this user
    hit, one MovementLog per set.

Algorithm

  1. Parse set_details (JSON array of {success, load}). If missing, blank, or fails to parse,
    fall through past the barbell-lift path (same as today's behavior for a blank barbell_lift
    column) — do not guess.
  2. Extract the per-set rep-count scheme from the title/description. This needs to be more
    robust than a simple dash-number regex — real examples include dash-joined numbers
    ("21-15-15-9-9-9"), explicit set breakdowns ("#1: 5 reps #2: 5 reps..."), and
    percentage-based prescriptions ("Set 1: 10 Reps @ 60% 1RM..."). Whatever the extraction
    method, it must produce a rep-count array whose length is validated against
    set_details.length
    before use.
  3. If the extracted scheme's length doesn't match set_details.length, skip the row with a
    clear reason (e.g. "could not align rep scheme with recorded sets") — never guess at
    alignment. (Real testing showed this mismatch is common with ad-hoc extraction — many
    apparent mismatches turned out to be extraction bugs in a throwaway analysis script, not
    genuine data problems, which is exactly why this needs a real validation gate rather than a
    best-effort regex.)
  4. Exclude failed attempts ("success": false) from being recorded at all, or at minimum
    from PR/max consideration — a missed lift is not a performance data point. (23 of 358 real
    rows contain at least one failed attempt; naively including them would risk crediting a missed
    lift as a PR.)
  5. Build the Workout: one segment containing N separate Exercise records (one per
    successful set), same movement, each with its own prescribed reps from the aligned scheme —
    reusing the existing multi-exercise-line architecture (build_exercise_lines), not the
    single-exercise FIND_MAX/DEFAULT_REPS shortcut.
  6. Build the Log: one MovementLog per successful set, each with its own reps (matching
    its Exercise) and load (from set_details, positionally aligned). best_result_raw is
    redundant with max(successful loads) (100% verified) and is not needed as a separate input
    once set_details is available — but should still gate this whole path (blank
    best_result_raw still means "no score recorded," per the existing blank-score guard.

Decision: failed sets are dropped entirely

A failed attempt ("success": false) does not become an Exercise/MovementLog at all — only
successful sets are recorded. Simpler, and zero risk of a missed lift ever being mistaken for a
PR.

New detectors

Both are conservative by construction: if a row doesn't cleanly match the narrow shape, it falls
through to skip — never to a guess, never to the LLM.

  • Monostructural: single monostructural movement + a distance or calorie/time target →
    distance/duration-scored exercise, no load.
  • Bodyweight max-rep: single gymnastics movement + max-effort language ("max unbroken," "max
    reps") → rep-scored exercise, no load.

Each validates its own completeness inline (monostructural requires distance/duration; bodyweight
requires reps) rather than a shared cross-cutting gate.

Testing

Real CSV examples already gathered anchor the test suite directly:

  • The 17 pyramid-scheme rows for the rep-scheme fix ("Deadlift 21-15-15-9-9-9",
    "Back Squat 10-8-6-4-2", etc.)
  • The set_details-driven multi-MovementLog behavior ("Front Squat 3-1-3-1-3-1-12" with its
    7-entry set_details array)
  • Failed-attempt exclusion ("Power Clean" with a failed 225 lb attempt after a successful 215)
  • Length-mismatch skip behavior (a scheme/set_details misalignment case)
  • "2K ROW" for the monostructural detector
  • Real skip-worthy multi-exercise rows, confirming they're correctly rejected rather than guessed
    at

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions