Drop structured-outputs enforcement for the exercise-details call

Sixth attempt at satisfying Anthropic's structured-outputs grammar
compiler for a rich per-exercise schema, after five straight failures
(optional-count limit, union-type limit, "too complex" twice, grammar
compilation timeout) -- every one of them in the exercise-details call
specifically. The workout-shape call has never failed once.

Rather than reshaping the schema a sixth time, the exercise-details call
now skips output_config/json_schema entirely: SystemPrompt#exercise_details_text
describes the exact JSON shape in prose (built programmatically from
EXERCISE_SCHEMA's properties, so the description can't silently drift
from the Exercise model) and instructs the model to respond with only a
JSON object, no markdown fences. parse_json_response gains a
strip_markdown_fences step before JSON.parse -- a no-op for the
workout-shape call's still-enforced, fence-free response, a real safety
net for the exercise-details call's unenforced one.

This trades away the hard guarantee that the response can't violate the
shape, but downstream validation already covers what that guarantee was
protecting against: lookup_movement! catches an unrecognized movement
name, the exercise-count-mismatch check catches a wrong array length,
ActiveRecord type-casts a stray numeric string automatically, and
Workout#valid? catches anything else.

Also drops `notes` from EXERCISE_SCHEMA per a separate simplification
call, unrelated to this specific fix.

EXERCISE_DETAILS_SCHEMA is removed (no longer used for output_config);
EXERCISE_SCHEMA stays, now purely as the source the prompt's field
descriptions are derived from. Test updates: exercise_payload helpers
drop notes; the schema test's "call 2 property counts" test is removed
since Anthropic's structured-outputs limits no longer apply to that call;
added a test confirming a markdown-fenced exercise-details response still
parses correctly.

Author: Yanchek99

app/services/workout_extraction/llm_parser.rb

@@ -6,17 +6,13 @@ class UnrepresentableWorkoutError < StandardError; end
     MODEL = 'claude-haiku-4-5'.freeze
     MAX_TOKENS = 2048
 
-    # Anthropic's structured-outputs grammar compiler caps how many *optional* (non-required)
-    # properties a schema can have (limit 24), how many *nullable/union-typed* properties it can have
-    # (limit 16), separately rejects schemas that are too large/deeply nested ("Schema is too
-    # complex"), and can time out compiling a schema outright ("Grammar compilation timed out") --
-    # observed specifically after union types (anyOf) were used inside array item schemas, which is
-    # consistent with union-typed array elements requiring the compiler to account for a much larger
-    # combined state space than a flat optional field would. So: extraction is split into two calls
-    # (one for the workout's shape -- segments, exercise text snippets, no per-exercise detail --
-    # and one that structures every snippet into a full exercise in a single flat array), and every
-    # field in both schemas is a single, non-union type: required where the field is unconditionally
-    # present, plainly optional (omittable) everywhere else. No anyOf/nullable types anywhere.
+    # Extraction is two calls: one for the workout's shape (segments, exercise text snippets, no
+    # per-exercise detail) and one that structures every snippet into full exercise details. The
+    # workout-shape call enforces WORKOUT_SHAPE_SCHEMA via output_config/json_schema; the
+    # exercise-details call does not enforce EXERCISE_SCHEMA that way (see the comment there) --
+    # every prior attempt to satisfy Anthropic's structured-outputs grammar compiler for a rich
+    # per-exercise schema (optional-property limits, nullable/union-type limits, "too complex",
+    # "grammar compilation timed out") failed specifically on that call, never on the shape call.
     SEGMENT_OUTLINE_SCHEMA = {
       type: 'object',
       properties: ModelSchema.properties_for(Segment, except: %w[id workout_id created_at updated_at position]),
@@ -55,24 +51,21 @@ class UnrepresentableWorkoutError < StandardError; end
       additionalProperties: false
     }.freeze
 
+    # Every failure above happened in the exercise-details call specifically (this one has never
+    # failed); the exercise-details call therefore doesn't enforce this schema via output_config at
+    # all -- it's used only to derive the prompt's field descriptions programmatically, so the
+    # prompt can't silently drift from the Exercise model. See SystemPrompt#exercise_details_text.
     EXERCISE_SCHEMA = {
       type: 'object',
       properties: ModelSchema.properties_for(
         Exercise,
-        except: %w[id workout_id movement_id segment_id created_at updated_at position],
+        except: %w[id workout_id movement_id segment_id created_at updated_at position notes],
         overrides: { movement_name: { type: 'string' } }
       ),
       required: %w[movement_name],
       additionalProperties: false
     }.freeze
 
-    EXERCISE_DETAILS_SCHEMA = {
-      type: 'object',
-      properties: { exercises: { type: 'array', items: EXERCISE_SCHEMA } },
-      required: %w[exercises],
-      additionalProperties: false
-    }.freeze
-
     def self.call(text) = new(text).parse
 
     def initialize(text)
@@ -118,8 +111,7 @@ def fetch_exercise_details(snippets)
         model: MODEL,
         max_tokens: MAX_TOKENS,
         system: SystemPrompt.exercise_details_text,
-        messages: [{ role: 'user', content: numbered_snippets }],
-        output_config: { format: { type: 'json_schema', schema: EXERCISE_DETAILS_SCHEMA } }
+        messages: [{ role: 'user', content: numbered_snippets }]
       )
       parse_json_response(response)[:exercises]
     end
@@ -128,11 +120,18 @@ def parse_json_response(response)
       text_block = response.content.find { |block| block.type == :text }
       raise ExtractionError, 'no text content in Anthropic response' unless text_block
 
-      JSON.parse(text_block.text, symbolize_names: true)
+      JSON.parse(strip_markdown_fences(text_block.text), symbolize_names: true)
     rescue JSON::ParserError => e
       raise ExtractionError, "malformed JSON from Anthropic: #{e.message}"
     end
 
+    # Only the (still schema-enforced) workout-shape call is guaranteed fence-free; the
+    # exercise-details call has no such guarantee, so strip a ```json fence if the model added one
+    # despite being told not to. A no-op when there's no fence to strip.
+    def strip_markdown_fences(text)
+      text.strip.sub(/\A```(?:json)?\s*\n?/, '').sub(/\n?```\s*\z/, '')
+    end
+
     def build_workout(shape, snippets, exercise_details)
       workout_attrs = shape.slice(:name, :score_type, :rounds, :time, :interval, :time_cap,
                                   :ladder_step, :team_size, :notes).compact

app/services/workout_extraction/system_prompt.rb

@@ -57,7 +57,18 @@ def self.workout_shape_text
     def self.exercise_details_text
       <<~PROMPT
         You convert a numbered list of CrossFit exercise prescription snippets into structured JSON,
-        one entry per snippet, in the same order, matching the provided schema's "exercises" array.
+        one entry per snippet, in the same order.
+
+        Respond with ONLY a JSON object -- no other text, no markdown code fences -- matching exactly
+        this shape:
+        {
+          "exercises": [
+            {
+              "movement_name": "<string, required -- copied verbatim from the recognized list below>",
+        #{exercise_field_lines}
+            }
+          ]
+        }
 
         Rules:
         - "movement_name" must be copied verbatim from this exact list of recognized movements (case
@@ -70,5 +81,14 @@ def self.exercise_details_text
         #{PRESCRIPTION_CHEAT_SHEET}
       PROMPT
     end
+
+    # Builds the exercise-details prompt's field list directly from EXERCISE_SCHEMA, so the prompt
+    # can't silently drift from the Exercise model the way a hand-written description could.
+    def self.exercise_field_lines
+      WorkoutExtraction::LlmParser::EXERCISE_SCHEMA[:properties].except(:movement_name).map do |name, property|
+        type_hint = property[:enum] ? property[:enum].map(&:inspect).join('/') : property[:type]
+        "      \"#{name}\": <#{type_hint}, omit if not specified>"
+      end.join(",\n")
+    end
   end
 end

test/services/workout_extraction/llm_parser_error_test.rb

@@ -94,7 +94,7 @@ def exercise_payload(overrides)
         movement_name: nil, reps: nil, duration_seconds: nil, load: nil, female_load: nil, male_load: nil,
         implement_count: nil, distance: nil, female_distance: nil, male_distance: nil, distance_unit: nil,
         distance_units_per_rep: nil, calories: nil, female_calories: nil, male_calories: nil,
-        ladder_step_every: nil, ladder_exempt: nil, notes: nil
+        ladder_step_every: nil, ladder_exempt: nil
       }.merge(overrides)
     end
   end

test/services/workout_extraction/llm_parser_schema_test.rb

@@ -6,7 +6,7 @@ class LlmParserSchemaTest < ActiveSupport::TestCase
       assert_equal(
         %i[reps duration_seconds load female_load male_load implement_count distance female_distance
            male_distance distance_unit distance_units_per_rep calories female_calories male_calories
-           ladder_step_every ladder_exempt notes movement_name].sort,
+           ladder_step_every ladder_exempt movement_name].sort,
         LlmParser::EXERCISE_SCHEMA[:properties].keys.sort
       )
       assert_equal(
@@ -19,7 +19,6 @@ class LlmParserSchemaTest < ActiveSupport::TestCase
         LlmParser::WORKOUT_SHAPE_SCHEMA[:properties].keys.sort
       )
       assert_equal(%i[text segment_index].sort, LlmParser::EXERCISE_SNIPPET_SCHEMA[:properties].keys.sort)
-      assert_equal(%i[exercises], LlmParser::EXERCISE_DETAILS_SCHEMA[:properties].keys)
     end
 
     test 'distance_unit is auto-constrained to the enum values and stays plainly optional' do
@@ -50,31 +49,23 @@ class LlmParserSchemaTest < ActiveSupport::TestCase
     test 'no schema uses anyOf/nullable types anywhere, since those were the cause of a grammar compilation timeout' do
       schemas = [
         LlmParser::WORKOUT_SHAPE_SCHEMA, LlmParser::SEGMENT_OUTLINE_SCHEMA, LlmParser::EXERCISE_SNIPPET_SCHEMA,
-        LlmParser::EXERCISE_SCHEMA, LlmParser::EXERCISE_DETAILS_SCHEMA
+        LlmParser::EXERCISE_SCHEMA
       ]
 
       schemas.each do |schema|
         assert_empty(schema[:properties].select { |_, prop| prop.key?(:anyOf) }, "#{schema} has an anyOf-typed property")
       end
     end
 
+    # Only the workout-shape call is still enforced via output_config/json_schema -- the
+    # exercise-details call stopped using structured outputs entirely after every prior attempt to
+    # satisfy the grammar compiler for a rich per-exercise schema failed, so EXERCISE_SCHEMA's
+    # property counts against Anthropic's structured-outputs limits are no longer a relevant check.
     test 'call 1 (workout shape) property counts stay within Anthropic structured-outputs limits' do
       call_1_schemas = [LlmParser::WORKOUT_SHAPE_SCHEMA, LlmParser::SEGMENT_OUTLINE_SCHEMA, LlmParser::EXERCISE_SNIPPET_SCHEMA]
 
-      assert_within_structured_output_limits(call_1_schemas)
-    end
-
-    test 'call 2 (exercise details) property counts stay within Anthropic structured-outputs limits' do
-      call_2_schemas = [LlmParser::EXERCISE_DETAILS_SCHEMA, LlmParser::EXERCISE_SCHEMA]
-
-      assert_within_structured_output_limits(call_2_schemas)
-    end
-
-    private
-
-    def assert_within_structured_output_limits(schemas)
-      total_optional = schemas.sum { |schema| schema[:properties].keys.map(&:to_s).length - schema[:required].length }
-      total_nullable = schemas.sum { |schema| schema[:properties].count { |_, prop| prop.key?(:anyOf) } }
+      total_optional = call_1_schemas.sum { |schema| schema[:properties].keys.map(&:to_s).length - schema[:required].length }
+      total_nullable = call_1_schemas.sum { |schema| schema[:properties].count { |_, prop| prop.key?(:anyOf) } }
 
       assert_operator total_optional, :<=, 24, "optional count #{total_optional} exceeds Anthropic's limit of 24"
       assert_operator total_nullable, :<=, 16, "nullable count #{total_nullable} exceeds Anthropic's limit of 16"

test/services/workout_extraction/llm_parser_test.rb

@@ -58,6 +58,24 @@ class LlmParserTest < ActiveSupport::TestCase
       assert top_level_exercise.segment.blank?
     end
 
+    test 'parses exercise details even when wrapped in a markdown code fence' do
+      stub_request(:post, 'https://api.anthropic.com/v1/messages').to_return(
+        anthropic_http_response(
+          extractable: true, name: 'Fran', score_type: 'time', rounds: nil, time: nil, interval: nil,
+          time_cap: nil, ladder_step: nil, team_size: nil, notes: nil, gap_reason: nil, segments: [],
+          exercise_snippets: [{ text: 'Thrusters (95/65)', segment_index: nil }]
+        ),
+        anthropic_http_response_with_raw_text(
+          "```json\n#{{ exercises: [exercise_payload(movement_name: @movement.name, reps: 1)] }.to_json}\n```"
+        )
+      )
+
+      workout = WorkoutExtraction::LlmParser.call('21-15-9 Thrusters (95/65)')
+
+      assert_equal 1, workout.exercises.size
+      assert_equal @movement, workout.exercises.first.movement
+    end
+
     private
 
     # Stubs the two sequential calls LlmParser makes: the workout-shape call, then (only if the shape
@@ -71,6 +89,10 @@ def stub_two_call_response(shape:, exercises: nil)
     end
 
     def anthropic_http_response(payload)
+      anthropic_http_response_with_raw_text(payload.to_json)
+    end
+
+    def anthropic_http_response_with_raw_text(text)
       {
         status: 200,
         headers: { 'Content-Type' => 'application/json' },
@@ -79,21 +101,21 @@ def anthropic_http_response(payload)
           type: 'message',
           role: 'assistant',
           model: 'claude-haiku-4-5',
-          content: [{ type: 'text', text: payload.to_json }],
+          content: [{ type: 'text', text: text }],
           stop_reason: 'end_turn',
           usage: { input_tokens: 100, output_tokens: 50 }
         }.to_json
       }
     end
 
     # A full EXERCISE_SCHEMA-shaped payload with every optional field nil except what's overridden --
-    # keeps individual tests from having to spell out all 18 fields every time.
+    # keeps individual tests from having to spell out all 17 fields every time.
     def exercise_payload(overrides)
       {
         movement_name: nil, reps: nil, duration_seconds: nil, load: nil, female_load: nil, male_load: nil,
         implement_count: nil, distance: nil, female_distance: nil, male_distance: nil, distance_unit: nil,
         distance_units_per_rep: nil, calories: nil, female_calories: nil, male_calories: nil,
-        ladder_step_every: nil, ladder_exempt: nil, notes: nil
+        ladder_step_every: nil, ladder_exempt: nil
       }.merge(overrides)
     end
   end

test/tasks/workout_extraction_rake_test.rb

@@ -19,7 +19,7 @@ class WorkoutExtractionRakeTest < ActiveSupport::TestCase
         { movement_name: movement.name, reps: 1, load: nil, female_load: 65, male_load: 95,
           duration_seconds: nil, implement_count: nil, distance: nil, female_distance: nil,
           male_distance: nil, distance_unit: nil, distance_units_per_rep: nil, calories: nil,
-          female_calories: nil, male_calories: nil, ladder_step_every: nil, ladder_exempt: nil, notes: nil }
+          female_calories: nil, male_calories: nil, ladder_step_every: nil, ladder_exempt: nil }
       ]
     )