Structured Responses Crash with Image Input

[tobiasbueschel]

Which version of LM Studio?
Version 0.3.24

Which operating system?
macOS 26.0 Beta (25A5351b)

What is the bug?
When using the LM Studio JS SDK's structured response feature (structured parameter with Zod schema or JSON schema) together with image input, the model crashes with exit code 11 (segmentation fault). This happens consistently when calling model.respond() with both:

1. Images in the message (images: [imageHandle])
2. Structured response configuration (structured: zodSchema or structured: { type: "json", jsonSchema: ... })

The same models work fine with:

• Structured responses WITHOUT images
• Images WITHOUT structured responses
• But NOT both together

Affected Models Tested:

• gemma-3-27b-it (crashes consistently)
• Other vision models may also be affected

Logs
Error output when attempting structured response with image:

┌ Error ─────────────────────────────────────────────────────────────┐
│                                                                    │
│    The model has crashed without additional information.           │
│    (Exit code: 11)                                                │
│                                                                    │
└────────────────────────────────────────────────────────────────────┘

To Reproduce
Steps to reproduce the behavior:

1. Install @lmstudio/sdk version 1.5.0
2. Load a vision model (e.g., gemma-3-27b-it or qwen2-vl-2b-instruct)
3. Prepare an image using client.files.prepareImage(imagePath)
4. Call model.respond() with both image input and structured response:

const { z } = require("zod");
const { LMStudioClient } = require("@lmstudio/sdk");

const client = new LMStudioClient();
const model = await client.llm.model("gemma-3-27b-it");

// Define schema
const schema = z.object({
  description: z.string(),
});

// Prepare image
const imageHandle = await client.files.prepareImage("/path/to/image.jpg");

// This will crash with exit code 11
const response = await model.respond(
  [
    {
      role: "user",
      content: "Describe this image",
      images: [imageHandle],
    },
  ],
  {
    structured: schema, // <-- This causes the crash when combined with images
  }
);

Expected Behavior
Structured responses should work with image input without crashing the model.

Additional Context

• This appears to be an issue with the constraint/grammar system when processing multimodal inputs
• The crash (exit code 11) suggests a segmentation fault in the underlying model runtime
• This significantly limits the usefulness of structured responses for vision tasks