Wrapper Method for Structured Output for Generable

[rudrankriyam]

Foundation Models has a simple method like:

// Generate a response using a custom type.
let response = try await session.respond(
    to: "Generate a cute rescue cat",
    generating: CatProfile.self
)

After its release, MLX Swift added a wrapper over their API:

let model = try await loadModel(id: "mlx-community/Qwen3-4B-4bit")
let session = ChatSession(model)
let response = try await session.respond(to: "What are two things to see in San Francisco?")

What do you think of something similar?

let model = try await loadModel(id: "mlx-community/Qwen3-4B-4bit")
let session = ChatSession(model)
let response = try await session.respond(to: "What are two things to see in San Francisco?", generating: Attractions.self)

[petrukha-ivan]

Thank you for the idea!

This could definitely simplify API usage and make it more convenient. While implementing it directly for ChatSession isn’t possible without changes to the MLXLMCommon library, I can add support for a simple generate function that returns a decoded model, similar to how it worked in MLXLMCommon before Foundation Models:

let result = try generate(input: input, parameters: parameters, context: context) { tokens in
    .more
}

For decoded output, it could look like this:

let (result, model) = try generate(input: input, parameters: parameters, context: context, generating: Model.self) { tokens in
    .more
}

I’m adding this to my to-do list, and I’ll also think about the possibility of a ChatSession extension 👍

[petrukha-ivan]

This is partially resolved in the 0.0.2 release, though the ChatSession issue remains unaddressed.

Now you can directly pass a grammar or a Generable type into the generate function:

let (result, model) = try await generate(input: input, context: context, generating: PersonInfo.self)
print(result.output) // Generated text
print(model) // Generated model

Streaming generation has also been added (only for Generable types), returning PartiallyGenerated content:

let stream = try await generate(input: input, context: context, generating: PersonInfo.self)
for await content in stream {
    print("Partially generated:", content)
}