Proposal: Structured Tool API

[ibrahimcetin]

The current ToolSpec is just a type alias and isn’t very ergonomic or type-safe when defining tools. I’d like to propose a more structured and Swifty API for defining and using tools, inspired by ollama-swift.

This new design makes it easier to define, register, and invoke tools with full Codable support for inputs and outputs. It also encourages safe and readable usage by reducing boilerplate and typos.

Example Usage

struct WeatherInput: Codable {
    let city: String
}

struct WeatherOutput: Codable {
    let temperature: Double
    let conditions: String
}

let weatherTool = Tool<WeatherInput, WeatherOutput>(
    name: "get_current_weather",
    description: """
    Get the current weather for a city,
    with conditions ("sunny", "cloudy", etc.)
    and temperature in °C.
    """,
    parameters: [
       .required("city", type: .string, description: "The city to get weather for")
    ]
) { input async throws -> WeatherOutput in
    // Implement weather lookup logic here
    return WeatherOutput(temperature: 18.5, conditions: "cloudy")
}

The Tool struct accepts a name, description, a list of parameters, and the actual implementation logic. It is generic over two types: one for the input and one for the output, both conforming to Codable. Internally, the parameters will eventually be represented as a dictionary. The initializer is just a more structured and type-safe way to define them.

let chat: [Chat.Message] = [
    .system("You are a helpful assistant that can check the weather."),
    .user("What's the weather like in Portland?")
]

let input = UserInput(
    chat: chat,
    tools: [weatherTool]
)

/* Generate output */

/* Handle tool call */

Handling Tool Calls

Ollama includes tool calls as part of the final message output. From my experience, when tools are provided, only a single message is returned, even if streaming is enabled. It holds back the output until the full response is generated, and only then returns the message along with any tool call metadata. While I'm not entirely sure how we could replicate this behavior, I think it's a practical and user-friendly approach and we should consider adopting a similar solution.

// ⚠️ Ollama Swift's Tool Call Handling
if let toolCalls = response.message.toolCalls {
    for toolCall in toolCalls {
        print("Tool called: \(toolCall.function.name)")
        print("Arguments: \(toolCall.function.arguments)")
    }
}

I’d like to discuss this with you. What are your thoughts?

[davidkoski]

I like the look of Tool -- this looks straightforward for how to integrate with existing models. The tricky part (and I think this is what you are alluding to in the last paragraph) is how to manage calls.

Actually I am not familiar with how the models use the tool calls. Does the model expect to get a "40° C" response back and then react to that (wow, that is pretty warm!) or is the caller supposed to make the calls and interpret the results themselves?

If the former then it suggests it would need to happen inside TokenIterator. If the latter it would be serialized calls in GenerateCompletionInfo. I think :-)

Anyway, I like the direction this is going.