I dont understand how to make tools work with local models

[dustContributor]

Hi! I've been using Continue for quite a while and I'm quite happy with it. With the notice that the index is being deprecated, I've been trying to make tools work, but I don't understand how to do so.

For example, I'm using a local ollama server with qwen2.5-coder:7b for chat, edit, autocomplete, etc.

I ask the model a question that would require to read the files of the repository I'm sitting on , it suggests me what it looks like a valid tool call (a file search by glob to find all python files), but when I click on the run button it does something baffling and alters the file I have currently opened trying to come up with a python block to list these files:

Does this makes any sense? I don't understand what do I have to do for the model to be able to execute the file reading tool. I've read about MCP Servers but they only reference things like searching on public repositories and all sorts of things that go way beyond what I want that is for the local model to be able to search through the files of the repository I have open in VSCode.

[dustContributor]

I think the problem is that this particular model, qwen2.5-coder:7b, doesn't seems to elaborate tool calls properly. It'll just return a JSON markdown block instead of a tool call understood by Continue.

It's a bit strange because looking at the model file, it's explicitly told not to do that https://ollama.com/library/qwen2.5-coder:7b/blobs/1e65450c3067

I re-downloaded the model just in case. On the other hand, qwen2.5:3b-instruct does seem to be able to issue a glob search but it starts breaking down, probably because it's not a good fit for this use case.

So I guess it's a matter of finding a model that can properly reproduce the tool calls.

I saw an user that I can't find right now mentioning that swapping the XML tag <tool_call> with [tool_call] in the modelfile, then re-creating the model, worked. Based on this comment here https://github.com/ollama/ollama/blob/66fb8575ced090a969c9529c88ee57a8df1259c2/tools/template.go#L13

But I haven't tried that yet.

[bdougie]

3b is a bit small to have tool calling built in. We have recommendations for this in the docs.

https://docs.continue.dev/features/autocomplete/model-setup#model-recommendations

If you are looking to use Qwen 3 30b or gpt-oss 20b would be the recommendation

[faetschi]

I'm also struggling with local agents with Ollama.

Currently using qwen3-coder:30b and gpt-oss:20b for Agents (with correct config.yaml setup), but the models fail to reliably use/call Tools in Continue Version 1.3.15

If multi-step instructions are sent, both models only use one tool successfully at max:

e.g. "Create a new project with a python calculator app"

Both models first start by creating the directory (mkdir), which is succesful.
Then all following tasks (like creating files) are just sent as plaintext/code in the chat, without the model using tools to actually create those files.

Maybe this is related to a already fixed bug?
#8098

[bdougie]

I found some existing related conversations. Let me know if these are helpful:

#	Title	Status
#6913	Feature Request: Support Qwen3 Coder 30B Local Model for Tool Calls	🟢 Open
#7724	Local ollama models not being used/overridden by Claude or GPT	🟢 Open
#7219	vscode extension select model hangs.	🟢 Open
#7828	MCP tools don't work with Ollama models	🟢 Open
#7158	Agent Mode - model recommendations	🟢 Open

[faetschi]

thanks, I will look into it - sounds promising!