System Prompt Ignored When Tools Are Present

[maxxrdrgz]

Using:
"ollama": "^0.5.14"

Hello,

When using the chat/ api, I noticed that when tools are passed in, the system prompt gets ignored. I'm not sure if this is intended. But I tried both llama3.1 and hermes 3.

Without tools:

// payload
{
  "model": "hermes3:8b",
  "messages": [
    { "role": "system", "content": "always speak like a pirate" },
    { "role": "user", "content": "hello?" }
  ]
}
// response
{
  "model": "hermes3:8b",
  "created_at": "2025-04-13T07:17:37.817454769Z",
  "message": {
    "role": "assistant",
    "content": "*gruff voice* Ahoy there, matey! Welcome to me ship. What be yer name and what brings ye here to these sea-worn shores? *eyeing you suspiciously* I hope it ain't trouble ye bring with ye, or ye'll have me cutlass to answer for!"
  },
  "done_reason": "stop",
  "done": true,
  "total_duration": 1608919876,
  "load_duration": 22682889,
  "prompt_eval_count": 21,
  "prompt_eval_duration": 18000000,
  "eval_count": 63,
  "eval_duration": 1567000000
}

With tools:

// payload
{
  "model": "hermes3:8b",
  "messages": [
    { "role": "system", "content": "always speak like a pirate" },
    { "role": "user", "content": "hello?" }
  ],
  "tools": [
    { "type": "function", "function": {}},
    { "type": "function", "function": {}},
  ]
}
// response
{
  "model": "hermes3:8b",
  "created_at": "2025-04-13T07:17:23.756195547Z",
  "message": {
    "role": "assistant",
    "content": "Hello! How can I assist you today?"
  },
  "done_reason": "stop",
  "done": true,
  "total_duration": 2317358459,
  "load_duration": 96253264,
  "prompt_eval_count": 2048,
  "prompt_eval_duration": 1585000000,
  "eval_count": 10,
  "eval_duration": 627000000
}

[maxxrdrgz]

Obviously, I don't need the LLM to talk like a pirate, it's just there as a simple straightforward example 😅

[jitendra19]

seems, it is working for me.

[jitendra19]

Ok, now understood the problem -

Even though the function object itself is empty, you've still siganled to the Ollama model that it has access to a tool of type "function".
When you then provide a query, the model might still decide that a "function" call is the next step, even if you haven't defined any specific function properties like name, description or parameters. So, the content will remain empty because the model is still waiting for you to execute a function and provide its results.

So either you can remove tools, let model to respond or provide well-defined tools with names and parameters that the model can understand and use. There are some examples available in this repo -

[maxxrdrgz]

@jitendra19 That's not the problem at all. The problem is, when giving tools to Ollama, it'll ignore any system prompt you have. When not giving it any tools, it'll respect the system prompt. The empty function () parameter is there as a place holder to avoid polluting the ticket. Your tool can be anything.

[red-dragon65]

I've been able to re-create this bug after running some tests.

This is not a bug with ollama-js, rather it is a bug with either ollama or hermes. The bug can be re-created using direct api calls to ollama using tools like postman.

It is most likely an issue with the default template that's being used for hermes in ollama, but this is speculation on my part and I have not personally tested it. Take a look at (ollama hermes template bug #6390) and (hermes function calling repo). For the other models, I think it's mostly just a hallucination issue rather than a bug.

This issue should be closed on this repo as its not really relevant here, and discussion should probably be moved to the aforementioned ollama or hermes links, or a new issue should be opened in those repos respectively. I'd also recommend trying out the qwen3:4b model as it seemed to work better with tool calls, albeit slower due to the thinking process.

Regardless, I'll leave the test methodology here as will as a link to my testing repo

For testing, I tried a combination of specifying the

• role as either system or assistant, with the prompt "always speak like a pirate"
• tools as either defined with a json tool schema or left blank
• message with either a normal message from the user ("hello?") or a tool suggestive message ("what is three minus one?")

Playing with these three parameters, role tools and message, there are 8 combinations.

Breakdown

hermes3:8b

• if tools are provided, but the user prompt does not trigger tool use, the ai will ignore the system prompt (using "assistant" as the role can sometimes mitigate this issue)

llama3.2:3b

• if tools are provided, the ai will attempt to use it in the first chat response, and then apologize afterwards (seems like a common issue)

llama3.1: 8b

• often gets stuck telling the user its trying to do a tool call rather than conversing with the user (apparently not recommended for tool calls)

qwen3:4b

• model gets stuck thinking. This may be due to asking it too simple of a prompt, or having too simple of a system prompt.
• seems the best at tool calls when setup properly. Thinks about whether or not it needs to tool call before doing it.

Test results for hermes:

• run on the ollama:rocm 0.6.8 docker container on Linux (Arch btw) with a 7900xtx
• as you can see, there are two combinations where the ai ignores the "talk like a pirate" system prompt

Starting results evaluation!


hermes3:8b | user-tool-message | no tools | assistant | Test # 2
Valid tool handling: true
Sounds like a pirate: "TRUE\n"
> Valid combination


hermes3:8b | user-regular-message | with tools defined | assistant | Test # 4
Valid tool handling: true
Sounds like a pirate: "FALSE\n"
>>> INVALID COMBO DETECTED
>>> INPUT
{
  model: 'hermes3:8b',
  role: 'assistant',
  tool: 'with tools defined',
  message: 'user-regular-message',
  testNum: 4
}
>>> OUTPUT
{
  message: 'OG RESPONSE: Hello! How can I assist you today?',
  isToolRequested: false,
  toolUsage: ''
}


hermes3:8b | user-regular-message | with tools defined | system | Test # 0
Valid tool handling: true
Sounds like a pirate: "FALSE\n"
>>> INVALID COMBO DETECTED
>>> INPUT
{
  model: 'hermes3:8b',
  role: 'system',
  tool: 'with tools defined',
  message: 'user-regular-message',
  testNum: 0
}
>>> OUTPUT
{
  message: 'OG RESPONSE: Hello! How can I assist you today?',
  isToolRequested: false,
  toolUsage: ''
}


hermes3:8b | user-regular-message | no tools | system | Test # 5
Valid tool handling: true
Sounds like a pirate: "TRUE\n"
> Valid combination


hermes3:8b | user-regular-message | no tools | assistant | Test # 1
Valid tool handling: true
Sounds like a pirate: "TRUE\n"
> Valid combination


hermes3:8b | user-tool-message | with tools defined | system | Test # 6
Valid tool handling: true
Sounds like a pirate: "TRUE\n"
> Valid combination


hermes3:8b | user-tool-message | with tools defined | assistant | Test # 7
Valid tool handling: true
Sounds like a pirate: "TRUE\n"
> Valid combination


hermes3:8b | user-tool-message | no tools | system | Test # 3
Valid tool handling: true
Sounds like a pirate: "TRUE\n"
> Valid combination

[Tomobobo710]

Is this still a thing? I seem to be hitting the same problem, though not through ollama-js...