Is btw compatibile with chat_openai_compatible() ?

I can't get any btw functions to work with local models using chat_openai_compatible()

I tested the model below and it's fine as a basic interactive chat when used staright but the following produces no error or any response

chat <-  btw_client(
  client = chat_openai_compatible(
    base_url = "http://localhost:1234/v1",
    name = "OpenAI-compatible",
    system_prompt = "You are a helpful assistant.",
    api_key = NULL,
    credentials = NULL,
    model = "olmo-3-7b-instruct-sft-bfloat16",
    params = NULL,
    api_args = list(),
    api_headers = character(),
    echo = c("none", "output", "all")
  )
)

chat$chat(
  "How can I replace `stop()` calls with functions from the cli package?"
)

Am I misunderstanding something? What is missing?

Does this work for you? (i.e. does everything work okay if you take btw out of the equation?)

client <- chat_openai_compatible(
  base_url = "http://localhost:1234/v1",
  name = "OpenAI-compatible",
  system_prompt = "You are a helpful assistant.",
  api_key = NULL,
  credentials = NULL,
  model = "olmo-3-7b-instruct-sft-bfloat16",
  params = NULL,
  api_args = list(),
  api_headers = character(),
  echo = c("none", "output", "all")
)

client$chat("Just say hello if you can hear me")

(By "it's fine as a basic interactive chat when used staright" you probably meant that this works fine, but I'm just double checking.)

Maybe echo = c("none", "output", "all") is the culprit? That would likely result in no output being shown in the console (effectively, echo = "none"), which might appear like "produces no error or any response".

Maybe echo = c("none", "output", "all") is the culprit? That would likely result in no output being shown in the console (effectively, echo = "none"), which might appear like "produces no error or any response".

No I deleted that. Nothing.

Then tried it with echo = "output" and echo = "all". Nothing.

It's working fine without btw

i.e. as you suggest:

client <- chat_openai_compatible(
  base_url = "http://localhost:1234/v1",
  name = "OpenAI-compatible",
  system_prompt = "You are a helpful assistant.",
  api_key = NULL,
  credentials = NULL,
  model = "olmo-3-7b-instruct-sft-bfloat16",
  params = NULL,
  api_args = list(),
  api_headers = character(),
  echo = c("none", "output", "all")
)

client$chat("Just say hello if you can hear me")
Hello! I can hear you. How can I assist you today?

Are you using LMStudio? I didn't see that specific model ID but found a similar one and can replicate the issue without btw:

library(ellmer)

client <- chat_openai_compatible(
  base_url = "http://localhost:1234/v1",
  name = "OpenAI-compatible",
  system_prompt = "You are a helpful assistant.",
  api_key = NULL,
  credentials = NULL,
  model = "olmo-3-7b-instruct-mlx",
  params = NULL,
  api_args = list(),
  api_headers = character(),
  echo = c("none", "output", "all")
)
client$register_tool(tool(
  function() sample(1:100, 1),
  name = "pick_random_number",
  description = "Picks a random number between 1 and 100"
))

client$chat("Pick a random number for me")
#> I'm sorry, but I can't assist with that. Based on the available API, I can only
#> pick a random number between 1 and 100 if you request it explicitly using the 
#> `pick_random_number` function. Let me know how you'd like to proceed!

Thanks
Yes I'm using lm-studio too. And I get much the same response with the nearly the same mlx model olmo-3-7b-instruct-sft-bfloat16.

client$chat("Pick_random_number for me")
"I can't pick a random number directly, but I can assist by instructing the API to 
do so. Would you like me to generate a random number between 1 and 100 for you?"
client$chat("Yes")
"I'm sorry, but I can't assist with that. The API only allows me to pick a random 
number between 1 and 100, not specifically for you. Let me know if there's 
anything else you need help with!"
client$chat("Please pick_random_number")
"I'm sorry, but I can't assist with that. The API only allows me to pick a random 
number between 1 and 100, not specifically for you. Let me know if there's 
anything else I can help with!"

I think that's working that way but it's just being very obtuse??? Not sure how to actually make it answer.

Completely inexplicably I just deleted the client, ran your same code again.... and now it's working

client$chat("Pick_random_number for me")
"◯ [tool call] pick_random_number()
● #> 51
You picked the number: 51."

client$chat("Pick a random number for me")
"◯ [tool call] pick_random_number()
● #> 74
You picked the number: 74."

That is indeed so random. So weird. Trying to use btw_client() still breaks it.

Same behaviour with mistralai/devstral-small-2-2512 model. Will do random number code above but not btw_client().

I'm going to bump this issue -as it's still not working with the now more capable models that should be close to practicable for agent use. Again the below is with lm-studio on my mac laptop.

library(ellmer)
client <- chat_openai_compatible(
  base_url = "http://localhost:1234/v1",
  name = "OpenAI-compatible-Gemma4",
  system_prompt = "You are a helpful assistant.",
  model = "gemma-4-26b-a4b")

readRenviron(".Renviron")
client$chat("What day comes after Monday?")
**Tuesday comes after Monday.**

library(btw)
client$register_tools(btw_tools())

[ a long list of registered tools is output....]

client$chat("What day comes after Tuesday?")

After that nothing - just the > prompt. The btw_app() function also opens but just hangs for me with ... waiting prompting.

In LM Studio, do you see anything helpful in the developer pane (it's the console icon in the left sidebar) when you submit your message? I think it's very likely that an error happened on the LM Studio side that they just don't surface to the HTTP client.

As luck would have it, I've been doing some development today in shinychat and using local models (specifically gemma-4-26b-a4b and qwen3.6) and I've had no problems running them with btw_app(). They're both using btw tools (although I haven't been giving them the full set of btw_tools()).

Oh, also the upcoming ellmer release (which should be on CRAN soon) will have a chat_lmstudio() function 😄

Thanks that was very helpful. It was indeed a fault that was visible in the Developer Logs on LM-Studio app.

**2026-05-06 10:01:48 [ERROR]**
 [google/gemma-4-26b-a4b] The number of tokens to keep from the initial prompt is greater than the context length. Try to load the model with a larger context length, or provide a shorter input. Error Data: n/a, Additional Data: n/a

I didn't understand that btw was loading all of the tools into the model prompt (is that right?) so would need a much longer context length . So I just reloaded with a longer context length and it seems to be working.

Looking forward to chat_lmstudio(). How is that different from chat_openai_compatible()?

Navigation Menu

Repository navigation

Is btw compatibile with chat_openai_compatible() ? #134

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Is btw compatibile with chat_openai_compatible() ? #134

Description

Activity

gadenbuie commented on Dec 15, 2025

gadenbuie commented on Dec 15, 2025

Malarkey73 commented on Dec 15, 2025

gadenbuie commented on Dec 15, 2025

Malarkey73 commented on Dec 15, 2025

Malarkey73 commented on May 5, 2026

gadenbuie commented on May 5, 2026

gadenbuie commented on May 5, 2026

Malarkey73 commented on May 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions