Ollama and qwen models

[qubit0]

Here's my .env file:

# LLM API Configuration - pointing to local Ollama
LLM_API_KEY=ollama
LLM_BASE_URL=http://localhost:11434/api
LLM_MODEL_NAME=qwen3.5:0.8b

I get the following in backend/logs/

[2026-04-01 21:26:03] DEBUG [mirofish.api.generate_ontology:158] 模拟需求: What would happen if they can't deliver 50% of their promise in 100 days?...
[2026-04-01 21:26:03] INFO [mirofish.api.generate_ontology:177] 创建项目: proj_450fa2d55c88
[2026-04-01 21:26:03] INFO [mirofish.api.generate_ontology:212] 文本提取完成，共 44882 字符
[2026-04-01 21:26:03] INFO [mirofish.api.generate_ontology:215] 调用 LLM 生成本体定义...
[2026-04-01 21:26:03] DEBUG [mirofish.request.log_response:62] 响应: 500

[dosubot]

Hi @qubit0! I'm Dosu and I’m helping the MiroFish team.

This is a known Ollama compatibility issue that's come up before (e.g., issue #330). There are two problems with your setup:

1. Wrong base URL — MiroFish uses the OpenAI Python SDK, so it needs Ollama's OpenAI-compatible endpoint. Change /api to /v1:

LLM_BASE_URL=http://localhost:11434/v1

[1] [2]

2. response_format incompatibility — Even after fixing the URL, you'll likely still hit 500 errors because MiroFish's chat_json() method unconditionally sends response_format={"type": "json_object"} [3], which Ollama doesn't support by default — especially with smaller models like qwen3.5:0.8b [4].

To work around this, PR #186 added fallback retry logic that automatically retries without response_format when the first attempt fails. If you're running via Docker, rebuild with docker compose up --build to pick up this fix.

Summary of recommended .env changes:

LLM_API_KEY=ollama
LLM_BASE_URL=http://localhost:11434/v1
LLM_MODEL_NAME=qwen3.5:0.8b

Note that MiroFish is officially tested only with Alibaba's qwen-plus model [5], so smaller local models may have additional output quality issues even if the API calls succeed.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[prenansantana]

Hi @ — opened #580 which I believe addresses the same root cause you're hitting (and a few others reporting 500s on /api/graph/ontology/generate).

Two things to check on your config:

1. Base URL path: Ollama's OpenAI-compatible endpoint is at /v1, not /api. Try LLM_BASE_URL=http://localhost:11434/v1.
2. response_format rejection: MiroFish currently hard-codes response_format={"type": "json_object"} on every JSON-producing call. Some OpenAI-compatible runtimes (LM Studio, llama.cpp server, certain Ollama builds) reject this parameter with HTTP 400 - 'response_format.type' must be 'json_schema' or 'text', which the backend surfaces as a 500.

PR #580 introduces an LLM_JSON_MODE env var (default json_object — fully backwards compatible). Setting LLM_JSON_MODE=none makes the backend skip the parameter and fall back to the existing markdown-tolerant JSON parser, which is enough for any OpenAI-compatible local runtime.

If you can apply the patch locally (or check out the branch from my fork), I'd love a 👍 / 👎 on whether it unblocks you — that helps the maintainer judge real-world impact.

Full setup recipes for Ollama / LM Studio / llama.cpp are in docs/LOCAL_LLM.md on that branch.