500 error

[alexa898]

i cant have access to aliyun for international users, can you update your code to work with openAi or other models

[dosubot]

Hi @alexa898! I'm Dosu and I’m helping the MiroFish team.

MiroFish does support OpenAI and other providers — but there's a known bug causing your 500 error. The system accepts any OpenAI-compatible API via .env configuration:

LLM_API_KEY=your-openai-key
LLM_BASE_URL=https://api.openai.com/v1
LLM_MODEL_NAME=gpt-4o-mini

The problem: MiroFish hardcodes response_format={"type": "json_object"} in chat_json(), which causes 500 errors with OpenAI, Groq, SiliconFlow, and other non-Aliyun providers. This has been reported multiple times including as recently as today.

Workaround: PR #186 fixes this with automatic fallback retry logic, but it hasn't been merged yet. You can manually patch backend/app/utils/llm_client.py:

def chat_json(self, messages, temperature=0.7, max_tokens=4096):
    try:
        response = self.chat(
            messages=messages,
            temperature=temperature,
            max_tokens=max_tokens,
            response_format={"type": "json_object"}
        )
    except Exception as e:
        # Retry without response_format if unsupported
        response = self.chat(
            messages=messages,
            temperature=temperature,
            max_tokens=max_tokens
        )
    # ... rest of existing JSON cleaning logic

This workaround has been tested successfully with OpenAI gpt-4o-mini.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[ecorpnu]

I am also getting 22:51:00.930
Exception in handleNewProject: Request failed with status code 500 and I am using qwen

[prenansantana]

Hi — this is exactly the use case behind #580.

The repo currently hard-codes response_format={"type": "json_object"} on every JSON-producing LLM call. OpenAI accepts that natively, so OpenAI alone should work — but if you've been hitting 500s with OpenAI or alternative providers (Anthropic OpenAI-compat, LM Studio, llama.cpp server, etc.), it's because some runtimes reject that parameter with HTTP 400.

PR #580 adds an LLM_JSON_MODE env var:

• Default json_object — preserves current behavior (OpenAI, Qwen, Ollama all keep working as before).
• Set LLM_JSON_MODE=none to opt out — works with LM Studio, llama.cpp server, Anthropic via the OpenAI-compat endpoint, and any other strict OpenAI-compatible runtime.

I've validated the patch end-to-end against:

• OpenAI cloud (default mode, no change)
• LM Studio + Qwen3-4B MLX (LLM_JSON_MODE=none)
• Anthropic Claude Haiku 4.5 via https://api.anthropic.com/v1/ (LLM_JSON_MODE=none) — full simulation completed, report generated correctly.

Setup recipes for each provider in docs/LOCAL_LLM.md on the PR branch. A 👍 on the PR helps the maintainer prioritize.