[Bug]: memory-llm hardcoded to gpt-4o-mini, breaks silently for non-OpenAI BYOK providers

[wellfitness]

Title suggestion: [Bug]: memory-llm hardcoded to gpt-4o-mini, breaks silently for non-OpenAI BYOK providers

---

Summary

OD's memory-llm subsystem makes cross-turn memory calls hardcoded to gpt-4o-mini. When using BYOK against any provider that isn't OpenAI itself (e.g. api.minimax.io, which has no gpt-4o-mini in its catalog), every memory call fails with HTTP 400. The agent runs without memory between turns and the user gets no visible error.

How to reproduce

1. Install Open Design 1.17.13 on Windows
2. Configure a BYOK provider via OpenAI-compatible protocol with baseUrl other than api.openai.com (e.g. https://api.minimax.io).
3. Trigger any BYOK agent run that requires cross-turn memory (i.e. essentially any multi-turn task).
4. Check %APPDATA%\Open Design\namespaces\release-stable-win\logs\daemon\latest.log.

Observed log excerpt (verbatim)

[provider:models] openai api.minimax.io → 8 models in 620ms
[proxy:openai] POST api.minimax.io model=MiniMax-M3
[memory-llm] openai call failed openai 400: {"type":"error","error":{"type":"bad_request_error","message":"invalid params, unknown model 'gpt-4o-mini' (2013)","http_code":"400"},"request_id":"06980b8869091118c9df2ca9db999302"}
[proxy:openai] POST api.minimax.io model=MiniMax-M3
[memory-llm] openai call failed openai 400: {"type":"error","error":{"type":"bad_request_error","message":"invalid params, unknown model 'gpt-4o-mini' (2013)","http_code":"400"},"request_id":"06980c066a7c9d9b602a8bf676a50a3b"}

Every proxy:openai chat call is paired with a memory-llm 400 failure. Silent.

Impact

• Agent runs without persistent memory across turns. The user can't tell because there's no error in the UI; only daemon log noise.
• In my specific repro: combined with the BYOK tools-forwarding bug (see [Bug]: BYOK MiniMax/MiMo via OpenAI-compatible protocol returns empty assistant turn and only chat text #5114), the agent produces output that depends on memory that's missing. Memory-less runs hallucinate project context.
• Affects every BYOK provider that isn't OpenAI itself: MiniMax/MiMo, OpenRouter (probably), Together, Groq, local Ollama, anything not OpenAI.

Diagnosis context (from @lefarcen on #5114)

Per maintainer investigation on main:

"memory-llm is not unconditionally hardcoded to OpenAI — it now prefers the per-turn BYOK chatProvider snapshot when the web app sends it, and only falls back to defaults like gpt-4o-mini when that path is missing or loses the race."

So in main there's at least a partial fix (prefer per-turn snapshot), but the fallback path still uses gpt-4o-mini and is reachable. My 1.17.13 build predates that fix and is hit every time.

Expected behavior

memory-llm should:

1. Use the same provider / model / baseUrl / apiKey that the chat turn is using (or refuse to run if it can't be determined).
2. If a fallback is needed, prefer a model that exists in the resolved provider's catalog (e.g. for api.minimax.io, use MiniMax-M2.5-highspeed or similar).
3. Surface failures to the user-visible run status, not just the daemon log. A "memory subsystem degraded" warning in the run metadata would be acceptable.

Workaround (until fixed)

None on the runtime side — the failure is invisible. Workaround at the user level: switch to the CLI route (minimax-coding-plan/MiniMax-M3) where memory goes through opencode-ai's managed plan and isn't affected. But this defeats the purpose of BYOK.

Related

• [Bug]: BYOK MiniMax/MiMo via OpenAI-compatible protocol returns empty assistant turn and only chat text #5114 — original BYOK OpenAI-compat bug report
• PR fix: finalize BYOK runs as failed when only raw tool_call markup is returned (OpenAI-compatible protocol) #5155 — narrow rawToolCallOnly guard (this issue is independent)
• Per maintainer comment on [Bug]: BYOK MiniMax/MiMo via OpenAI-compatible protocol returns empty assistant turn and only chat text #5114: memory-llm path is separate from chat path; should get its own fix thread

---

Originally requested by @lefarcen in #5114 (comment) to keep the tool-forwarding discussion focused while the memory regression gets its own tracking path.

[lefarcen]

Hey @wellfitness — thanks for splitting this out cleanly and for including the exact daemon log shape. That made this much easier to verify against the current code.

What I can confirm from main is that memory-llm is no longer unconditionally pinned to OpenAI: the web-side contract now carries a per-turn chatProvider snapshot, /api/memory/extract forwards it, and memory-llm.pickProvider() prefers that BYOK snapshot before falling back to legacy defaults. So your 1.17.13 repro makes sense as a pre-fix build, and it also supports the narrower remaining bug you called out: when that snapshot is missing or loses the race, the OpenAI-family fallback still lands on gpt-4o-mini, which is wrong for non-OpenAI BYOK providers like api.minimax.io.

So the issue here looks real in two layers:

• older stable builds can fall back to gpt-4o-mini on every memory pass, producing the exact 400 you captured;
• current main reduced that risk, but the fallback path is still OpenAI-shaped and the failure is still silent from the user’s point of view.

The product-facing part I’d want to keep central in this thread is your last point: even when extraction degrades, the run should surface that visibly instead of only leaking through daemon logs. That gives us a clean follow-up target alongside the provider-selection fix.

This is a good separate tracker from #5114 / #5155, and the repro here is specific enough to keep it focused. If you do end up testing a fix build later, your MiniMax setup would be especially valuable for confirming both the provider-selection path and the user-visible warning path. ❤️

[xxiaoxiong]

/claim

[lefarcen]

谢谢认领这个问题，@xxiaoxiong！

你这边大概预计多久方便处理完？几小时内 / 1-3 天 / 大约 1 周 / 更久 都可以，给个大致预估就行，这样我们后面的 check-in 节奏会更合适。

[xxiaoxiong]

@lefarcen 感谢反馈！我已在本地分析 root cause，预计 几小时内 完成修复并提交 PR。修复方向：

• BYOK chat-config snapshot 路径已通，但 fallback 时仍用 （)
• 当 带自定义 baseUrl 时， 应优先于 ，以确保非 OpenAI provider（如 MiniMax）能正常工作
• 同时会在 fallback 失败时让状态可见（而非静默失败）

修复完成后会更新进展。"

[xxiaoxiong]

@lefarcen 感谢反馈！预计 几小时内 能提交 PR。当前 的 BYOK snapshot 路径已接近完整，但还剩两个问题：

1. route 层过滤 chatProvider provider 列表不包含 / 等 provider，导致非 OpenAI 协议 BYOK 用户的 chatProvider 被丢弃
2. legacy fallback（env/media-config 路径）仍是 OpenAI 硬编码，对 MiniMax BYOK 用户不适用

正在分析最小改动的修复方案。

[xxiaoxiong]

已提交 PR #5177，修复完成。

根因：两处代码导致用户 chat model 在到达 pickProvider() 前被丢弃：

1. Web 端 byokChatProvider snapshot 缺 model 字段（只传了 protocol/apiKey/baseUrl/apiVersion）
2. Daemon 端 /api/memory/extract route 没读 body.chatModel（其他 memory route 都有）

修复验证：

• Web typecheck 通过
• 改动模式跟现有三个 sibling route 完全一致（lines 408, 456, 504）
• 不影响 PROVIDER_DEFAULTS 的 env/media-config fallback 路径

@lefarcen 关于你之前提到的 user-visible warning 问题，这个跟进作为后续独立 issue 会更合适，不会堵在这个 PR 里。请 review，感谢！

[xxiaoxiong]

PR #5177 已提交。变更摘要：两处共 6 行修改——Web 端 ProjectView.tsx BYOK snapshot 加 model 字段，Daemon /api/memory/extract route 加 chatModel 提取和传递。typecheck 通过，改动跟其他三个 sibling route 完全一致。请 review，感谢！

[xxiaoxiong]

已提交 PR #5177。两处改动：

apps/web/src/components/ProjectView.tsx (+1 line)：BYOK snapshot 加 model 字段
apps/daemon/src/routes/memory.ts (+5 lines)：/api/memory/extract 提取并转发 body.chatModel

@lefarcen 请 review，感谢！

[lefarcen]

收到，@xxiaoxiong —— 这次两处改动的范围很清楚：Web 端把 model 带进 BYOK snapshot，Daemon 端把 body.chatModel 继续传到 /api/memory/extract，方向和这个问题对得上。

我这边会把详细 review 放到 PR #5177 里跟进，issue 这边先保留给后续验证 / 收尾用。谢谢你这么快把 root cause 和最小修复一起收敛出来。 🙌

[lefarcen]

收到，@xxiaoxiong —— 你补的这版两处改动说明已经很具体了，和这个问题的修复路径是对齐的。

我这边确认一下节奏：详细的阻塞/修改点现在都放在 PR #5177 里继续跟进，issue 先保持打开，等 PR 路径走完之后再回来做验证和收尾。谢谢你把改动范围整理得这么清楚。 🙌