Could Pi underperform when models are tuned for native harnesses?

[marcusraty]

Could Pi’s minimal read/write/edit/bash harness perform worse than provider-native harnesses like Claude Code or Codex because frontier models may be trained using RL/gyms or evaluated around their own tool shapes, prompts, patch formats, and search workflows?

[jukofyork]

There is at least one example of this for kimi-2.5 running in opencode:

anomalyco/opencode#20258

anomalyco/opencode#20259

[marcusraty]

Thanks for your reply. Do you think those examples are related to system prompts which are specific to models vs what I was suggesting which is that frontier model companies are using different kinds of training to make their models works best with their Tool interfaces.

An example would be that Claude Opus is trained on examples of / post trained to use Claude Code harness style Tool calls Read() Edit() while GPT is trained on codex style Tool calls Bash(). That could mean these models may not work as well in pi or opencode harnesses.

[aussetg]

Absolutely yes.

It is best if possible to try to "match" the harness the specific LLM was trained in. So for example if you love/use GPT it's best to have apply_patch instead of edit/write