Skip to content

Add support for Jinja-based tool calling (MiniCPM5 and other recent models).#1014

Draft
vladislavdonchev wants to merge 2 commits into
mozilla-ai:mainfrom
Independent-AI-Labs:llamacpp_c818263f2
Draft

Add support for Jinja-based tool calling (MiniCPM5 and other recent models).#1014
vladislavdonchev wants to merge 2 commits into
mozilla-ai:mainfrom
Independent-AI-Labs:llamacpp_c818263f2

Conversation

@vladislavdonchev

Copy link
Copy Markdown

Description

Bumps the llama.cpp submodule to c818263f2, which adds upstream MiniCPM5 chat/tool-call parsing (ggerganov/llama.cpp#24889). Regenerates the full llama.cpp.patches/ set so the cosmocc build still applies cleanly, and wires new upstream sources into the llamafile build.

Branch: llamacpp_c818263f2 (per CONTRIBUTING.md / bump workflow — not committed directly to main)

Submodule

  • llama.cppc818263f2 (chat: implement minicpm5 parser)

Patch / build reconciliation

  • Regenerated all 39 patches via generate-patches after proving a clean build + make check
  • llamafile-files/BUILD.mk: drop removed json-partial.cpp; add dflash.cpp and server-stream.cpp
  • llamafile/BUILD.mk: link server-stream.cpp.o in LLAMAFILE_SERVER_SUPPORT_OBJS
  • New src_models_dflash.cpp.patch: cosmocc/clang fix — move build_arch_graph() after graph<true>/graph<false> template specializations (same pattern as the existing eagle3 patch)
  • Refreshed drifted hunks in backend/server patches (CUDA, Vulkan, CPU ops, server models, etc.)

Tests

  • Tool-calling integration tests pass --jinja -ngl 0 so Jinja-templated models (including MiniCPM5) parse tool calls correctly

Verification performed locally

  • make setup — all patches apply cleanly
  • Clean build of o/llamafile/llamafile
  • make check — unit tests green
  • llamafile:verify-clean round-trip (reset → setup → clean build → check)
  • Integration: tests/test_tool_calling.py (3/3) against built binary and rebundled .llamafile with MiniCPM5-1B Q8_0

Host handoff (not run in-session — per bump doc)

  • CUDA / ROCm smoke test
  • Windows smoke test
  • macOS Metal runtime compile + run
  • Web UI curl check (GET /, /_app/immutable/...)
  • Long-run stability

PR Type

  • 🆕 New Feature
  • 🚦 Infrastructure

Relevant issues

  • Enables MiniCPM5 tool calling via upstream ggerganov/llama.cpp#24889
  • Searched mozilla-ai/llamafile issues and open PRs — no duplicate; this bump is novel
  • No pre-discussion issue filed: confined routine llama.cpp submodule patch bump (not a new user-facing feature or architectural change per CONTRIBUTING.md §Discuss major changes)

Checklist

  • I understand the code I am submitting.
  • I have run this code locally and verified the change.
  • New and existing tests pass locally, or I have explained why tests were not run.
  • Documentation was updated where necessary.
  • If I changed code in llama.cpp/, whisper.cpp/, or stable-diffusion.cpp/, I also updated the matching *.patches/ files.
  • I have read and followed the contribution guidelines.
  • AI Usage:
    • No AI was used.
    • AI was used in an assistive capacity.
    • This PR includes substantial AI-generated content.

AI Usage Information

  • AI Model used: Grok (xAI, Research, Code), MiniCPM5 (Action Classifier)
  • AI Developer Tool used: opencode an autonomous VM sandbox
  • Any other info you'd like to share: AI assisted with the canonical llama.cpp bump workflow (check_patches.sh → reconcile → generate-patchesverify-clean), patch regeneration, BUILD.mk reconciliation for upstream file moves, the cosmocc dflash.cpp compile-order fix, integration-test updates (--jinja), and local verification (build, unit tests, MiniCPM5 tool-calling integration tests). All changes were reviewed against llamafile's patch workflow docs and CONTRIBUTING.md; submodule edits were made only in-place for patch generation, with llama.cpp.patches/ as the committed source of truth.

When answering reviewer questions, please respond yourself rather than pasting reviewer comments into an AI system and posting the reply back unchanged.

  • I am an AI Agent filling out this form (check box if true)

workspace-agent added 2 commits July 10, 2026 07:44
Fold the reconciled llama.cpp bump (c818263f2, MiniCPM5 parser #24889)
into the canonical patch set after proving a clean build and unit tests.

- Regenerate 39 patches via generate-patches; add dflash cosmocc compile fix
- Update BUILD.mk: drop removed json-partial, add dflash + server-stream sources
- Link server-stream.o in llamafile/BUILD.mk LLAMAFILE_SERVER_SUPPORT_OBJS
- Pass --jinja -ngl 0 in tool_calling integration tests for Jinja-templated models

Verified: generate-patches → verify-clean (setup + clean build + check) green;
MiniCPM5 tool_calling integration tests pass (direct build and .llamafile bundle).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant