llamafile v0.10.3

What's Changed

This release is mainly a bugfix to a regression introduced in 0.10.2 (many many thanks to @niutech for opening the issue! 🙏)

As releases become more frequent, we are working to better automate the process and making this more open. This PR brings a new llamafile/release.sh script that's the one used to create all the new release files (we'll be testing it until the next release before updating the RELEASE.md doc).

• Update release scripts by @aittalam in #990
• Fix uncaught SIGSEGV when GPU init fails, restore CPU fallback (#988) by @aittalam in #989

Full Changelog: 0.10.2...0.10.3

llamafile v0.10.2

What's Changed

• Add gpu accel probe by @aittalam in #953
• Added executable dir to list of dylib valid paths by @aittalam in #959
• Update README with Windows executable size note by @ParthProLegend in #962
• Update llama.cpp submodule to 7b8443ac7 by @aittalam in #951
• Reduce cuda dylibs size by @aittalam in #963
• Clarify Linux GPU offload diagnostics by @adminlip in #967
• docs: rename example_llamfiles to pre-built-llamafiles for better seo by @angpt in #972
• ggml-cpu: re-hook llamafile MoE matmul on x86 (partial fix for #973) by @aittalam in #974
• CPU flash-attention fixes for #975 (workaround + AVX-512 helpers + simd_gemm) by @aittalam in #978
• common.cpp: fix Apple Silicon physical-core detection under cosmocc (#976) by @aittalam in #977
• Updated version to 0.10.2 by @aittalam in #984
• Modernise Diffusionfile Support by @cbk-res in #970
• Update llama.cpp submodule to dbe9c0c (+ embed real web UI) by @aittalam in #983

New Contributors

• @ParthProLegend made their first contribution in #962
• @adminlip made their first contribution in #967
• @cbk-res made their first contribution in #970

Full Changelog: 0.10.1...0.10.2

llamafile v0.10.1

What's Changed

Summary:

• Added support for vulkan dylibs
• Added windows build scripts -> we now have cuda, rocm, vulkan both as .so and as .dll libraries
• Updated llama.cpp submodule to 5e9c63546 -> we now have llama.cpp support for new models (e.g. gemma-4, bonsai, Qwen3.6) and functionalities (e.g. llama.cpp internal tools for agents)

Details:

• Fix README_0.10.0 by @aittalam in #918
• Add support for vulkan dylibs by @aittalam in #892
• Add tinyblasStrsmBatched kernel by @aittalam in #923
• Fix: GGUF Q5_1 quant crashes llamafile on aarch64 cpu by @aittalam in #928
• fix broken link 'feel free to choose' -> example llamafiles by @bquast in #927
• Fix block-size assumption by @aittalam in #935
• Add windows build scripts for CUDA by @aittalam in #924
• Fix whisperfile documentation link in README by @martin0258 in #939
• Update llama.cpp submodule to 5e9c63546 by @aittalam in #941
• Migrate docs from MkDocs/GitHub Pages to GitBook by @angpt in #946
• Win build improvements by @aittalam in #940
• Fix cuda: /lib/x86_64-linux-gnu/libstdc++.so.6: version GLIBCXX_3.4.32' not found` by @aittalam in #948
• Docs Updates by @angpt in #949
• New release version: update version.h and docs by @aittalam in #952

New Contributors

• @bquast made their first contribution in #927
• @martin0258 made their first contribution in #939
• @angpt made their first contribution in #946

Full Changelog: 0.10.0...0.10.1

llamafile v0.10.0

llamafile versions starting from 0.10.0 use a new build system, aimed at keeping our code more easily
aligned with the latest versions of llama.cpp. This means they support more recent models and functionalities,
but at the same time they might be missing some of the features you were accustomed to (check out this doc for a high-level description of what has been done).

If you liked the "classic experience" more, you will always be able to access the previous versions from our releases page. Our pre-built llamafiles show which version of the server they have been bundled with (0.9.* example, 0.10.* example), so you will always know which version of the software you are downloading.

What's Changed

• Accept array in chat message content field by @henfiber in #760
• chore: Update README.md to include call for community feedback on llamafile by @njbrake in #812
• chore: integrate whisper.cpp as a submodule by @njbrake in #813
• chore: convert stable diffusion to submodule by @njbrake in #818
• chore: llama.cpp as submodule by @njbrake in #819
• feat: move docs to mkdocs by @njbrake in #824
• chore: Add update-llama-cpp workflow. by @daavoo in #846
• fix(update-llama-cpp): Use new_build_wip as base ref. by @daavoo in #850
• Fixed broken llamafile URL in docs by @aittalam in #873
• update supported OpenBSD versions by @sthen in #897
• llamafile reloaded (v0.10.0) by @aittalam in #867

New Contributors

• @henfiber made their first contribution in #760
• @njbrake made their first contribution in #812
• @daavoo made their first contribution in #846
• @sthen made their first contribution in #897

Full Changelog: 0.9.3...0.10.0

llamafile v0.9.3

What's Changed

• Fix link to troubleshooting guide by @rsanheim in #740
• Preserve URL path when building relative URLs in JS by @dmcardle in #733
• Add Plaintext output option to LocalScore + Respect NO_COLOR env var by @cjpais in #742
• Update README.md, fix llama 8B table stats by @cjrh in #745
• Add phi4 support by @cjpais in #744
• Qwen3 Support by @cjpais in #743

New Contributors

• @rsanheim made their first contribution in #740
• @dmcardle made their first contribution in #733
• @cjrh made their first contribution in #745

Full Changelog: 0.9.2...0.9.3

llamafile v0.9.2

Llamafile

Llamafile v0.9.2 is a significant release. It adds support for:

• DeepSeek Distil R1 Models
• Gemma 3
• IBM Granite

LocalScore

In addition 0.9.2 introduces LocalScore, a benchmarking utility and website.

LocalScore is an open-source tool that both benchmarks how fast Large Language Models (LLMs) run on your specific hardware and serves as a repository for these results. We created LocalScore to provide a simple, portable way to evaluate computer performance across various LLMs while making it easy to share and browse hardware performance data.

LocalScore is now part of the release of Llamafile under the new CLI utility localscore

You can run it ./localscore -m <model>. It is also included in every llamafile so you can benchmark models on your hardware easily using ./llamafile --localscore.

LocalScore was created with support from Mozilla Builders.

What's Changed

• [llamafiler] doc/v1_chat_completions.md: remove duplicate entry by @mseri in #607
• Update server readme with code completion (FIM) example by @heaversm in #637
• URL constructor to get a clean url_prefix (fix #640) by @sizvix in #641
• Fix translation bug from cpp to js in TS highlight by @emilbayes in #681
• Add whisperfile server documentation by @alonsosilvaallende in #707
• Unify button look and rearrange buttons to make them more compact by @corebonts in #712
• add stable-diffusion.cpp to install target (fix #580) by @rgroesslinger in #635
• Improve OpenAI compatibility for /v1/* endpoints by @corebonts in #713
• Update WSL troubleshooting in README.md by @halter73 in #585
• Granite three support by @gabe-l-hart in #608
• Initial support for Gemma 3 models by @corebonts in #717
• Add copy and info buttons to the chat window and improve small screen UX by @corebonts in #719
• Avoid streaming incomplete UTF-8 characters by @corebonts in #727
• Introduce LocalScore CLI by @cjpais in #734

New Contributors

• @mseri made their first contribution in #607
• @heaversm made their first contribution in #637
• @sizvix made their first contribution in #641
• @emilbayes made their first contribution in #681
• @alonsosilvaallende made their first contribution in #707
• @corebonts made their first contribution in #712
• @rgroesslinger made their first contribution in #635
• @halter73 made their first contribution in #585
• @gabe-l-hart made their first contribution in #608

Full Changelog: 0.9.1...0.9.2

llamafile v0.9.1

This release adds support for DeepSeek Distil models. It improves some documentation, and fixes a segfault when running with an Nvidia GPU.

What's Changed

• Update Makefile: Fix PHONY from check to cosmocc and cosmocc-ci respectively by @mofosyne in #683
• Updated README to reflect WSL 2 command for Windows 11 by @peteski22 in #685
• Add Support for DeepSeek-R1 models by @Xydane in #687
• Revert Cosmopolitan to 3.9.7 by @cjpais in #705

New Contributors

• @peteski22 made their first contribution in #685
• @Xydane made their first contribution in #687

Full Changelog: 0.9.0...0.9.1

llamafile v0.9.0

We've solved all the known issues with the new llamafiler server, and
improved its web gui. In addition to the llamafiler binary below, the
new server is available as llamafile --server --v2. Its manual can be
accessed via llamafile --server --v2 --help.

• e64c7e2 Include llamafiler in llamafile binary
• a8fd4d2 Improve management of multiple slots
• 4158265 Show progress bar for prompt processing in web ui
• 38677b5 Support relocating matching suffixes in KV cache
• 956e62c Visually indicate messages truncated by context
• 08e7a21 Forget old messages when running out of context
• 59a5d97 8fa1702 Make pledge() security not break things
• 1fc35e2 Add upload button and support text files
• 43bc1eb fe514ef Improve buttons in web ui

This change upgrades our cosmocc toolchain, whose recent release has
fixed all known issues and made performance improvements to memory
allocation. See the cosmopolitan releases page.

• c293359 Upgrade to Cosmopolitan v4.0.2

The following improvements have been made to the terminal --chat bot.

• f51e535 Fix emoji editing in chatbot

Additional changes:

• 9b03e32 Call appropriate hip api's (#651)

llamafile v0.8.17

llamafiler has a new web UI which supports two modes of operation:
chatbot and raw completion. Its syntax highlighting is just as advanced
as the CLI chatbot. It looks much nicer than the old web ui. In a future
release, llamafiler will be folded into llamafile to replace the old server.

• 988c9ec Introduce raw completions web ui
• 241bf21 Introduce /v1/completions endpoint in new server
• 6d89f8f Add binary safety check to server
• d18ddf1 Add redo button to new web ui
• bc82424 Add settings modal to web ui
• bb917bd Add vision model support to new server
• 4c7b7d5 Implement data URI parser
• fb4b3e6 Fix JSON parser bug
• 9d6f89f Improve look and printability of new web ui
• 25b6910 Make chatbot ui more printer friendly
• 30518ca Respond to HTTP OPTIONS requests
• 41abfa3 Work around multiple image handling
• 35bc088 Make default system prompt configurable on web
• 28c8e22 Scale and decimate images as needed in browser
• 14713b5 Get basic chatbot web gui working in llamafiler
• ef08074 Start porting syntax highlighter to JavaScript
• fdfdb13 Port remaining highlighting code to javascript

The following improvements have been made to our terminal chatbot.

• 12c3761 Make CLI chatbot work better with base models
• e5c0921 Improve VT100 support
• 4b61791 Fix VT102 support
• d25c077 Introduce /upload and /forget commands to chatbot
• 880ebc7 Handle empty system prompt better in cli chatbot

General improvements to this project.

• f581c40 Fix futex prototype
• 54d3c72 Make LLaVA fast again
• 01b8d49 Remove n-gpu-layer limitation (#534)
• 566cdc1 Improve Gemma system prompt generation
• 46284fe Reduce attack surface of stb_image
• 9bb262b Log CUDA kernel vs. runtime versions

Syntax highlighting improvements for chatbot and web ui.

• d979a1c Add BNF syntax highlighting
• 4a8311a Add cmake syntax highlighting
• 40e92cf Add Ocaml syntax highlighting
• 0995343 Add more Clojure keywords
• 0068a37 Make D syntax highlighting better
• 0965a4b Make some markdown improvements
• 9b96502 Improve JS/HTML syntax highlighting
• c0622da Put more work into markdown rendering
• fa1c98f Improve markdown to html rendering
• 8915432 Further improve markdown to html
• d25fa3a Improve highlighting in new web ui
• f5a0bd4 Fix JS regex highlighting issue
• 2807ae6 Improve Ada syntax highlighting
• d30da30 Syntax highlight D properly
• 33a057e Improve Ruby some more
• 5b0fff1 Improve Ruby syntax highlighting
• 8413a21 Fix Ruby builtins in web gui

The latest cosmopolitan upgrade introduces a new more powerful syntax
for your .args files. They're now parsed more similarly to the shell,
with support for C style escaping in double-quoted strings. You can also
now add shell-style comments to .args files too. See tool/args/args2.c
in the cosmopolitan codebase for the definitive reference.

• fb59488 Upgrade to Cosmo v3.9.7
• 21af0bf Import upstream bestline changes

The following example of the new .args file syntax is provided:

# specify model
-m Qwen2.5-Coder-34B-Instruct.Q6_K.gguf

# prevent flags below from being changed
...

# specify system prompt
--system-prompt "\
you are a friendly ai assistant\n
your job is to be helpful and intelligent"

# hide some stuff from user interfaces
--nologo
--no-display-prompt

You can put .args files inside llamafile, llamafiler, and whisperfile
using the zipalign program.

The following screenshots are provided of the llamafiler web ui.

llamafile v0.8.16

• Add Julia syntax highlighting support
• Fix possible crash on Windows due to MT bug
• Improve accuracy of chatbot context window management
• The new llamafiler server now supports GPU. Pass the -ngl 999 flag.
• The new llamafiler server's /v1/chat/completions endpoint now supports prompt caching. It may be configured using the --slots COUNT and --ctx-size TOKENS flags.