Ternary Bonsai 2 27B is PrismML's 1.58-bit ternary model (PTQ1_0, 5.9 GB). This repo serves it with the full
262,144-token window on 12 GB and 8 GB NVIDIA cards. It has a patched PrismML llama.cpp,
a launcher that sizes the K/V cache for the card (q4_0 on 12 GB, q3_0 on 8 GB), MTP drafting, and an optional layer
for agents. The weights are not changed.
Most of the code and agent losses people report on this model come from the runtime, not from the compression. With
the same weights, the server fixes them: effort words that the template rejects, output caps that end the thinking,
the xhigh template default (runaway thinking), and tool calls in JSON. Details: docs/quality.
- Download
bonsai-bundle-win-x64.zipfrom Releases and unzip it into this repo. - Put
Ternary-Bonsai-2-27B-PTQ1_0.ggufand the MTP head file inmodels\(details). - Run
.\start-server.ps1. The OpenAI-compatible API is onhttp://<host>:8080/v1.
Hugging Face, Linux and builds from source: docs/build.
| Directory | What |
|---|---|
| docs/12gb | 12 GB cards: setup, the q4_0 cache, decode and prefill by depth |
| docs/8gb | 8 GB cards: setup, the q3_0 cache, choosing the window, numbers, two weeks on 8 GB |
| docs/quality | KV precision, reasoning effort, tool calls, HumanEval; benchmark your own model |
| docs/agents-layer | the Bonsai layer (API cards, sandboxed Python), agent use, saved conversations |
| docs/build | install, Linux, build from source, launcher settings, vision, the patch stack, upgrading |
| docs/speed | decode and prefill past the VRAM line, ideas tested and not adopted |
| docs/receipts | raw logs behind the numbers; measure it yourself |
| docs/research | every study with its plan, gates and records |
| docs/cards | the card sources: each card keeps its numbers in one place |
Credits and thanks: docs/CREDITS.md. MIT for everything here; weights are PrismML's (Apache 2.0). Not affiliated with PrismML.