Qwen3.8-27B on a laptop CPU: 8 GB RAM and same-GGUF bit-exact inference

[shyringo]

I built qwen3.8-27b-in-c, a native C inference engine for running Qwen3.8-27B locally on a single laptop CPU.

It implements the Qwen3.8 graph directly, including 48 Gated DeltaNet layers, 16 full-attention layers, recurrent state, GGUF loading, low-bit CPU kernels, the tokenizer, sampling, and multi-turn chat. It does not require a GPU, CUDA, Python, PyTorch, model conversion, or another inference runtime.

On an Intel Core i5-1340P laptop with 32 GB host memory, the best measured resident-request result with Unsloth Dynamic V3 IQ1_M was:

• TTFT: 4.064 s
• TPOT: 0.397 s/token
• Throughput: 2.52 token/s
• Peak RSS: 6.13 GiB

The IQ1_M path also completed a normal first-token request under an 8 GiB process-address limit. Machines with more memory can automatically select the higher-quality Dynamic V3 Q4_K_M file.

For inference accuracy, comparisons always use the exact same GGUF. Optimized and native baseline paths produce 100% bit-identical full 248,320-dimensional logits, including layer-major prefill and the optional IQ1 runtime repack. IQ1_M and Q4_K_M themselves remain weight-quality/memory trade-offs; the runtime adds zero accuracy loss beyond the chosen quantization.

Quick start:

git clone https://github.com/shyringo/qwen3.8-27b-in-c.git
cd qwen3.8-27b-in-c
./qwen38.sh

Feedback on the CPU kernels, Gated DeltaNet execution, and laptop deployment path would be very welcome.