Feature Request: vlut.cpp

[HaisongDing]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

This paper claims that it can achieve a 3 times speedup compared with TQ2_0 llama.cpp on CPU for bitnet using a LUT-based inference engine. Can you have a look and see if it can be integreted into the ik_llama,cpp engine.

Paper: https://arxiv.org/pdf/2512.06443
Code: https://github.com/Cipherxzc/vlut.cpp

Motivation

This paper claims that it can achieve a 3 times speedup compared with TQ2_0 llama.cpp on CPU for bitnet using a LUT-based inference engine.

Possible Implementation

Code: https://github.com/Cipherxzc/vlut.cpp

[ikawrakow]

My suggestion is to 1st compare the ik_llama.cpp Bitnet implementation with that paper. I wouldn't be surprised if ik_llama.cpp is significantly faster, especially when there are 2 or more CPU cores involved. The Bitnet quantization types here are IQ2_BN and IQ1_BN. IQ2_BN is faster, but uses 2 bits per weight (bpw). IQ1_BN uses 1.625 bpw.