Skip to content

Commit 186365a

Browse files
authored
Update README for BitNet Embedding 0.6B/270M release (#589)
* Update README for BitNet Embedding 0.6B/270M release - Add NEW section with performance highlights and quick start guide - Add embedding prefill performance comparison chart - Add header banner for embedding release - Update Official Models table with embedding models (x86 only) - Add What's New entry for HuggingFace release * Update README for BitNet Embedding 0.6B/270M release - Add NEW section with performance highlights and quick start guide - Add embedding prefill performance comparison chart - Add header banner for embedding release - Update Official Models table with embedding models (x86 only) - Add What's New entry for HuggingFace release
1 parent 6f22876 commit 186365a

3 files changed

Lines changed: 44 additions & 3 deletions

File tree

README.md

Lines changed: 44 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,20 @@
22
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://opensource.org/licenses/MIT)
33
![version](https://img.shields.io/badge/version-1.0-blue)
44

5-
[<img src="./assets/header_model_release.png" alt="BitNet Model on Hugging Face" width="800"/>](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T)
5+
[<img src="./assets/header_embedding_release.png" alt="BitNet Embeddings on Hugging Face" width="800"/>](https://huggingface.co/microsoft/BitNet-embedding-0.6B)
6+
7+
We are excited to release **[BitNet-embedding-0.6B](https://huggingface.co/microsoft/BitNet-embedding-0.6B)** and **[BitNet-embedding-270M](https://huggingface.co/microsoft/BitNet-embedding-270M)**, the first 1-bit embedding models that deliver competitive embedding quality with significantly faster inference on CPUs. Key highlights:
8+
9+
- **1.42x to 2.28x speedup** over F16 on bitnet-embedding-0.6B prefill (8 threads)
10+
- **1.32x to 1.74x speedup** over F16 on bitnet-embedding-270M prefill (8 threads)
11+
- Supports I2_S conversion with optimized kernels on x86 CPUs
12+
- Lossless inference with 2 bits per weight
13+
14+
<img src="./assets/embedding_prefill_performance.png" alt="BitNet Embedding Prefill Performance" width="800"/>
15+
16+
For detailed usage and technical information, see the [BitNet Embeddings I2_S Guide](docs/bitnet-embeddings-i2s-guide.md).
17+
18+
## About bitnet.cpp
619

720
Try it out via this [demo](https://demo-bitnet-h0h8hcfqeqhrf5gf.canadacentral-01.azurewebsites.net/), or build and run it on your own [CPU](https://github.com/microsoft/BitNet?tab=readme-ov-file#build-from-source) or [GPU](https://github.com/microsoft/BitNet/blob/main/gpu/README.md).
821

@@ -14,15 +27,15 @@ The first release of bitnet.cpp is to support inference on CPUs. bitnet.cpp achi
1427

1528
<img src="./assets/performance.png" alt="performance_comparison" width="800"/>
1629

17-
1830
## Demo
1931

2032
A demo of bitnet.cpp running a BitNet b1.58 3B model on Apple M2:
2133

2234
https://github.com/user-attachments/assets/7f46b736-edec-4828-b809-4be780a3e5b1
2335

2436
## What's New:
25-
- 07/16/2026 [BitNet Embeddings 0.6B/270M: I2_S Conversion and Inference Optimization](docs/bitnet-embeddings-i2s-guide.md) ![NEW](https://img.shields.io/badge/NEW-red)
37+
- 07/20/2026 [BitNet-embedding-0.6B and BitNet-embedding-270M on Hugging Face](https://huggingface.co/microsoft/BitNet-embedding-0.6B) ![NEW](https://img.shields.io/badge/NEW-red)
38+
- 07/16/2026 [BitNet Embeddings 0.6B/270M: I2_S Conversion and Inference Optimization](docs/bitnet-embeddings-i2s-guide.md)
2639
- 01/15/2026 [BitNet CPU Inference Optimization](https://github.com/microsoft/BitNet/blob/main/src/README.md)
2740
- 05/20/2025 [BitNet Official GPU inference kernel](https://github.com/microsoft/BitNet/blob/main/gpu/README.md)
2841
- 04/14/2025 [BitNet Official 2B Parameter Model on Hugging Face](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T)
@@ -65,6 +78,34 @@ This project is based on the [llama.cpp](https://github.com/ggerganov/llama.cpp)
6578
<td>&#9989;</td>
6679
<td>&#10060;</td>
6780
</tr>
81+
<tr>
82+
<td rowspan="2"><a href="https://huggingface.co/microsoft/BitNet-embedding-0.6B">BitNet-embedding-0.6B</a></td>
83+
<td rowspan="2">0.6B</td>
84+
<td>x86</td>
85+
<td>&#9989;</td>
86+
<td>&#10060;</td>
87+
<td>&#10060;</td>
88+
</tr>
89+
<tr>
90+
<td>ARM</td>
91+
<td>&#10060;</td>
92+
<td>&#10060;</td>
93+
<td>&#10060;</td>
94+
</tr>
95+
<tr>
96+
<td rowspan="2"><a href="https://huggingface.co/microsoft/BitNet-embedding-270M">BitNet-embedding-270M</a></td>
97+
<td rowspan="2">270M</td>
98+
<td>x86</td>
99+
<td>&#9989;</td>
100+
<td>&#10060;</td>
101+
<td>&#10060;</td>
102+
</tr>
103+
<tr>
104+
<td>ARM</td>
105+
<td>&#10060;</td>
106+
<td>&#10060;</td>
107+
<td>&#10060;</td>
108+
</tr>
68109
</table>
69110

70111
## Supported Models
144 KB
Loading
23.8 KB
Loading

0 commit comments

Comments
 (0)