Skip to content

Commit 3e89e5f

Browse files
committed
docs: make the comments and the README shorter
Keep each comment to no more than two lines and remove the comments that say again what the code says. Write the README in Simplified Technical English with short sentences.
1 parent 3427a75 commit 3e89e5f

7 files changed

Lines changed: 114 additions & 163 deletions

File tree

.github/workflows/pages.yml

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
11
# Build the page with TinyGo and put it on GitHub Pages.
2-
#
3-
# Nothing binary lives in the repo. The WebAssembly builds of llama.cpp come
4-
# down here, and the Go program is compiled here.
2+
# Nothing binary lives in the repo, so llama.cpp comes down here.
53
name: pages
64

75
on:
@@ -29,8 +27,8 @@ jobs:
2927
- name: Checkout
3028
uses: actions/checkout@v5
3129

32-
# TinyGo 0.41.1 works with Go 1.26 and not with Go 1.27. Go back to
33-
# 'stable' after the release of TinyGo that adds Go 1.27.
30+
# TinyGo 0.41.1 works with Go 1.26 and not with Go 1.27. Use 'stable'
31+
# again after a release of TinyGo that adds Go 1.27.
3432
- name: Install Go
3533
uses: actions/setup-go@v6
3634
with:

Makefile

Lines changed: 12 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,18 @@
11
# Build the page into build/, then serve it.
2-
#
3-
# The first build downloads about 20 MB of llama.cpp, which is cached by the
4-
# tag in the name, so only "make clean" makes it happen again.
2+
# The first build downloads about 20 MB of llama.cpp, and then caches it.
53

64
BUILD_DIR ?= build
75
PORT ?= 8080
86

97
# The version of llama.cpp that the current release of yzma is built against.
108
LLAMA_VERSION ?= $(shell curl -s https://hybridgroup.github.io/llama-cpp-builder/version.json | jq -r .tag_name)
119

12-
# yzma ships yzma-loader.js next to the Go package, so take it from the module
13-
# that go.mod pins rather than keeping a copy here that can drift.
10+
# Take yzma-loader.js from the module that go.mod pins, not from a copy
11+
# here that can drift.
1412
YZMA_DIR = $(shell go list -m -f "{{.Dir}}" github.com/hybridgroup/yzma)
1513

16-
# The yzma command has to be the version that go.mod pins, not whatever is
17-
# already on the PATH: the two move together.
14+
# The yzma command has to be the version that go.mod pins, because the two
15+
# move together.
1816
YZMA_VERSION = $(shell go list -m -f "{{.Version}}" github.com/hybridgroup/yzma)
1917
# GOBIN when it is set, and GOPATH/bin when it is not.
2018
YZMA_BIN = $(firstword $(shell go env GOBIN) $(shell go env GOPATH)/bin)
@@ -26,8 +24,8 @@ all: build
2624

2725
build: llama.cpp program assets
2826

29-
# llama.cpp brings down all three WebAssembly builds: WebGPU, more than one
30-
# thread, and one thread. yzma-loader.js takes the best one at run time.
27+
# This brings down all three WebAssembly builds. yzma-loader.js takes the
28+
# best one at run time.
3129
llama.cpp:
3230
go install github.com/hybridgroup/yzma@$(YZMA_VERSION)
3331
mkdir -p $(BUILD_DIR)
@@ -43,19 +41,18 @@ assets:
4341
cp -f $(YZMA_DIR)/wasm/yzma-loader.js $(BUILD_DIR)/
4442
cp web/* $(BUILD_DIR)/
4543

46-
# The service worker sets the headers this server also sets, so either one is
47-
# enough on localhost. On GitHub Pages only the service worker can.
44+
# This server sets the same headers as the service worker, so either one is
45+
# enough on localhost.
4846
serve:
4947
go run github.com/hybridgroup/yzma/wasm/serve -dir $(BUILD_DIR) -port $(PORT)
5048

51-
# check builds with the standard toolchain, which catches a mistake in the API
52-
# without waiting for TinyGo.
49+
# check builds with the standard toolchain, which is faster than TinyGo.
5350
check:
5451
GOOS=js GOARCH=wasm go build -o /dev/null .
5552
GOOS=js GOARCH=wasm go vet ./...
5653

57-
# test holds a two turn conversation in Node, with no browser. It needs a
58-
# model with a chat template:
54+
# test holds a two turn conversation in Node. It needs a model with a chat
55+
# template.
5956
#
6057
# make test MODEL=~/models/Qwen2.5-0.5B-Instruct-Q4_K_M.gguf
6158
MODEL ?=

README.md

Lines changed: 57 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,20 @@
22

33
![yzma in browser](./images/yzma-in-browser.png)
44

5-
A chat page where the whole language model runs in the tab. No server, no API key, nothing sent anywhere.
5+
A chat page that runs the language model in the browser tab. There is no server,
6+
no API key, and no data goes anywhere.
67

78
**<https://hybridgroup.github.io/yzma-wasm-example/>**
89

910
## How it works
1011

1112
[![yzma logo](https://raw.githubusercontent.com/hybridgroup/yzma/refs/heads/main/images/yzma-logo-full-color-small.png)](https://github.com/hybridgroup/yzma)
1213

13-
The code is written in Go and compiled by TinyGo. Using the [yzma](https://github.com/hybridgroup/yzma) package, the web page runs [llama.cpp](https://github.com/ggml-org/llama.cpp), which has been compiled into a WebAssembly module. It all runs in a local Web Worker for best browser performance.
14+
The code is written in Go and compiled by TinyGo. The page uses the
15+
[yzma](https://github.com/hybridgroup/yzma) package to run
16+
[llama.cpp](https://github.com/ggml-org/llama.cpp), which is compiled into a
17+
WebAssembly module. All of it runs in a local Web Worker for the best browser
18+
performance.
1419

1520
```
1621
index.html
@@ -21,91 +26,92 @@ The code is written in Go and compiled by TinyGo. Using the [yzma](https://githu
2126
(Go, TinyGo) -> (llama.cpp, Emscripten)
2227
```
2328

24-
The page stores no record of the conversation. The conversation data is only in the TinyGo WASM module, and each turn puts the whole conversation back through the model.
29+
The page keeps no record of the conversation. The conversation stays in the
30+
TinyGo WASM module. Each turn sends the full conversation through the model
31+
again.
2532

2633
## Build and run
2734

28-
Needs [TinyGo](https://tinygo.org/getting-started/install/) 0.41 or later, Go
35+
You need [TinyGo](https://tinygo.org/getting-started/install/) 0.41 or later, Go
2936
1.26, `jq`, and `node` for the test.
3037

3138
```
3239
make build
3340
make serve
3441
```
3542

36-
Then open <http://localhost:8080>, press **Load**, and wait for the model to come
37-
down. The default is
38-
[Qwen2.5-0.5B-Instruct Q4_K_M](https://huggingface.co/bartowski/Qwen2.5-0.5B-Instruct-GGUF),
39-
about 400 MB, which the browser caches. Any GGUF URL works as long as the host
40-
sends CORS headers, which Hugging Face does.
43+
Open <http://localhost:8080>, push **Load**, and wait for the download of the
44+
model. The default model is
45+
[Qwen2.5-0.5B-Instruct Q4_K_M](https://huggingface.co/bartowski/Qwen2.5-0.5B-Instruct-GGUF).
46+
It is approximately 400 MB, and the browser caches it. You can use any GGUF URL
47+
if the host sends CORS headers. Hugging Face sends them.
4148

42-
`make build` downloads about 14 MB of llama.cpp into `build/`, compiles the Go
43-
program, and copies the page. Nothing binary lives in the repo.
49+
`make build` downloads approximately 14 MB of llama.cpp into `build/`, compiles
50+
the Go program, and copies the page. No binary files are in the repository.
4451

4552
## The test
4653

47-
A two turn conversation, in Node, with no browser. The second question only
48-
makes sense if the first one is still in the prompt, so a sensible answer means
49-
the chat template came out right:
54+
The test holds a two turn conversation in Node, with no browser. The second
55+
question makes sense only if the first question is still in the prompt. Thus a
56+
sensible answer shows that the chat template is correct.
5057

5158
```
5259
make test MODEL=~/models/Qwen2.5-0.5B-Instruct-Q4_K_M.gguf
5360
```
5461

55-
## Threads, and why there is a service worker
62+
## Threads and the service worker
5663

57-
llama.cpp comes in three WebAssembly builds, and `yzma-loader.js` takes the best
58-
one the browser can run:
64+
llama.cpp has three WebAssembly builds. `yzma-loader.js` selects the best build
65+
that the browser can run.
5966

6067
| Build | What it needs |
6168
| --- | --- |
6269
| `yzma_wasm_webgpu` | WebGPU with f16 shaders, and JSPI. Chrome and Edge 137 and later. |
63-
| `yzma_wasm_mt` | `SharedArrayBuffer`, so a page with the COOP and COEP headers. |
64-
| `yzma_wasm` | Nothing. It works everywhere. |
70+
| `yzma_wasm_mt` | `SharedArrayBuffer`, thus a page with the COOP and COEP headers. |
71+
| `yzma_wasm` | Nothing. It runs everywhere. |
6572

66-
GitHub Pages cannot send the COOP and COEP headers, so without help a browser
67-
gives the page no `SharedArrayBuffer` and llama.cpp runs on one thread. In Node
73+
GitHub Pages cannot send the COOP and COEP headers. Without help, the browser
74+
gives the page no `SharedArrayBuffer`, and llama.cpp runs on one thread. In Node
6875
that is the difference between 0.9 and 9.6 tokens a second on Qwen2.5 0.5B.
6976

70-
So the page loads
71-
[`coi-serviceworker.js`](https://github.com/gzuidhof/coi-serviceworker) first.
72-
It registers a service worker that adds the two headers and reloads the page
73-
once, and from then on the page is cross-origin isolated and the loader takes
74-
the build with every thread. It works on localhost too, so a plain static server
75-
is enough for development. `make serve` sets the headers as well, which makes no
76-
difference but does no harm.
77+
Thus the page loads
78+
[`coi-serviceworker.js`](https://github.com/gzuidhof/coi-serviceworker) first. It
79+
registers a service worker that adds the two headers and reloads the page once.
80+
After that the page is cross origin isolated, and the loader selects the build
81+
with all of the threads. This also operates on localhost, thus a usual static
82+
server is sufficient for development. `make serve` sets the headers too, which
83+
makes no difference but does no damage.
7784

78-
Cross-origin isolation does mean a model has to come from a host that sends CORS
79-
headers. Hugging Face does.
85+
Cross origin isolation makes it necessary for the model to come from a host that
86+
sends CORS headers. Hugging Face sends them.
8087

81-
The line at the top right of the page says which build won. Force one with
82-
`?mode=cpu` or `?mode=webgpu` on the URL.
88+
The line at the top right of the page shows the selected build. To force a
89+
build, add `?mode=cpu` or `?mode=webgpu` to the URL.
8390

8491
## Notes
8592

86-
- The system prompt box tells the model how to answer. The page sends it with
87-
each question, so a change applies to the next answer. An empty box gives the
88-
default prompt back.
89-
- The model has to be smaller than 2 GB. One JavaScript ArrayBuffer holds no
90-
more, so anything larger has to be in GGUF splits.
91-
- Use a model with a chat template. A base model has none, and the page says so,
92-
but the answers wander.
93-
- `ChatApplyTemplate` formats one message at a time, so `prompt` in `main.go`
94-
puts the conversation together one message after another. That is exactly
95-
right for a chatml model such as Qwen. A model whose template puts something
96-
once at the top of a conversation, such as Gemma folding the system message
97-
into the first user turn, comes out slightly off.
98-
- A discrete NVIDIA card does not give f16 shaders in Chrome, so such a machine
99-
falls back to the CPU unless Chrome starts with
93+
- The system prompt box tells the model how to answer. The page sends the prompt
94+
with each question, thus a change applies to the next answer. An empty box
95+
gives the default prompt again.
96+
- The model must be smaller than 2 GB. One JavaScript ArrayBuffer holds no more,
97+
thus a larger model must be in GGUF splits.
98+
- Use a model with a chat template. A base model has no template. The page shows
99+
a message, but the answers are poor.
100+
- `ChatApplyTemplate` formats one message at a time. Thus `prompt` in `main.go`
101+
puts the conversation together one message after the other. This is correct
102+
for a chatml model such as Qwen. It is not fully correct for a model whose
103+
template puts something one time at the top of a conversation, such as Gemma,
104+
which folds the system message into the first user turn.
105+
- A discrete NVIDIA card does not give f16 shaders in Chrome. Such a machine
106+
falls back to the CPU unless you start Chrome with
100107
`--enable-dawn-features=vulkan_enable_f16_on_nvidia`.
101108

102109
## Deploying
103110

104-
`.github/workflows/pages.yml` builds and deploys on every push to `main`. Set
105-
**Settings → Pages → Source** to **GitHub Actions** once, and that is all.
111+
`.github/workflows/pages.yml` builds and deploys on each push to `main`. Set
112+
**Settings → Pages → Source** to **GitHub Actions** one time. That is all.
106113

107114
## License
108115

109-
Apache 2.0, the same as yzma. `web/min.css`
110-
([min](https://mincss.com)) and `web/coi-serviceworker.js` are MIT, and keep
111-
their own notices.
116+
Apache 2.0, the same as yzma. `web/min.css` ([min](https://mincss.com)) and
117+
`web/coi-serviceworker.js` are MIT, and they keep their own notices.

main.go

Lines changed: 20 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,7 @@
11
//go:build js && wasm
22

3-
// A chat that runs entirely in the browser.
4-
//
5-
// llama.cpp is a WebAssembly module. This program drives it through the
6-
// pkg/llamawasm package of yzma, keeps the turns of the conversation, and sends
7-
// each piece of the answer to the page as it arrives.
8-
//
9-
// Build it with TinyGo:
10-
//
11-
// tinygo build -target wasm -o build/yzma.wasm .
12-
//
13-
// See README.md for the rest.
3+
// Package main is a chat that runs a language model in the browser.
4+
// It drives llama.cpp through yzma and sends each piece of the answer to the page.
145
package main
156

167
import (
@@ -23,8 +14,7 @@ import (
2314
)
2415

2516
const (
26-
// modelPath is where the model goes in the filesystem of the llama.cpp
27-
// module, once the page has downloaded it.
17+
// modelPath is where the model goes in the filesystem of llama.cpp.
2818
modelPath = "/models/model.gguf"
2919

3020
// defaultSystem opens a conversation until the page sends another one.
@@ -37,7 +27,7 @@ const (
3727
defaultMaxTokens = 512
3828
)
3929

40-
// A turn is one message of the conversation.
30+
// turn is one message of the conversation.
4131
type turn struct {
4232
Role string
4333
Content string
@@ -68,7 +58,6 @@ func main() {
6858
js.Global().Set("yzmaReset", js.FuncOf(reset))
6959
js.Global().Set("yzmaSetSystem", js.FuncOf(setSystem))
7060

71-
// The page shows this in its system prompt box.
7261
post("system", defaultSystem)
7362
post("ready", backendReport())
7463

@@ -103,9 +92,8 @@ func loadModel(this js.Value, args []js.Value) any {
10392
return nil
10493
}
10594

106-
// openModel(path) loads a model that is already in the filesystem of the
107-
// llama.cpp module. The page downloads one instead; a test puts it there
108-
// itself.
95+
// openModel(path) loads a model that is already in the filesystem of llama.cpp.
96+
// The page downloads one instead, but the test puts it there itself.
10997
func openModel(this js.Value, args []js.Value) any {
11098
path := modelPath
11199
if len(args) > 0 && args[0].Truthy() {
@@ -123,8 +111,8 @@ func open(path string) {
123111

124112
params := llamawasm.ModelDefaultParams()
125113

126-
// A build with WebGPU has a device, so put every layer on it. A build on the
127-
// CPU has none, and the value does nothing there.
114+
// A build with WebGPU has a device, so put every layer on it.
115+
// A build on the CPU has none, and the value does nothing.
128116
if llamawasm.GPUDevice() != "" {
129117
params.NGpuLayers = 999
130118
}
@@ -146,8 +134,8 @@ func open(path string) {
146134

147135
vocab = llamawasm.ModelGetVocab(model)
148136

149-
// One sampler for the whole conversation. A chat wants some variety, which
150-
// is what this chain gives and the greedy sampler does not.
137+
// One sampler for the whole conversation. This chain gives the variety
138+
// that a chat needs and the greedy sampler does not.
151139
if sampler != 0 {
152140
llamawasm.SamplerFree(sampler)
153141
}
@@ -160,8 +148,7 @@ func open(path string) {
160148

161149
history = history[:1]
162150

163-
// A base model has no chat template, and the answers of one in a chat are
164-
// poor. Say so rather than letting the reader wonder.
151+
// A base model has no chat template, and its answers in a chat are poor.
165152
if llamawasm.ModelChatTemplate(model, "") == "" {
166153
post("status", "this model has no chat template, so the answers will wander")
167154
}
@@ -227,8 +214,8 @@ func ask(this js.Value, args []js.Value) any {
227214
return
228215
}
229216

230-
// The whole conversation goes in again each turn, so the state of the
231-
// last one has to go.
217+
// The whole conversation goes in again each turn, so the state of
218+
// the last turn has to go.
232219
if err := llamawasm.MemoryClear(ctx, true); err != nil {
233220
post("error", err.Error())
234221
return
@@ -277,27 +264,23 @@ func ask(this js.Value, args []js.Value) any {
277264
return nil
278265
}
279266

280-
// prompt puts the conversation into the chat format of the model and tokenizes
281-
// it. It drops the oldest turns until what is left has room for an answer.
282-
//
283-
// ChatApplyTemplate takes one message at a time, so a conversation is the
284-
// messages one after another, with the opening of the turn of the assistant on
285-
// the last one. That is what makes the model answer instead of carrying on.
267+
// prompt puts the conversation into the chat format of the model and tokenizes it.
268+
// It drops the oldest turns until what is left has room for an answer.
286269
func prompt(maxTokens int32) []llamawasm.Token {
287270
for {
288271
var text strings.Builder
289272
for i, message := range history {
290273
formatted, err := llamawasm.ChatApplyTemplate(model, message.Role, message.Content, i == len(history)-1)
291274
if err != nil {
292-
// A model with no template, or a build of llama.cpp too old to
293-
// have the call. The bare question is the best that is left.
275+
// The model has no template, or llama.cpp is too old to have
276+
// the call. The bare question is all that is left.
294277
return llamawasm.Tokenize(vocab, history[len(history)-1].Content, true, false)
295278
}
296279
text.WriteString(formatted)
297280
}
298281

299-
// parseSpecial has to be true, or the markers of the template tokenize
300-
// as ordinary text and the model never sees the shape of a chat.
282+
// parseSpecial has to be true, or the markers of the template
283+
// tokenize as ordinary text and the model sees no chat.
301284
tokens := llamawasm.Tokenize(vocab, text.String(), true, true)
302285

303286
if int32(len(tokens)) <= nCtx-maxTokens || len(history) <= 2 {
@@ -313,8 +296,7 @@ func prompt(maxTokens int32) []llamawasm.Token {
313296
}
314297
}
315298

316-
// post sends a message to whatever holds this module. In a Web Worker that is
317-
// the page, and in Node it is the harness.
299+
// post sends a message to the page in a Web Worker, or to the harness in Node.
318300
func post(kind, text string) {
319301
message := map[string]any{"kind": kind, "text": text}
320302

0 commit comments

Comments
 (0)