Skip to content

Commit c0e06f5

Browse files
Schaerer DamienSchaerer Damien
authored andcommitted
fix(engine, code): a model nobody could start, a window silently ignored, and six editor defects
THE ENTRY THAT COULD NOT START ON ANY MAC. A registry model whose only measurement is a 92 GB cache, taken on a far larger machine, made eco ask for more than perf: the mode names mean smallest and largest footprint, and unclamped eco returned that 92 GB while perf returned the ceiling. The step-down from perf therefore walked toward a HEAVIER footprint and never converged, so every mode was impossible. The card was shown as installable, min_ram_gb was satisfied, a speed was printed, and the user found out after downloading 202 GB and building a 190 GB pack. Both measured branches are clamped now, like the no-curve branch already was. The test states it as the invariant rather than as a number: if perf can start, eco must be able to start. Verified by removing the fix, where it reports perf planning 14 GB while eco demands 100. THE CONTEXT WINDOW SETTING WAS COSMETIC ON TWELVE MODELS. ctx_per_slot_for clamps the request to what the model declares, and no MoE entry declared anything, so all of them fell to a cautious 32K ceiling while the button stayed lit on whatever had been chosen: picking 128K left 128K on screen over an engine serving 32K. Nine entries now carry the window from their own published config (Qwen3-Coder and Qwen3-Next at 256K, gpt-oss and GLM-4.5-Air at 128K, OLMoE at 4K), and the engine reports the window it is actually serving so the panel can say when a request was capped. The three whose configuration is not public keep the cautious ceiling, which is what they had. SIX IN THE EDITOR. The change gutter never updated while typing: the marks were computed once at open and frozen, so twenty lines in, the bar still described the file as it had been. The debounce and the cache were already there for it. Creating, renaming or deleting from the tree left the file index stale for the whole session: a new file was not offered by the palette and a renamed one kept answering under its old path in project search. The outline recomputed on every pause in typing whether or not anyone was looking at it, forcing a syntax tree with a 50 ms budget on the main thread for a panel that was not on screen. And in a split, the linter ran the other pane's document under the focused pane's filename: a .rs on the right was diagnosed as if it were the .md on the left, so its sources went silent or returned another file's positions. The linter now reads the path from the state it is given, which is exactly what the docRel facet exists for.
1 parent 745abcf commit c0e06f5

7 files changed

Lines changed: 147 additions & 17 deletions

File tree

app/src-tauri/src/lib.rs

Lines changed: 75 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -248,6 +248,57 @@ fn root_set(path: String) -> Result<(), String> {
248248
})
249249
}
250250

251+
#[cfg(test)]
252+
mod cache_ceiling_tests {
253+
use super::*;
254+
255+
/// If perf can start, eco must be able to start.
256+
///
257+
/// That is what the three modes mean: eco is the smallest footprint, perf
258+
/// the largest. The registry entry that broke it has ONE measured point, a
259+
/// 92 GB cache taken on a machine far bigger than the one being planned
260+
/// for, and eco returned that number unclamped. So eco asked for MORE than
261+
/// perf, the step-down from perf walked toward a heavier footprint, and
262+
/// every mode came back impossible, on a card the app had shown as
263+
/// installable after a 202 GB download.
264+
///
265+
/// Stated as an invariant between modes rather than as a number, because
266+
/// the numbers depend on the machine and the invariant does not.
267+
#[test]
268+
fn eco_is_never_heavier_than_perf() {
269+
let entry = json!({
270+
"id": "single-point",
271+
"gguf_bytes": 60_000_000_000u64,
272+
"expert_bytes_total": 50_000_000_000u64,
273+
"non_expert_bytes": 4_000_000_000u64,
274+
"record_bytes": 13_000_000u64,
275+
"experts": 128,
276+
"experts_used": 8,
277+
"layers_moe": 40,
278+
"min_cache_bytes": 5_000_000_000u64,
279+
"status": "certified_bit_transparent",
280+
// The only measurement comes from a much larger machine.
281+
"measured": [{"cache_gb": 92.0, "gen_tps": 6.0, "mac_gb": 512}],
282+
});
283+
for ram in [32u64, 64, 128] {
284+
let machine = MachineLimits { ram_gb: ram, available: None, gpu_working_set: None };
285+
let perf = plan_cache(&entry, machine, None, "perf", false, 1, CTX_PER_SLOT);
286+
let eco = plan_cache(&entry, machine, None, "eco", false, 1, CTX_PER_SLOT);
287+
if let Ok(p) = perf {
288+
let e = eco.unwrap_or_else(|err| {
289+
panic!("{ram} GB: perf planned {} but eco refused: {err}", p.cache_bytes)
290+
});
291+
assert!(
292+
e.cache_bytes <= p.cache_bytes,
293+
"{ram} GB: eco planned {} against perf's {}",
294+
e.cache_bytes,
295+
p.cache_bytes
296+
);
297+
}
298+
}
299+
}
300+
}
301+
251302
#[cfg(test)]
252303
mod settings_read_tests {
253304
use super::*;
@@ -1681,10 +1732,18 @@ struct ServerState {
16811732
generation: u64,
16821733
/// Port actually bound by the running server (0 when stopped).
16831734
port: u16,
1684-
/// Engine regime: resident-metal | streamed-metal | cpu-bit-exact.
1735+
/// Engine regime: resident-bit-exact | streamed-bit-exact | cpu-bit-exact
1736+
/// | stock-llamacpp (a dense model, which streams nothing).
16851737
mode: String,
16861738
/// Decode slots the running server was started with (--parallel).
16871739
slots: u32,
1740+
/// Context window per slot the running server was started with.
1741+
///
1742+
/// The setting offers 8K to 128K, and ctx_per_slot_for clamps it to what
1743+
/// the model declares (or to a cautious 32K when it declares nothing). The
1744+
/// UI painted the STORED value, so someone who chose 128K saw 128K on a
1745+
/// server running 32K, with nothing saying the request had been reduced.
1746+
ctx_per_slot: u32,
16881747
/// Measured tool-calling verdict for the running model (see ServerStatus).
16891748
tools_ok: Option<bool>,
16901749
/// The footprint mode this server was actually started in, and why. None
@@ -1705,6 +1764,7 @@ fn server_state() -> &'static Mutex<ServerState> {
17051764
port: 0,
17061765
mode: String::new(),
17071766
slots: 1,
1767+
ctx_per_slot: 0,
17081768
tools_ok: None,
17091769
footprint: None,
17101770
})
@@ -1859,6 +1919,9 @@ struct ServerStatus {
18591919
/// have been cheaper and would have been wrong: it depends on the build,
18601920
/// the chat template and the quantization, not on the model name.
18611921
tools_ok: Option<bool>,
1922+
/// The context window per slot the engine is actually serving, which is
1923+
/// not always the one that was asked for.
1924+
ctx_per_slot: u32,
18621925
/// The memory-footprint decision this engine was started with: the mode
18631926
/// asked for, the mode actually used, and the two numbers that separate
18641927
/// them. The UI says so out loud when they differ, because a user who
@@ -1878,6 +1941,7 @@ fn server_status() -> ServerStatus {
18781941
// Stopped: report what the NEXT start would give, so the UI never
18791942
// promises a concurrency the engine will not have.
18801943
slots: if s.child.is_some() { s.slots } else { engine_slots() },
1944+
ctx_per_slot: s.ctx_per_slot,
18811945
tools_ok: s.tools_ok,
18821946
footprint: s.footprint.clone(),
18831947
}
@@ -3118,7 +3182,14 @@ fn plan_cache(
31183182
return if mode == "perf" || floor == 0 { ceiling } else { floor.min(ceiling) };
31193183
}
31203184
match mode {
3121-
"eco" => (measured[0].0 * 1e9) as u64,
3185+
// Clamped, like the no-curve branch above already was. A measured
3186+
// point is a number from a bigger machine, not a promise this one
3187+
// can keep: a model whose only measurement is a 92 GB cache made eco
3188+
// ask for MORE than perf, so the step-down walked toward a heavier
3189+
// footprint and never reached anything that fitted. The user saw an
3190+
// installable card, downloaded 202 GB, and then could not start it
3191+
// on any Mac.
3192+
"eco" => ((measured[0].0 * 1e9) as u64).min(ceiling),
31223193
"perf" => ceiling,
31233194
_ => {
31243195
// Full residency first, when the ceiling already reaches every
@@ -3144,7 +3215,7 @@ fn plan_cache(
31443215
.fold(measured[0].1, f64::max);
31453216
for (c, t) in &measured {
31463217
if *t >= 0.9 * reachable {
3147-
return (*c * 1e9) as u64;
3218+
return ((*c * 1e9) as u64).min(ceiling);
31483219
}
31493220
}
31503221
ceiling
@@ -4596,6 +4667,7 @@ async fn server_start(app: AppHandle, model_id: String, cache_gb: Option<u64>) -
45964667
s.generation = generation;
45974668
s.port = port;
45984669
s.slots = slots;
4670+
s.ctx_per_slot = ctx_per_slot;
45994671
s.footprint = Some(plan.decision.clone());
46004672
}
46014673
let _ = app.emit(

app/src/api.ts

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -137,12 +137,19 @@ export interface ServerStatus {
137137
model_id?: string;
138138
port: number;
139139
phase: string; // stopped | starting | ready
140-
mode?: string; // resident-metal | streamed-metal | cpu-bit-exact
140+
mode?: string; // resident-bit-exact | streamed-bit-exact | cpu-bit-exact | stock-llamacpp
141141
/**
142142
* Concurrent decode streams the engine serves (llama-server --parallel).
143143
* The hard bound on how many conversations may generate at the same time.
144144
*/
145145
slots?: number;
146+
/**
147+
* The context window per slot the engine is actually serving.
148+
*
149+
* Not always the one that was asked for: each model is capped by the window
150+
* it was trained on, and by a cautious ceiling when it declares none.
151+
*/
152+
ctx_per_slot?: number;
146153
/**
147154
* Whether the running model actually emits tool calls, MEASURED at warmup.
148155
* Undefined while unknown. False disables every agent surface, because

app/src/code.ts

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ import type { PermissionRequest } from "./agent";
3737
import { isElevatedCommand } from "./agent";
3838
import { t, getLang } from "./i18n";
3939

40-
import { Docs, type Doc } from "./code/docs";
40+
import { Docs, type Doc, docRel } from "./code/docs";
4141
import { editorExtensions, intelComp } from "./code/extensions";
4242
import { tabsHtml, onTabsClick } from "./code/tabs";
4343
import { cmPhrases } from "./code/phrases";
@@ -425,7 +425,11 @@ function newRegistry(): Docs {
425425
onSave: () => void saveOpenFile(),
426426
phrases: cmPhrases(getLang()),
427427
language: langFor(rel),
428-
diagnostics: [diagnosticsExtension(() => docs.activeRel())],
428+
// The document this state belongs to, read from the state itself. It
429+
// used to be the FOCUSED pane's file: a .rs on the right was linted as
430+
// if it were the .md on the left, so the sources went silent or, worse,
431+
// put another file's positions on this one.
432+
diagnostics: [diagnosticsExtension((state) => state.facet(docRel))],
429433
}),
430434
oneDark,
431435
syntaxHighlighting(galactusHighlight),
@@ -1437,6 +1441,10 @@ async function newEntry(anchor: string | null, isDir: boolean, folder: boolean):
14371441
deps?.toast(String(e?.message ?? e));
14381442
return;
14391443
}
1444+
// The palette and the project search read a cached file list. Creating,
1445+
// renaming or deleting from the tree left it stale for the whole session:
1446+
// a new file was not offered by Cmd+P and a renamed one kept its old path.
1447+
indexStale = true;
14401448
await refreshDirOf(rel);
14411449
// A new file opens; a new folder opens in the tree, where the next click is.
14421450
if (folder) {
@@ -1464,6 +1472,10 @@ async function renameEntry(rel: string): Promise<void> {
14641472
// the next save cannot recreate the file under the name that just went away.
14651473
const wasOpen = docs.list().some((d) => d.rel === rel);
14661474
if (wasOpen) closeTab(rel);
1475+
// The palette and the project search read a cached file list. Creating,
1476+
// renaming or deleting from the tree left it stale for the whole session:
1477+
// a new file was not offered by Cmd+P and a renamed one kept its old path.
1478+
indexStale = true;
14671479
await refreshDirOf(dest);
14681480
if (wasOpen) await openFile(dest);
14691481
}
@@ -1488,6 +1500,10 @@ async function deleteEntry(rel: string): Promise<void> {
14881500
if (docs.list().some((d) => d.rel === rel)) closeTab(rel);
14891501
expanded.delete(rel);
14901502
treeCache.delete(rel);
1503+
// The palette and the project search read a cached file list. Creating,
1504+
// renaming or deleting from the tree left it stale for the whole session:
1505+
// a new file was not offered by Cmd+P and a renamed one kept its old path.
1506+
indexStale = true;
14911507
await refreshDirOf(rel);
14921508
// Says where it went, because "moved to Trash" and "moved to .galactus/trash"
14931509
// are two different places to go looking for it.
@@ -1558,6 +1574,9 @@ function onEditorUpdate(u: ViewUpdate): void {
15581574
d.state = u.state;
15591575
if (u.docChanged) {
15601576
scheduleOutline();
1577+
// The marks were computed once at open and then frozen: twenty lines typed
1578+
// and the bar still described the file as it was before.
1579+
scheduleChangeBar();
15611580
if (tsActive()) tsintel!.updateBuffer(d.rel, u.state.doc.toString());
15621581
if (isRust(d.rel)) syncRustBuffer(d.rel, u.state.doc.toString());
15631582
}
@@ -2032,6 +2051,9 @@ function scheduleOutline(): void {
20322051
}
20332052

20342053
async function computeOutline(): Promise<void> {
2054+
// Nobody is looking: this forces a syntax tree with a 50 ms budget on the
2055+
// main thread at every pause in typing, for a panel that is not on screen.
2056+
if (leftTab !== "outline") return;
20352057
const d = docs.active();
20362058
if (!d || d.error !== null) {
20372059
outlineItems = [];

app/src/code/diagnostics.ts

313 Bytes
Binary file not shown.

app/src/i18n.ts

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -763,6 +763,10 @@ const dict: Record<string, { en: string; fr: string }> = {
763763
en: "Family, size on disk, and how many of its experts run per token: a Mixture-of-Experts model uses a few of them at a time, which is why it can be larger than your memory.",
764764
fr: "Famille, taille sur le disque, et combien de ses experts tournent par jeton : un modèle Mixture-of-Experts n'en utilise que quelques-uns à la fois, et c'est pour cela qu'il peut dépasser votre mémoire.",
765765
},
766+
"settings.ctxCapped": {
767+
en: "Running at %s tokens: this model was not trained for more.",
768+
fr: "En cours à %s jetons : ce modèle n'a pas été entraîné pour plus.",
769+
},
766770
"news.title": { en: "What changed in %s", fr: "Ce qui a changé dans la %s" },
767771
"news.sub": {
768772
en: "Since the version you were running.",

app/src/main.ts

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3952,7 +3952,7 @@ function settingsView(): HTMLElement {
39523952
<button data-sl="4">4</button>
39533953
</div>
39543954
</div>
3955-
<div class="set-row"><div class="grow"><b>${esc(t("settings.ctx"))}</b><span>${esc(t("settings.ctxHint"))}</span></div>
3955+
<div class="set-row"><div class="grow"><b>${esc(t("settings.ctx"))}</b><span>${esc(t("settings.ctxHint"))}</span><span class="d ctxnote"></span></div>
39563956
<div class="seg" id="ctxseg">
39573957
<button data-ctx="8192">8K</button>
39583958
<button data-ctx="16384">16K</button>
@@ -4113,7 +4113,23 @@ function settingsView(): HTMLElement {
41134113
const paintCtx = (v: string) =>
41144114
ctxseg.querySelectorAll("button").forEach((b) =>
41154115
(b as HTMLElement).classList.toggle("on", (b as HTMLElement).dataset.ctx === v));
4116-
api.settingsGet().then((s) => paintCtx(s["engine_ctx"] || "8192"));
4116+
api.settingsGet().then((s) => {
4117+
paintCtx(s["engine_ctx"] || "8192");
4118+
// What the engine is ACTUALLY serving, when it differs from what was
4119+
// asked for. Each model is capped by the window it was trained on (or
4120+
// by a cautious ceiling when it declares none), and the segment painted
4121+
// the stored value regardless: choosing 128K left the button on 128K
4122+
// over a server running 32K, with nothing saying so.
4123+
const live = server.ctx_per_slot ?? 0;
4124+
const want = Number(s["engine_ctx"] || "8192");
4125+
const note = ctxseg.parentElement?.querySelector<HTMLElement>(".ctxnote");
4126+
if (note) {
4127+
note.textContent =
4128+
server.running && live > 0 && live !== want
4129+
? t("settings.ctxCapped").replace("%s", String(live))
4130+
: "";
4131+
}
4132+
});
41174133
ctxseg.addEventListener("click", async (e) => {
41184134
const b = (e.target as HTMLElement).closest("[data-ctx]") as HTMLElement | null;
41194135
if (!b) return;

scripts/models-registry.json

Lines changed: 18 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,8 @@
108108
"regime": "cross-check path (CPU experts and biases)",
109109
"date": "2026-08-11",
110110
"recorded_by": "scripts/certify.py"
111-
}
111+
},
112+
"context_length": 131072
112113
},
113114
{
114115
"id": "glm-5.2-744b",
@@ -149,7 +150,8 @@
149150
"GLM-5.2-UD-IQ1_S-00005-of-00006.gguf",
150151
"GLM-5.2-UD-IQ1_S-00006-of-00006.gguf"
151152
]
152-
}
153+
},
154+
"context_length": 202752
153155
},
154156
{
155157
"id": "qwen3-30b-a3b",
@@ -232,7 +234,8 @@
232234
"writing",
233235
"scripting"
234236
],
235-
"min_ram_gb": 16
237+
"min_ram_gb": 16,
238+
"context_length": 40960
236239
},
237240
{
238241
"id": "llama4-scout",
@@ -398,7 +401,8 @@
398401
"max",
399402
"reasoning",
400403
"writing"
401-
]
404+
],
405+
"context_length": 40960
402406
},
403407
{
404408
"id": "glm-4.5-air",
@@ -491,7 +495,8 @@
491495
"tasks": [
492496
"writing",
493497
"reasoning"
494-
]
498+
],
499+
"context_length": 131072
495500
},
496501
{
497502
"id": "qwen3-next-80b",
@@ -591,7 +596,8 @@
591596
"tasks": [
592597
"general",
593598
"writing"
594-
]
599+
],
600+
"context_length": 262144
595601
},
596602
{
597603
"id": "qwen3-coder-30b",
@@ -676,7 +682,8 @@
676682
"regime": "cross-check path (CPU experts and biases)",
677683
"date": "2026-08-11",
678684
"recorded_by": "scripts/certify.py"
679-
}
685+
},
686+
"context_length": 262144
680687
},
681688
{
682689
"id": "olmoe-1b-7b",
@@ -751,7 +758,8 @@
751758
"prompt_spread_pct": 5.0
752759
}
753760
],
754-
"measured_note": "MacBook Pro M5 Max, 128 GB, internal SSD, shipped path (Metal bit-exact experts), batch 512, planner ubatch, 192 predicted tokens on a 24-paragraph prompt, bench 2026-08-10 by scripts/bench-curve.py; cache = app planning ceiling per Mac tier, curve stops at full residency (3.90 GB)"
761+
"measured_note": "MacBook Pro M5 Max, 128 GB, internal SSD, shipped path (Metal bit-exact experts), batch 512, planner ubatch, 192 predicted tokens on a 24-paragraph prompt, bench 2026-08-10 by scripts/bench-curve.py; cache = app planning ceiling per Mac tier, curve stops at full residency (3.90 GB)",
762+
"context_length": 4096
755763
},
756764
{
757765
"id": "phi35-moe",
@@ -830,7 +838,8 @@
830838
"prompt_spread_pct": 0.0
831839
}
832840
],
833-
"measured_note": "MacBook Pro M5 Max, 128 GB, internal SSD, shipped path (Metal bit-exact experts), batch 512, planner ubatch, 192 predicted tokens on a 24-paragraph prompt, bench 2026-08-10 by scripts/bench-curve.py; cache = app planning ceiling per Mac tier, curve stops at full residency (24.38 GB)"
841+
"measured_note": "MacBook Pro M5 Max, 128 GB, internal SSD, shipped path (Metal bit-exact experts), batch 512, planner ubatch, 192 predicted tokens on a 24-paragraph prompt, bench 2026-08-10 by scripts/bench-curve.py; cache = app planning ceiling per Mac tier, curve stops at full residency (24.38 GB)",
842+
"context_length": 131072
834843
},
835844
{
836845
"id": "mellum2-12b",

0 commit comments

Comments
 (0)