Skip to content

Commit a023a15

Browse files
committed
feat(diffusion): native FLUX.2-dev text-to-image + /v1/images serving + GUI
Add the `lumen-diffusion` crate: an MLX-native FLUX.2 text-to-image pipeline, fully separate from the autoregressive LLM stack. Components (each parity- validated against mflux / Swift references): - VAE decoder, Mistral-Small-3.2 text encoder (layers 10/20/30 -> [1,512,15360]), 32B rectified-flow DiT (8 double + 48 single stream, 4D RoPE, AdaLN, guidance), flow-match Euler scheduler, dev pipeline, and the FLUX.2 prompt tokenizer. - Both 4-bit (36 GB) and non-quantized bf16 (128 GB) weights are auto-detected per tensor (.scales presence), so the same loaders serve either footprint. Serving (lumen-server): - POST /v1/images/generations (OpenAI-compatible, base64 PNG). - LUMEN_SERVE = chat | image | hybrid serve modes; image mode leaves the MLX memory governors at their defaults (the ~31 GB diffusion working set exceeds the LLM-tuned wired cap, which would OOM the load). Desktop app (lumen-app): - IMAGE-MODELS catalog card with a diagonal "IMAGE" corner badge, an image- generation panel (prompt / size / steps / seed / guidance), and image-mode launch wiring (+ en/ko strings). Housekeeping: - Resolve all FLUX.2 component repos from the local HuggingFace cache via `hf_cache` (no machine-specific hardcoded paths). - Fix safe pre-existing warnings (unused imports / mut, minijinja serde feature, toml dep semver metadata). chore(release): v0.11.0
1 parent ca5dbc8 commit a023a15

47 files changed

Lines changed: 5819 additions & 72 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

Cargo.lock

Lines changed: 51 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ members = [
44
"crates/turboquant-cache",
55
"crates/lumen-metal",
66
"crates/lumen-mlx",
7+
"crates/lumen-diffusion",
78
"crates/lumen-model",
89
"crates/lumen-server",
910
"crates/paged-attention",
@@ -18,6 +19,7 @@ default-members = [
1819
"crates/turboquant-cache",
1920
"crates/lumen-metal",
2021
"crates/lumen-mlx",
22+
"crates/lumen-diffusion",
2123
"crates/lumen-model",
2224
"crates/lumen-server",
2325
"crates/paged-attention",

crates/lumen-app/Cargo.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "lumen-app"
3-
version = "0.10.1"
3+
version = "0.11.0"
44
edition.workspace = true
55
rust-version.workspace = true
66
license.workspace = true
@@ -36,7 +36,7 @@ reqwest = { version = "0.13.3", default-features = false, features = [
3636
# Config persistence (TOML at ~/Library/Application Support/Lumen/config.toml)
3737
serde = { workspace = true }
3838
serde_json = { workspace = true }
39-
toml = "1.1.2+spec-1.1.0"
39+
toml = "1.1.2"
4040

4141
# Model registry — local scan only. Downloads use raw HTTPS via reqwest.
4242
walkdir = "2.5.0"

crates/lumen-app/frontend/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "lumen-app-frontend",
33
"private": true,
4-
"version": "0.10.1",
4+
"version": "0.11.0",
55
"type": "module",
66
"scripts": {
77
"predev": "cargo build --manifest-path ../../../Cargo.toml -p lumen-server --release",

crates/lumen-app/frontend/src/App.svelte

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
import DoctorPanel from "./lib/DoctorPanel.svelte";
2626
import UpdatePanel from "./lib/UpdatePanel.svelte";
2727
import ApiTabs from "./lib/ApiTabs.svelte";
28+
import ImagePanel from "./lib/ImagePanel.svelte";
2829
import { bytes, duration } from "./lib/format";
2930
import { i18n, t, LOCALE_LABELS, type Locale } from "./lib/i18n.svelte";
3031
@@ -223,6 +224,24 @@
223224
return models.find((m) => m.id === config!.active_model) ?? null;
224225
});
225226
227+
// Text-to-image catalog entries. `image_models` is `#[serde(default)]` on the
228+
// Rust side so it may be absent against an older server binary — default [].
229+
let imageModels = $derived(catalog.image_models ?? []);
230+
231+
// The active model id, if it is a diffusion image model. Drives both the
232+
// image-models card's ✓ state and the generation panel's visibility.
233+
let activeImageModelId = $derived.by(() => {
234+
const id = config?.active_model;
235+
if (!id) return null;
236+
return imageModels.some((m) => m.id === id) ? id : null;
237+
});
238+
239+
// Show the generation panel only when an image model is active AND the
240+
// server is up (image mode serves POST /v1/images/generations).
241+
let imagePanelVisible = $derived(
242+
activeImageModelId != null && status.state === "running",
243+
);
244+
226245
// Memory the model can realistically claim, reclaim-aware. macOS evicts /
227246
// swap-compresses inactive + compressor + other apps' active pages when a big
228247
// MLX allocation arrives, so the only genuinely-unavailable memory is WIRED
@@ -1465,6 +1484,49 @@
14651484
{/if}
14661485
</section>
14671486

1487+
<!-- IMAGE MODELS (text-to-image / diffusion) -->
1488+
{#if imageModels.length > 0}
1489+
<section class="{cardBase} col-span-3">
1490+
<h2 class={cardH2}>{t("imageModels.title")} <span class="dim">{t("imageModels.titleHint")}</span></h2>
1491+
<div class="flex flex-col gap-2">
1492+
{#each imageModels as im}
1493+
{@const isActive = config?.active_model === im.id}
1494+
{@const fits = !systemInfo || im.min_ram_gb <= systemInfo.ram_gb}
1495+
<div
1496+
class={`image-card relative overflow-hidden flex flex-col gap-1.5 px-3 py-2.5 rounded-md bg-panel-2 border ${
1497+
isActive
1498+
? "border-accent shadow-[0_0_0_1px_var(--color-accent)_inset] bg-accent/6"
1499+
: "border-transparent hover:border-border"
1500+
}`}
1501+
>
1502+
<!-- Diagonal corner ribbon marking this as an image / diffusion model -->
1503+
<span class="image-ribbon" aria-hidden="true">{t("imageModels.badge")}</span>
1504+
<div class="flex items-center gap-2 pl-5">
1505+
<span class="mono text-ok font-bold w-4">{isActive ? "" : ""}</span>
1506+
<div class="min-w-0 flex-1">
1507+
<div class="text-[13px] text-text">{im.label}</div>
1508+
<div class="dim mono text-[11px]">{im.id} · {im.approx_size_gb}GB · ≥{im.min_ram_gb}GB RAM</div>
1509+
</div>
1510+
<button
1511+
class="primary"
1512+
onclick={() => setActive(im.id)}
1513+
disabled={isActive}
1514+
title={fits ? "" : t("imageModels.action.title.overBudget")}
1515+
>{isActive ? t("imageModels.active") : t("action.use")}</button>
1516+
</div>
1517+
{#if !fits}
1518+
<div class="text-warn text-[11px] leading-normal">⚠ {t("imageModels.overBudget")}</div>
1519+
{/if}
1520+
<div class="dim text-[11px] leading-normal">{im.notes}</div>
1521+
</div>
1522+
{/each}
1523+
</div>
1524+
{#if activeImageModelId != null && status.state !== "running"}
1525+
<div class="dim mt-2.5 text-[11px] leading-normal">{t("imageModels.startHint")}</div>
1526+
{/if}
1527+
</section>
1528+
{/if}
1529+
14681530
<!-- SERVER -->
14691531
<section class="{cardBase} col-span-3">
14701532
<h2 class={cardH2}>{t("server.title")}</h2>
@@ -1591,6 +1653,12 @@
15911653
{/if}
15921654
</section>
15931655

1656+
<!-- IMAGE GENERATION PANEL — only when a diffusion model is active and the
1657+
server is running in image mode (serves POST /v1/images/generations). -->
1658+
{#if imagePanelVisible && activeImageModelId}
1659+
<ImagePanel modelId={activeImageModelId} />
1660+
{/if}
1661+
15941662
{/if}
15951663

15961664
{#if activeTab === "debug"}
@@ -1959,4 +2027,41 @@
19592027
details.mem-explainer[open] > summary::before {
19602028
content: "";
19612029
}
2030+
2031+
/* Diagonal "IMAGE" corner ribbon for text-to-image / diffusion model cards.
2032+
Classic technique: a wide banner pinned beyond the top-left corner and
2033+
rotated -45° so it crosses the corner at a diagonal. The parent card sets
2034+
`position: relative; overflow: hidden;` (Tailwind `relative overflow-hidden`)
2035+
so the banner is clipped to a neat triangle. Colors come from the app's
2036+
accent token — no hardcoded brand values. */
2037+
.image-ribbon {
2038+
position: absolute;
2039+
top: 9px;
2040+
left: -30px;
2041+
width: 96px;
2042+
transform: rotate(-45deg);
2043+
transform-origin: center;
2044+
text-align: center;
2045+
background: var(--accent);
2046+
color: var(--bg);
2047+
font-size: 8px;
2048+
font-weight: 700;
2049+
letter-spacing: 0.12em;
2050+
line-height: 14px;
2051+
text-transform: uppercase;
2052+
box-shadow: 0 1px 3px rgba(0, 0, 0, 0.45);
2053+
pointer-events: none;
2054+
user-select: none;
2055+
z-index: 1;
2056+
}
2057+
2058+
/* When this image model is the selected / active one, brighten the ribbon
2059+
so the active image model is unmistakably distinguished. */
2060+
.image-card.border-accent .image-ribbon {
2061+
background: var(--accent);
2062+
color: var(--bg);
2063+
box-shadow:
2064+
0 0 0 1px var(--accent),
2065+
0 1px 4px rgba(0, 0, 0, 0.55);
2066+
}
19622067
</style>

0 commit comments

Comments
 (0)