From d37d70ce672a6b5e128a2d3bed01169c23cd1bb4 Mon Sep 17 00:00:00 2001 From: ausimian Date: Wed, 8 Jul 2026 12:03:53 +0100 Subject: [PATCH 1/3] docs: bump install requirement to ~> 1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit README and the livebooks pinned `{:emily, "~> 0.7"}`. A `~> 0.7` requirement won't resolve the upcoming 1.0.0 release, so move the install snippets to `~> 1.0`. Other deps (Bumblebee/Axon/Nx/…) are unchanged. --- README.md | 4 ++-- livebooks/distilbert_qa.livemd | 2 +- livebooks/fast_kernels.livemd | 2 +- livebooks/mnist_training.livemd | 2 +- livebooks/modernbert_classification.livemd | 2 +- livebooks/nomic_embeddings.livemd | 2 +- livebooks/qwen3_quantized.livemd | 2 +- livebooks/smollm3_chat.livemd | 2 +- livebooks/whisper_transcription.livemd | 2 +- 9 files changed, 10 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index f016288..84fb22c 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Add `:emily` to your `mix.exs` deps: ```elixir def deps do [ - {:emily, "~> 0.7"} + {:emily, "~> 1.0"} ] end ``` @@ -99,7 +99,7 @@ Releases on first `mix compile`. ### As a hex consumer -Add `{:emily, "~> 0.7"}` to `mix.exs`, then: +Add `{:emily, "~> 1.0"}` to `mix.exs`, then: ```sh mix deps.get diff --git a/livebooks/distilbert_qa.livemd b/livebooks/distilbert_qa.livemd index 19cabf7..62d77af 100644 --- a/livebooks/distilbert_qa.livemd +++ b/livebooks/distilbert_qa.livemd @@ -3,7 +3,7 @@ ```elixir Mix.install( [ - {:emily, "~> 0.7"}, + {:emily, "~> 1.0"}, {:bumblebee, "~> 0.7"}, {:tokenizers, "~> 0.5"}, {:nx, "~> 0.12"}, diff --git a/livebooks/fast_kernels.livemd b/livebooks/fast_kernels.livemd index e121b7f..16c4bb0 100644 --- a/livebooks/fast_kernels.livemd +++ b/livebooks/fast_kernels.livemd @@ -3,7 +3,7 @@ ```elixir Mix.install( [ - {:emily, "~> 0.7"}, + {:emily, "~> 1.0"}, {:bumblebee, "~> 0.7"}, {:tokenizers, "~> 0.5"}, {:axon, "~> 0.8"}, diff --git a/livebooks/mnist_training.livemd b/livebooks/mnist_training.livemd index 780db83..31b9df0 100644 --- a/livebooks/mnist_training.livemd +++ b/livebooks/mnist_training.livemd @@ -3,7 +3,7 @@ ```elixir Mix.install( [ - {:emily, "~> 0.7"}, + {:emily, "~> 1.0"}, {:axon, "~> 0.8"}, {:scidata, "~> 0.1"}, {:nx, "~> 0.12"}, diff --git a/livebooks/modernbert_classification.livemd b/livebooks/modernbert_classification.livemd index 99fe637..53b7bdb 100644 --- a/livebooks/modernbert_classification.livemd +++ b/livebooks/modernbert_classification.livemd @@ -3,7 +3,7 @@ ```elixir Mix.install( [ - {:emily, "~> 0.7"}, + {:emily, "~> 1.0"}, {:bumblebee, "~> 0.7"}, {:tokenizers, "~> 0.5"}, {:nx, "~> 0.12"}, diff --git a/livebooks/nomic_embeddings.livemd b/livebooks/nomic_embeddings.livemd index 4ca194a..434709d 100644 --- a/livebooks/nomic_embeddings.livemd +++ b/livebooks/nomic_embeddings.livemd @@ -3,7 +3,7 @@ ```elixir Mix.install( [ - {:emily, "~> 0.7"}, + {:emily, "~> 1.0"}, {:bumblebee, "~> 0.7"}, {:tokenizers, "~> 0.5"}, {:nx, "~> 0.12"}, diff --git a/livebooks/qwen3_quantized.livemd b/livebooks/qwen3_quantized.livemd index 128501a..04100fe 100644 --- a/livebooks/qwen3_quantized.livemd +++ b/livebooks/qwen3_quantized.livemd @@ -3,7 +3,7 @@ ```elixir Mix.install( [ - {:emily, "~> 0.7"}, + {:emily, "~> 1.0"}, {:bumblebee, "~> 0.7"}, {:tokenizers, "~> 0.5"}, {:nx, "~> 0.12"}, diff --git a/livebooks/smollm3_chat.livemd b/livebooks/smollm3_chat.livemd index ae595a1..a4e731b 100644 --- a/livebooks/smollm3_chat.livemd +++ b/livebooks/smollm3_chat.livemd @@ -3,7 +3,7 @@ ```elixir Mix.install( [ - {:emily, "~> 0.7"}, + {:emily, "~> 1.0"}, {:bumblebee, "~> 0.7"}, {:tokenizers, "~> 0.5"}, {:nx, "~> 0.12"}, diff --git a/livebooks/whisper_transcription.livemd b/livebooks/whisper_transcription.livemd index be2f14f..efb42d1 100644 --- a/livebooks/whisper_transcription.livemd +++ b/livebooks/whisper_transcription.livemd @@ -3,7 +3,7 @@ ```elixir Mix.install( [ - {:emily, "~> 0.7"}, + {:emily, "~> 1.0"}, {:bumblebee, "~> 0.7"}, {:tokenizers, "~> 0.5"}, {:nx, "~> 0.12"}, From 47266a46f60f4ceca16f165fedcc345eb0919d2c Mon Sep 17 00:00:00 2001 From: ausimian Date: Wed, 8 Jul 2026 12:03:53 +0100 Subject: [PATCH 2/3] docs: fix stale bumblebee/axon versions in FastKernels moduledoc The moduledoc told consumers to add `{:bumblebee, "~> 0.6"}` and `{:axon, "~> 0.7"}`, but mix.exs has required `~> 0.7` and `~> 0.8`. Align the example with the actual optional-dep requirements. --- lib/emily/bumblebee/fast_kernels.ex | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/emily/bumblebee/fast_kernels.ex b/lib/emily/bumblebee/fast_kernels.ex index cd0ab1e..729f5f2 100644 --- a/lib/emily/bumblebee/fast_kernels.ex +++ b/lib/emily/bumblebee/fast_kernels.ex @@ -24,8 +24,8 @@ if Code.ensure_loaded?(Axon) and Code.ensure_loaded?(Bumblebee.Layers) do entirely when either dep is missing. To use the shim, add both to your own `deps/0`: - {:bumblebee, "~> 0.6"}, - {:axon, "~> 0.7"} + {:bumblebee, "~> 0.7"}, + {:axon, "~> 0.8"} ## Usage From f7432b13e5877b423d043d81ac0214cff5c5b52e Mon Sep 17 00:00:00 2001 From: ausimian Date: Wed, 8 Jul 2026 12:12:50 +0100 Subject: [PATCH 3/3] docs: describe the fused quantized_matmul path, not dequantize The README quantization bullet and the qwen3_quantized livebook still described the quantized path as running through `dequantize_defn/1` (dequantize-then-dot). Since the fused quantized_matmul work (61b36f4) the native quantized path lowers to the fused `mx::quantized_matmul` kernel, streaming the packed low-bit weights; dequantize_defn is now only the non-MLX fallback. Update the prose to match. --- README.md | 8 +++++--- livebooks/qwen3_quantized.livemd | 6 +++--- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 84fb22c..23a7668 100644 --- a/README.md +++ b/README.md @@ -65,9 +65,11 @@ Emily backend smoke test. other backends. - **Affine group-wise quantization.** `Emily.QuantizedWeight` + `Emily.Quantization` wrap MLX `quantize` / `dequantize` / - `quantized_matmul` for int2 / int4 / int8 inference. Includes a - defn-native `dequantize_defn/1` for quantized layers inside Axon - forward passes. + `quantized_matmul` for int2 / int4 / int8 inference. Quantized dense + layers lower to the fused `quantized_matmul` kernel — streaming the + packed low-bit weights instead of dequantizing the whole weight per + token — with a `dequantize_defn/1` + `Nx.dot` fallback on non-MLX + backends. - **Mixed-precision training.** `Emily.MixedPrecision` provides the bf16 recipe (cast params for the forward, keep f32 master, dynamic loss scaling with overflow detection). diff --git a/livebooks/qwen3_quantized.livemd b/livebooks/qwen3_quantized.livemd index 04100fe..7fba122 100644 --- a/livebooks/qwen3_quantized.livemd +++ b/livebooks/qwen3_quantized.livemd @@ -57,7 +57,7 @@ dense_text `native: true, native_fallback: :raise` on `:defn_options` lowers the per-token decode through Emily's native Expr compiler and asserts it stays fully native — the quantized serving below passes the same -options, so the int4 dequantize path lowers native too. +options, so the int4 quantized matmul lowers native too. ## Quantization transform @@ -168,8 +168,8 @@ end > recursive rewriter. The `transpose: true` default stores weights as > `[out, in]` (MLX / AWQ convention, groups along the reduction axis); > set `false` if you're feeding a checkpoint that's already laid out -> the other way. Bits must be one of `[2, 3, 4, 6, 8]` for the -> defn-native `dequantize_defn/1` path. +> the other way. Bits must be one of `[2, 3, 4, 6, 8]` for MLX affine +> quantization. ## Quantized inference