feat: make native compilation the Emily.Compiler default

`Emily.Compiler`'s `:native` option now defaults to `true` (read from
`config :emily, :native`, itself defaulting to `true`), so a bare
`compiler: Emily.Compiler` compiles to the single-NIF native lane —
the biggest perf win, bit-identical to the evaluator, and safe because
un-lowerable ops still route through the evaluator under
`native_fallback: :eval`. Opt out with `native: false` (per call) or
`config :emily, native: false` (app-wide) — e.g. on a
memory-constrained host where the one-shot compile peak is too large.

The suite uses a bare `compiler: Emily.Compiler` as its eval oracle,
so `config/test.exs` pins `native: false`; tests that exercise the
native lane pass `native: true` explicitly. The native-config test
gains cases for the default-native and `config: false` paths.

Docs updated to match: the Emily.Compiler moduledoc, README "Native
compilation" / "Choosing a mode", ARCHITECTURE, and RELEASE.md.

Author: ausimian

ARCHITECTURE.md

@@ -9,10 +9,11 @@ without sifting milestone history. For per-milestone rationale see
 ```
 Emily.Compiler    (Nx.Defn.Compiler) — validates opts, pins the result backend,
                                         selects one of two lowering lanes:
-                                          • default (native: false): op-by-op
-                                            via Nx.Defn.Evaluator + Emily.Backend
-                                          • native (native: true): lower once to
-                                            Emily.IR/Program, single-NIF replay
+                                          • native (default, native: true):
+                                            lower once to Emily.IR/Program,
+                                            single-NIF replay
+                                          • eval (native: false): op-by-op via
+                                            Nx.Defn.Evaluator + Emily.Backend
 Emily.IR + Program (native lane)     — flat IR compiled once; whole-graph replay,
                                         optionally mx::compile-fused (:fuse)
 Emily.Backend     (Nx.Backend)       — op-by-op translation to Native
@@ -33,24 +34,25 @@ see [Testing philosophy](#testing-philosophy).
 ## Core design decisions
 
 1. **Backend-first; compiler layered on top.** The `Nx.Backend` is
-   enough to run Bumblebee; the `Nx.Defn` compiler is additive. By
-   default `Emily.Compiler` delegates the expression walk to
-   `Nx.Defn.Evaluator` and adds two adjustments: pin the result
-   backend to `Emily.Backend` via `__to_backend__/1`, and cap
-   `:max_concurrency` at 1. `native: true` switches to the single-NIF
-   lane (decision 10), and `:fuse` wraps `mx::compile` on top of it.
+   enough to run Bumblebee; the `Nx.Defn` compiler is additive. In
+   both lanes `Emily.Compiler` pins the result backend to
+   `Emily.Backend` via `__to_backend__/1` and caps `:max_concurrency`
+   at 1. By default (`native: true`) it lowers to the single-NIF lane
+   (decision 10); `native: false` delegates the expression walk to
+   `Nx.Defn.Evaluator` op-by-op instead, and `:fuse` wraps
+   `mx::compile` on top of the native lane.
    The original PLAN M6 measurement found whole-graph `mx::compile`
    below the 1.20× gate; the single-NIF replay changed that economics
    (fuse the elementwise runs the replay leaves separate, and cache a
    fused `defn while` body per stream), so fusion now ships as the
    opt-in `:fuse` mode.
 
 2. **Trace in Elixir, not in C++.** `Nx.Defn.Expr` is already a
-   fully traced tree. The default lane walks it from Elixir and emits
-   one Native NIF call per node (`lib/emily/native.ex`); the native
-   lane lowers the same tree to `Emily.IR` and emits a single NIF call
-   for the whole graph. Either way the trace is consumed in Elixir,
-   never re-traced in C++.
+   fully traced tree. The default native lane lowers it to `Emily.IR`
+   and emits a single NIF call for the whole graph; the op-by-op lane
+   (`native: false`) walks it from Elixir and emits one Native NIF
+   call per node (`lib/emily/native.ex`). Either way the trace is
+   consumed in Elixir, never re-traced in C++.
 
 3. **One resource type: `Tensor`** wrapping `mlx::array`. MLX's
    refcount does the heavy lifting; `fine`'s `ResourcePtr` adds one
@@ -112,12 +114,12 @@ see [Testing philosophy](#testing-philosophy).
     the NIF boundary once, captured by the compiled program. Anything
     the IR can't lower routes the *whole* defn back through
     `Nx.Defn.Evaluator` under the default `native_fallback: :eval`
-    (firing `[:emily, :compiler, :fallback]`), so a global
-    `native: true` install is safe on any model. Use
-    `native_fallback: :raise` to fail instead — the conformance gates
-    use it to prove full native lowering. The default comes from
-    `config :emily, :native` (itself `false`); a per-call `native:`
-    wins. `:fuse` wraps the replay in `mx::compile`, cached per stream
+    (firing `[:emily, :compiler, :fallback]`), so native is safe as
+    the default on any model. Use `native_fallback: :raise` to fail
+    instead — the conformance gates use it to prove full native
+    lowering. The default comes from `config :emily, :native` (itself
+    `true`); set `config :emily, native: false` (or pass
+    `native: false`) to opt a memory-constrained host back to op-by-op. `:fuse` wraps the replay in `mx::compile`, cached per stream
     and fusing `defn while` bodies for decode loops — not
     bit-identical, hence opt-in.
 

README.md

@@ -204,24 +204,26 @@ and direct MLX round-trips, but most users should go through Nx.
 
 ## Native compilation
 
-`Emily.Compiler` has three modes. Opt into a mode per-call, or globally
-via `Nx.Defn.global_default_options/1`; the `native` lane can also be
-switched on application-wide with `config :emily, native: true` (a
-per-call `native:` option always overrides the app-env default):
+`Emily.Compiler` has three modes. `native` is **on by default**; opt
+into a different mode per-call, globally via
+`Nx.Defn.global_default_options/1`, or app-wide with
+`config :emily, native: false | true` (a per-call `native:` option
+always overrides the app-env default):
 
 ```elixir
-# Default: op-by-op dispatch through Emily.Backend. Bit-identical to
-# the Nx evaluator.
+# Default: native single-NIF replay. Lowers the traced Nx.Defn.Expr to
+# a flat IR and replays the whole forward graph in one NIF call per
+# invocation — ~5× decode throughput on Qwen3-0.6B, bit-identical to
+# the evaluator. Anything the IR can't lower routes through
+# Nx.Defn.Evaluator under the default `native_fallback: :eval` (with a
+# `[:emily, :compiler, :fallback]` telemetry event).
 Nx.Defn.jit(&forward/1, compiler: Emily.Compiler).(input)
 
-# Native single-NIF replay. Lowers the traced Nx.Defn.Expr to a flat
-# IR and replays the whole forward graph in one NIF call per
-# invocation — ~5× decode throughput on Qwen3-0.6B, bit-identical to
-# the evaluator. Safe to install globally: anything the IR can't lower
-# routes through Nx.Defn.Evaluator under the default
-# `native_fallback: :eval` (with a `[:emily, :compiler, :fallback]`
-# telemetry event).
-Nx.Defn.jit(&forward/1, compiler: Emily.Compiler, native: true).(input)
+# Op-by-op dispatch through Emily.Backend — opt out of native with
+# `native: false`. Bit-identical to the Nx evaluator, with no
+# whole-graph compile step, so no one-shot compile-time memory spike;
+# use it as a numerics reference or on memory-constrained hosts.
+Nx.Defn.jit(&forward/1, compiler: Emily.Compiler, native: false).(input)
 
 # Native + mx::compile kernel fusion. Fuses elementwise runs the plain
 # replay leaves as separate kernels (RMSNorm / softmax / SiLU gating /
@@ -239,11 +241,14 @@ ids matched the evaluator's exactly in our benchmarks), but
 **sampling strategies will diverge** from the evaluator even with a
 fixed seed.
 
-**Choosing a mode.** `native: true` is the right default for model
-inference: it's bit-identical to the evaluator, safe to install
-globally (un-lowerable ops route through the evaluator), and is the
-single biggest win — eager → native is the largest jump in every
-[benchmark](#performance) tier. `fuse: true` is **not** a universal
+**Choosing a mode.** `native` is the default and the right one for
+model inference: it's bit-identical to the evaluator, safe globally
+(un-lowerable ops route through the evaluator), and the single biggest
+win — eager → native is the largest jump in every
+[benchmark](#performance) tier. Opt out with `native: false` only when
+you need the op-by-op evaluator — a numerics reference, or a
+memory-constrained host where the one-shot compile peak is too large.
+`fuse: true` is **not** a universal
 add-on. It pays off only when the fused callable is *reused*, which in
 practice means autoregressive decode: the `defn while` body is
 `mx::compile`d once and cache-hits every step (the best lane on

RELEASE.md

@@ -1,14 +1,15 @@
 ### Added
 
-- **Native Expr compiler — `Nx.Defn.jit` / `compile` with
-  `compiler: Emily.Compiler, native: true`.** Lowers a traced
-  `Nx.Defn.Expr` to a flat IR once and replays the whole forward graph
-  in a **single NIF call per invocation**, collapsing the per-op
-  BEAM↔worker round-trips a step-evaluated decode loop would otherwise
-  pay. Weights cross the NIF boundary once (captured by the compiled
-  program) and are never re-serialised per call. Opt in per call:
+- **Native Expr compiler — on by default under
+  `compiler: Emily.Compiler`.** Lowers a traced `Nx.Defn.Expr` to a
+  flat IR once and replays the whole forward graph in a **single NIF
+  call per invocation**, collapsing the per-op BEAM↔worker round-trips
+  a step-evaluated decode loop would otherwise pay. Weights cross the
+  NIF boundary once (captured by the compiled program) and are never
+  re-serialised per call. It is the default, so a bare
+  `compiler: Emily.Compiler` compiles native:
 
-      Nx.Defn.jit(&forward/1, compiler: Emily.Compiler, native: true).(input)
+      Nx.Defn.jit(&forward/1, compiler: Emily.Compiler).(input)
 
   Coverage is the full Nx primitive set (with `Emily.Backend`'s
   dtype-coercion and op-composition semantics ported into the
@@ -20,15 +21,13 @@
   `defn while` (with the host loop driven entirely from the worker
   thread). Anything the IR can't lower yet routes through
   `Nx.Defn.Evaluator` under the default `native_fallback: :eval` (with
-  a one-shot `[:emily, :compiler, :fallback]` telemetry event), so
-  installing the compiler globally is safe on any model:
-
-      Nx.Defn.global_default_options(compiler: Emily.Compiler, native: true)
-
-  The `native` default is also read from `config :emily, :native`
-  (defaulting to `false`), so `config :emily, native: true` opts every
-  defn into the native lane application-wide without a per-call option;
-  a per-call `native:` option always wins over the app-env default.
+  a one-shot `[:emily, :compiler, :fallback]` telemetry event), so the
+  native lane is safe as the default on any model. The default is read
+  from `config :emily, :native` (defaulting to `true`), so
+  `config :emily, native: false` opts every defn out of the native lane
+  application-wide — e.g. on a memory-constrained host where the
+  one-shot compile peak is too large; a per-call `native:` option
+  always wins over the app-env default.
 
   `native_fallback: :raise` fails instead — the conformance suites use
   this to prove a model lowers fully native.

config/test.exs

@@ -17,3 +17,10 @@ config :emily,
 # fallback); tests that exercise the fallback pass `native_fallback: :eval`
 # per call.
 config :emily, native_fallback: :raise
+
+# The runtime default for `:native` is `true`; a bare
+# `compiler: Emily.Compiler` compiles native. The suite pins it to
+# `false` so a bare `compiler: Emily.Compiler` stays the *eval* oracle
+# the equivalence tests compare the native lane against — tests that
+# exercise the native path pass `native: true` explicitly.
+config :emily, native: false

lib/emily/compiler.ex

@@ -68,14 +68,16 @@ defmodule Emily.Compiler do
       and Bumblebee passes `:cache` through for its own per-scope
       cache suffixing. Neither is used by the Evaluator walk, but
       rejecting them would break those servings.
-    * `:native` — `true` compiles the traced `Nx.Defn.Expr` to a flat
-      IR and replays the whole graph in a single NIF call per invocation.
-      Defaults to `false`, which runs the op-by-op Evaluator walk. The
-      default is read from `config :emily, :native` (itself defaulting to
-      `false`), so `config :emily, native: true` opts every defn into the
-      native lane application-wide without a per-call option. The per-call
-      option wins over the app env, so an explicit `native: false`
-      overrides a global `config :emily, native: true`. A non-boolean
+    * `:native` — `true` (the default) compiles the traced
+      `Nx.Defn.Expr` to a flat IR and replays the whole graph in a
+      single NIF call per invocation; `false` runs the op-by-op
+      Evaluator walk instead. The default is read from
+      `config :emily, :native` (itself defaulting to `true`), so
+      `config :emily, native: false` opts every defn out of the native
+      lane application-wide — e.g. on a memory-constrained host where
+      the one-shot compile peak is too large. The per-call option wins
+      over the app env, so an explicit `native: false` overrides a
+      global `config :emily, native: true` and vice versa. A non-boolean
       raises `ArgumentError`.
     * `:native_fallback` — `:eval` (default) or `:raise`. Controls what
       happens when `native: true` but the expression contains an op or
@@ -174,15 +176,15 @@ defmodule Emily.Compiler do
   end
 
   # Per-call `:native` opt wins over `config :emily, :native`, defaulting
-  # to `false` (the op-by-op Evaluator walk). `Keyword.fetch/2` (not
+  # to `true` (the native single-NIF lane). `Keyword.fetch/2` (not
   # `Keyword.get/3`) so an explicit per-call `native: false` overrides a
-  # global `config :emily, native: true`, rather than being treated as
-  # "unset" and falling through to the app env.
+  # global default, rather than being treated as "unset" and falling
+  # through to the app env.
   defp native?(opts) do
     enabled =
       case Keyword.fetch(opts, :native) do
         {:ok, native} -> native
-        :error -> Application.get_env(:emily, :native, false)
+        :error -> Application.get_env(:emily, :native, true)
       end
 
     case enabled do

test/emily/compiler_native_config_test.exs

@@ -1,13 +1,14 @@
 defmodule Emily.CompilerNativeConfigTest do
   @moduledoc """
   `config :emily, :native` sets the default for the compiler's `:native`
-  option when a call passes none. A per-call `native:` always wins over the
-  app env. Mutates the application env, so `async: false`.
+  option when a call passes none; the shipped default (no config) is
+  `true`. A per-call `native:` always wins over the app env. Mutates the
+  application env, so `async: false`.
 
   Probe: `Nx.reduce/3` with a BEAM reducer can never lower to the single-NIF
   replay (the reducer would have to run on the host mid-graph). Under
   `native_fallback: :raise` it therefore raises *iff* native is enabled —
-  a clean signal for how `:native` resolved, without needing to inspect the
+  a clean signal for how `:native` resolved, without inspecting the
   compiled program.
   """
   use ExUnit.Case, async: false
@@ -29,7 +30,17 @@ defmodule Emily.CompilerNativeConfigTest do
     :ok
   end
 
-  test "config :emily, native: true enables native with no per-call option" do
+  test "with no config the default is native (no per-call option)" do
+    Application.delete_env(:emily, :native)
+
+    assert_raise ArgumentError, ~r/reduce/, fn ->
+      Nx.Defn.jit(unsupported_fun(), compiler: Emily.Compiler, native_fallback: :raise).(
+        t([1.0, 2.0])
+      )
+    end
+  end
+
+  test "config :emily, native: true enables native (no per-call option)" do
     Application.put_env(:emily, :native, true)
 
     assert_raise ArgumentError, ~r/reduce/, fn ->
@@ -39,8 +50,8 @@ defmodule Emily.CompilerNativeConfigTest do
     end
   end
 
-  test "defaults to false — an un-lowerable op routes through the evaluator" do
-    Application.delete_env(:emily, :native)
+  test "config :emily, native: false routes through the evaluator" do
+    Application.put_env(:emily, :native, false)
 
     out =
       Nx.Defn.jit(unsupported_fun(), compiler: Emily.Compiler, native_fallback: :raise).(