Lower Nx.top_k (Nx.Block.TopK) in the native Expr compiler - #187
Merged
Conversation
Nx.top_k now compiles under native instead of raising "does not yet
lower the block Nx.Block.TopK". It's a multi-output block {values,
indices}: Emily.Backend has no top_k override (mx::topk yields values
only, not the indices Nx's contract requires), so the evaluator computes
it via the block's default expansion (argsort desc -> take_along_axis ->
slice the top k).
The compiler lowers that same expansion and projects the two leaves via
:elem (the {:multi_refs, ...} machinery the tuple-cond path added).
Crucially, Nx.Defn.Expr.expr_block builds the expansion against FRESH
per-position :parameter nodes, not the real in_args, so the parameters
must be BOUND to the lowered in_args (the same binding `while` does via
param_seed). Without that, a block parameter falls through to the OUTER
function's {:input, pos} — a different tensor — which is exactly how
modernbert_classification hit "argsort axis 1 on a 1-D array" (its outer
input 0 is the 1-D input_ids). Result is bit-identical to the evaluator,
which runs the same bound expansion.
- ir.ex: lower_block clause for Nx.Block.TopK — lower the in_args, seed
the cache so each block :parameter resolves to its arg, lower the
expansion leaves, return {:multi_refs, [vref, iref]}.
collect_block_params/2 walks the (small) expansion to find them.
- compiler_equivalence_test.exs: values + s32 indices match; a downstream
op consuming both leaves; and a regression case where outer input 0 is
1-D, so an unbound parameter would crash (the ModernBERT shape).
Closes the last gap from the forced-native livebook probe: all 8 example
livebooks now lower fully native (modernbert_classification was the
holdout, on top_k).
ausimian
force-pushed
the
feat/expr-compiler-topk
branch
from
June 6, 2026 07:47
a3e1217 to
d95fff5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Nx.top_know compiles under native instead of raising "does not yet lowerthe block
Nx.Block.TopK". With this, all 8 example livebooks lower fullynative —
modernbert_classificationwas the lone holdout (found by theforced-native livebook probe).
What it is
Nx.Block.TopKis a multi-output block ({values, indices}), andEmily.Backendhas notop_koverride (mx::topkyields values only, notthe indices Nx's contract requires). So the evaluator computes it via the
block's default expansion:
argsort(desc) → take_along_axis → slice the top k. Every op there already lowers, so the compiler lowers that sameexpansion and projects the two leaves via
:elem(the{:multi_refs, …}machinery the tuple-
condfix added) — bit-identical to the evaluator.The subtle part — binding the block's parameters
Nx.Defn.Expr.expr_blockbuilds a block's expansion against freshper-position
:parameternodes, not the realin_args. So lowering theexpansion directly makes a block
parameter 0fall through to the outerfunction's
{:input, 0}— a completely different tensor. That's exactly howModernBERT failed: its outer input 0 is the 1-D
input_ids, so top_k'sargsortran on a 1-D array → "invalid axis 1 for array with 1 dimensions"at replay, even though
logits/softmaxwere correct{1, num_labels}.The fix binds the block parameters to the lowered
in_args(the same thingwhiledoes viaparam_seed) before lowering the expansion. Eval never trippedon this because it evaluates the bound expansion directly on real tensors.
Changes
ir.ex—lower_blockforNx.Block.TopK: lower thein_args, seed thecache so each block
:parameterresolves to its arg, lower the expansionleaves, return
{:multi_refs, [vref, iref]}.collect_block_params/2walksthe (small) expansion to find the parameters.
compiler_equivalence_test.exs— values +s32indices match; adownstream op consuming both leaves; and a regression case where outer
input 0 is 1-D (the ModernBERT shape), which an unbound parameter would crash.
Verification
mix precommitgreen: 694 tests, 0 failures.modernbert_classificationlivebook, forced tonative: true, native_fallback: :raise, now runs end-to-end native (was the 1 failing of8). The other 7 were already native.