M18: observability & fallback telemetry - #35
Merged
Merged
Conversation
Makes silent `via_binary` round-trips and long-running memory drift
observable. The canonical motivator is Whisper-before-M8, which spent
90% of forward-pass time on a BinaryBackend round-trip with no signal.
- Emily.Telemetry: new module; moduledoc enumerates every event,
memory_stats/0 samples the MLX allocator, maybe_warn_fallback/2
handles the opt-in dev warning.
- Span events via :telemetry.span/3 at [:emily, :eval, *],
[:emily, :to_binary, *] (with shape/dtype/byte_size metadata), and
[:emily, :fallback, *] (with op/input_shapes/input_dtypes).
- One-shot Logger.warning per {op, input_shapes} pair, off by default.
Enable with `config :emily, :warn_on_fallback, true` — typically in
config/dev.exs when chasing perf regressions. The telemetry event
fires regardless.
- via_binary/3 → /4 + tuple variant take an op-name atom; all call
sites + apply_scatter threaded through.
- Dedup state lives in a :public :named_table ETS set owned by
Emily.Application, created before the supervisor starts.
"Each Native dispatch" from PLAN.md is implemented as the evaluation
boundary (eval / to_binary), not per-op graph-construction
instrumentation: MLX is lazy, graph-construction NIFs are <10µs and
do no work, and wrapping 300+ sites would add noise for no histogram
anyone wants. If per-op events are ever needed, the right answer is
a centralised dispatch helper, not per-callback decoration.
Tests cover fallback metadata, 100-call dedup, default-silent
behaviour, to_binary span metadata, and memory_stats/0 emission.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
:telemetryspans at the evaluation boundary ([:emily, :eval, *],[:emily, :to_binary, *]) and at everyNx.BinaryBackendfallback ([:emily, :fallback, *]).Emily.Telemetrymodule enumerates every event and exposesmemory_stats/0for allocator polling.Logger.warningper{op, input_shapes}when a fallback fires. Off by default — the telemetry event fires regardless, the log is a dev-time convenience. Enable withconfig :emily, :warn_on_fallback, true.Motivation
Whisper-before-M8 spent 90% of forward-pass time on a BinaryBackend round-trip with no signal. The same class of bug will keep happening as ops rotate on/off
via_binary; M18 makes it observable.Design notes
falsechosen after observing that the test suite intentionally exercises many fallback paths (backend_fallbacks_test.exs, window property tests, cumulative fallbacks). Default-on would mean CI logs and every dev's firstNx.to_binarycall spray warnings unsolicited. Default-off lets library consumers stay quiet and opt in when chasing a perf regression.via_binary/3→/4,via_binary_tuple/3→/4,apply_scatter/7→/8. All ~7 call sites updated.:publicETS table owned byEmily.Application, created before the supervisor starts. No GenServer on the hot path.Test plan
mix precommitgreen (1 doctest, 76 properties, 406 tests, 0 failures)test/emily/telemetry_test.exs: fallback event metadata, 100-call dedup assertion, default-silent behaviour,to_binaryspan metadata,memory_stats/0emissionFollow-ups (out of M18 scope)