feat(email): on-device E2B (NPU/FLM) model integration (#1282)

[itomek]

Closes #1282

Users on Ryzen AI NPU hardware couldn't pick the lighter, faster on-device model for email triage — only the larger E4B was in the catalog. Now gemma4-it-e2b-FLM (the NPU-native FastFlowLM build) is a first-class catalog model, selectable via EmailAgentConfig(model_id="gemma4-it-e2b-FLM"). Validated on real Strix Halo NPU hardware: device=npu, recipe=flm, served at :13305, ~23 tok/s decode / ~1s TTFT. The model downloads lazily on first use, so gaia init --profile all never pulls it (it is pulled by --profile npu, by design).

Two hardware findings shaped the entry: the FLM build serves a 4096-token window (not 32K), and it 500-errors on a native OpenAI tools payload — so the entry declares tool_calling=False and the agent uses the embedded-JSON tool path. Email triage itself parses a JSON object from a plain completion (no native tool calls), so it's unaffected.

Test plan

• python -m pytest tests/unit/agents/test_email_agent_local_llm_enforcement.py -q — catalog entry (FLM id, ctx 4096, tool_calling=False), local-only (AC3) enforcement, and a subprocess-isolated no-import-time-download guard
• On a Strix Halo NPU box: load gemma4-it-e2b-FLM via Lemonade; confirm device=npu / recipe=flm in /api/v1/health; run LEMONADE_MODEL=gemma4-it-e2b-FLM python -m pytest tests/integration/test_email_bench_throughput.py

[github-actions]

Review: feat(email) — on-device E2B (NPU/FLM) model integration (#1282)

Approve with suggestions. This is a tight, additive change: one new MODELS catalog entry plus a thorough test class, no production code paths altered. The entry is internally consistent with the rest of the codebase (init_command.py:101, registry.py:277, and the NPU docs all use the same gemma4-it-e2b-FLM id and ctx_size=4096), and I confirmed the headline safety claim: gemma-4-e2b is not referenced by any AGENT_PROFILE, so get_required_models("all") never resolves it — gaia init --profile all will not pull the weights. The one thing worth confirming before merge is whether tool_calling=True actually holds for the FLM build (see below), since email triage depends on it.

Summary

The catalog entry is correct, the lazy-download guard test is genuinely well-constructed (patches requests.adapters.HTTPAdapter.send + subprocess at the real chokepoints before re-import — and lemonade_client.py does route all HTTP through requests, so the patch target is right), and the AC3 cloud-base_url rejection test reflects real EmailAgentConfig.validate() behavior. No security concerns, no silent fallbacks, no breaking changes.

Issues

🟡 tool_calling=True is asserted for an FLM build, but the field's own contract scopes it to GGUF (src/gaia/llm/lemonade_client.py:226)

ModelRequirement.tool_calling's default comment reads "True for GGUF models via Lemonade --jinja (Tier 0 empirical)" — but this entry is the FastFlowLM/NPU build, a different inference engine. The test test_e2b_tool_calling_enabled only asserts the literal True we set; it doesn't prove FLM-on-NPU actually serves native tool calls. The docstring claims the email agent "relies on native tool calls for triage, organize, and reply" — if FLM doesn't support --jinja-style tool calling, NPU email triage breaks at runtime for exactly the users this PR targets. The PR says the model was hardware-validated (device=npu, ~24 tok/s), but doesn't state that tool calling specifically was exercised there. Please confirm tool calls round-trip on the FLM build (e.g. the unchecked test_email_bench_throughput.py run, or a single triage call on the Strix Halo box) before merge. If it's verified, a one-line note on the entry distinguishing it from the GGUF --jinja assumption would save the next reader the same question.

🟢 Several tests mirror the catalog literals rather than behavior (tests/unit/agents/test_email_agent_local_llm_enforcement.py:66-111)

test_e2b_model_id_is_flm_npu_build, test_e2b_is_llm_type, test_e2b_tool_calling_enabled, and test_e2b_display_name_set each re-assert a value set three lines apart in the source — they're change-detectors that fail only if someone edits the entry, not tests of behavior. They're cheap and the failure messages are good, so this isn't worth removing; just flagging that the real coverage value lives in the two EmailAgentConfig integration tests and the lazy-download guard. No action needed.

🟢 PR description slightly overstates "never enters the install path" (description, not code)

The "not pulled by gaia init --profile all" claim is correct and I verified it. But gaia init --profile npu does pull gemma4-it-e2b-FLM (init_command.py:101) — which is the intended behavior, not a bug. Consider narrowing the wording to "--profile all" so it doesn't read as "no init profile ever pulls it."

Strengths

• The import-time side-effect guard is the right test, done right — popping the module, patching the actual requests/subprocess boundaries before re-import, and the comment correctly explaining why patching load_model would be vacuous under importlib. This is the kind of test that catches a real future regression (an import-time weight pull), not boilerplate.
• Cross-file consistency is clean — model id, recipe (flm), and ctx_size=4096 match across the catalog, init profile, agent registry, and NPU docs. Docs were already updated in prior feat(email): primary on-device model integration (gemma4-it-e2b-FLM) #1282 work, so there's no doc gap here.
• Fails loud, stays local — the AC3 test confirms requesting the E2B model can't be used as a side door to a cloud base_url.

Verdict

Approve with suggestions — merge once the FLM tool-calling behavior is confirmed (🟡). The change is additive, well-tested, and consistent with the surrounding code; the only real risk is the tool-calling assumption on the NPU engine, which a single hardware triage call would settle.

[kovtcharov-amd]

Approving. Clean first-class catalog entry. The lazy-download guard (re-import crosses no requests/subprocess boundary) and the AC3 cloud-base_url block even when E2B is explicitly requested are well-targeted tests. The one thing not exercisable in CI is tool_calling=True on the FLM build — you've validated that on the Strix Halo box (device=npu/recipe=flm), so good.

[itomek]

Thanks for the thorough review — addressed the feedback and fixed the CI red.

🟡 tool_calling on the FLM build — good catch, and it does not hold. Verified directly on the Strix Halo NPU box: the FLM server 500-errors on a native OpenAI tools payload ("type must be string, but is object"), while plain completions work fine (~23 tok/s). The GGUF --jinja tool-calling assumption does not carry over to the FastFlowLM build. Changed the entry to tool_calling=False — the agent then uses the embedded-JSON tool path for this model, and email triage (which parses a JSON object from a plain completion, no native tool calls) is unaffected. The entry now carries a comment distinguishing it from the GGUF builds, and the test asserts False with that rationale.

🟢 "never enters the install path" — narrowed to --profile all in the description (you're right that --profile npu pulls it, by design).

CI failure — this was test pollution, not the catalog change. The lazy-download guard popped + re-imported lemonade_client in-process, rebuilding its classes and corrupting module identity (e.g. HardwareRequirementError) for later tests (test_hardware_selection and others), which red-listed the unit suite. Rewrote the guard to run its import probe in an isolated subprocess — no pollution, and a stronger guard (fresh interpreter, requests/subprocess instrumented to fail). Confirmed via a full-suite diff that the fixed branch introduces zero new failures vs main.