fix(agent): execute fenced tool calls with inline args and route bare email tool names#3681
Conversation
d3ca43c to
25dc723
Compare
|
This PR may also needs to update On current A regression test could extend |
… tool names
Two bugs made local (Ollama) models unable to use email tools, leaving
raw fences like ```list_email_accounts {}``` in the chat:
1. _TOOL_BLOCK_RE required a newline right after the fence tag, so a
tool call with args on the same line ("```list_email_accounts {}")
never matched and was never executed. The fence now matches with
optional spaces/newline after the tag.
2. Even when parsed, bare email tool names had no dispatch branch in
tool_execution.py and fell through to "Unknown tool type". They now
route to the email MCP server as mcp__email__<name>, matching how
function_call_to_tool_block already maps them for native callers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… fence-tag regex Review follow-up on odysseus-dev#3681 (thanks @vgalin): 1. Routing bare email names made 10 of the 14 email tools executable by non-admin owners — is_public_blocked_tool() runs on the bare name before dispatch, and NON_ADMIN_BLOCKED_TOOLS only listed 4. Define the full email tool set once (BUILTIN_EMAIL_TOOLS in tool_security.py) and derive the blocklist, the fence tags (TOOL_TAGS), the bare-name dispatch, and the native-call mapping from it so they can't drift. This also fixes 4 tools (search_emails, draft_email, draft_email_reply, ai_draft_email_reply) that were missing from the old tool_schemas copy and therefore unreachable even for native function-calling models. 2. The relaxed fence regex from the previous commit could prefix-match longer fence tags: ```python3 parsed as tool "python" with content "3\nprint(...)" and executed as code. Add a (?![\w-]) boundary after the tag. Tests: test_public_agent_policy_blocks_sensitive_tools now covers all 14 bare email names + the mcp__email__ form; new tests/test_fenced_inline_args.py pins inline-args parsing, the python3/hyphenated-tag non-matches, and strip/parse display mirroring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
25dc723 to
544a664
Compare
|
Good catch — confirmed and fixed in 544a664, plus a bit more than you listed. The gap was real: Rather than appending names to Your test suggestion is in: While re-reviewing the parsing change I also found and fixed a false-positive risk it introduced: the relaxed fence regex could prefix-match longer tags, so This comment was written via Claude and verified by a human. |
…-args-email-dispatch
… executing Markdown info strings Review follow-up on odysseus-dev#3681 (thanks @RaresKeY): 1. P1: execute_tool_block() checked disabled_tools / the turn ToolPolicy only against the incoming block name, then the bare-email branch qualified it to mcp__email__<name> and called the MCP manager. Plan mode and the MCP settings toggle write the QUALIFIED name into the denylist, so a bare fence like ```list_emails``` sailed past a mcp__email__list_emails entry. Both gates now match on both spellings (bare <-> mcp__email__-qualified), in either direction. 2. P2: the relaxed fence regex accepted arbitrary same-line text after a recognized tag, which made ordinary Markdown info strings executable: ```python title="example.py" ran as a python tool call. Same-line content now only counts as tool input when it starts with { or [ (JSON args); anything else leaves the fence as display text, and strip_tool_blocks mirrors that (the fence stays visible). Tests: disabled-tools alias regression (qualified entry blocks bare name and vice versa, never reaching the MCP manager), ToolPolicy alias regression, python/bash title="..." non-execution + display retention, and inline JSON-array args still parsing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Both confirmed and fixed in 9076be3; the branch is also merged up to current P1 (disabled-tools bypass): exactly as described — P2 (fence metadata): the same-line allowance is narrowed to JSON args only — after the tag, the rest of the opening line must be empty or start with Full suite after the dev merge: 3265 passed, 1 skipped. This comment was written via Claude and verified by a human. |
|
One field note from human testing of this PR build (gemma4:26b-a4b-it via Ollama, macOS), for anyone following along: now that fenced email calls actually execute, two pre-existing behaviours became visible that are not caused by this PR — the parsing/dispatch/persistence layers all behave correctly — but its testing exposed them:
Keeping these here for now since this PR's testing surfaced them — happy to split them into their own issues if maintainers prefer. This comment was written via Claude and verified by a human. |
|
I rechecked the latest head after the follow-up fixes. The earlier disabled-tool alias and simple metadata cases are covered now, but two gaps remain in the narrowed parser/policy surface. Findings
ValidationRan focused parser/tool-policy tests, chat-route tool-policy tests, adjacent fenced-parser tests, PR Hygiene
|
…-args-email-dispatch
…l set in the friendly toggle Review follow-up round 3 on odysseus-dev#3681 (thanks @RaresKeY): 1. Brace-style fence metadata no longer executes. The previous narrowing still treated any same-line {/[ after a recognized tag as tool input, so ```bash {title="setup"} ran as a bash call. The fence header is now captured separately and judged by one predicate shared between parse_tool_blocks and strip_tool_blocks (_fenced_tool_call), so the execute and display decisions can't disagree: same-line content only counts as inline args when the tag is NOT a code tag (bash/python never take same-line args — that text is Markdown fence attributes) AND the inline text (plus any continuation lines) parses as standalone JSON. ```bash {title="setup"}, ```python {"title":"example.py"} and ```list_emails {title="x"} all stay visible and inert. 2. The friendly `disable_tool email` toggle covered 3 of the 14 email tools (mcp__email__{list_emails,read_email,send_email}); the other bare aliases this PR routes stayed executable after an operator disabled email. The alias now derives from BUILTIN_EMAIL_TOOLS in BOTH spellings — bare (function-schema hiding, bare-fence dispatch) and mcp__email__* (MCP schema hiding, qualified runtime blocks) — so the toggle and the runtime gate can't drift apart. Tests: brace/bracket metadata regressions for parse and strip symmetry (code tags, invalid-JSON inline on a JSON tool, multi-line inline JSON still parsing), and disable_tool/enable_tool email covering all 14 names in both spellings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Re: botinate's field note on PR #3681: https://github.com/pewdiepie-archdaemon/odysseus/pull/3681#issuecomment-4683549687 That is the comment calling out the two pre-existing behaviors:
I rechecked both against latest Confirmed and split out:
So this PR can stay scoped to the inline-args and bare-email routing change. |
… email tool for plan mode Deep self-review follow-up on odysseus-dev#3681. Three review rounds each found another hand-maintained copy of the email tool list that had drifted; this commit hunts down ALL remaining copies and pins them to BUILTIN_EMAIL_TOOLS. The same 5 tools (search_emails, draft_email, draft_email_reply, ai_draft_email_reply, download_attachment) were missing from every advertising surface, so they were dispatchable but never offered: - FUNCTION_TOOL_SCHEMAS: native function-calling models never saw them (the round-1 fix covered dispatch only); schemas added, mirroring the email server's inputSchema definitions. - TOOL_SECTIONS: fenced-block models were never told about them; prompt sections added. - tool_index: absent from the RAG embedding registry (never retrievable), the email keyword hints, and the scheduled assistant's always-available set — the latter two now derive from BUILTIN_EMAIL_TOOLS. - agent_loop._DOMAIN_TOOL_MAP["email"], tool_policy._COMMON_TOOL_NAMES, the assistant tool-selector UI groups (assistant.js), and the default Assistant crew seed (task_scheduler) now derive from / cover the set. Plan mode now classifies every email tool explicitly: - list_email_accounts and search_emails join PLAN_MODE_READONLY_TOOLS. Without this, list_email_accounts sat in the plan-mode bare denylist (schema-derived) while its qualified form passed the MCP read-only filter — and the round-2 bare/qualified alias gate would have blocked the qualified call too, regressing read-only email discovery in plan mode. - draft_email, draft_email_reply, ai_draft_email_reply, and download_attachment join the fail-closed mutator backstop (drafts create documents; download_attachment writes to disk). Tests: tests/test_email_registry_sync.py pins every registry (including the email server source and assistant.js) to BUILTIN_EMAIL_TOOLS and asserts the plan-mode partition, so the next email tool can't drift; a parse/strip mirror grid covers 192 fence shapes (tag x header x body) asserting executed <=> stripped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ssistant seed constant Code-quality pass over the PR's own changes: - The bare<->qualified email aliasing rule lived inline in the generic dispatcher (_execute_tool_block_impl). It is policy knowledge, so it moves next to BUILTIN_EMAIL_TOOLS as email_tool_policy_names(); the dispatcher just consumes it, and the rule gets its own unit test (including the mcp__email__<not-a-tool> and mcp__other__ non-alias cases). - The default Assistant's enabled_tools list was an inline literal inside the CrewMember seed, and its registry-sync test asserted a source-code substring. Extracted to DEFAULT_ASSISTANT_ENABLED_TOOLS so the test imports and checks the actual value. - _fenced_tool_call return type tightened to Optional[Tuple[str, str]]. No behavior change; suite green (3295 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Both round-3 findings are fixed, and a deeper self-audit closed the rest of the drift class. Branch is merged up to current Brace-style metadata (b3d43ad): the fence header is captured separately and judged by one predicate shared between Email category toggle (b3d43ad): Drift audit (db29046, 016ce47): since each round found another hand-maintained email list, I audited the whole tree for copies. Seven more surfaces were missing the same 5 tools ( Validation: full suite 3294+ passed. Additionally, I replayed a corpus of real captured gemma/Ollama chat traffic through the parser: every legitimate fenced call from the previous revision still executes (including This comment was written via Claude and verified by a human. |
|
One reviewer/triage scope note before this grows further. The original #337 symptom is broad, but the narrow fix here looked like: local/fenced models need inline-arg parsing and bare email tool dispatch so configured email tools actually execute. Some follow-up broadening was necessary once bare email names became executable, especially the non-admin blocklist, disabled-tool alias checks, and fence metadata hardening. The later registry “drift audit” is useful cleanup, but it moves this PR into a broader email tool registry/policy/prompt/UI consolidation. That may be worth doing, but it changes the review and validation surface. Can we pick one framing?
Either way, I would avoid closing #337 solely from this PR until the email path has a real app retest and the calendar part has a clear disposition. |
|
Two more caught during self-audits and fixed (branch now at
Beyond those, the manual UAT and the audits turned up four pre-existing items not in this PR's core change. Rather than open issues on my own read, I'd value your call on whether any are worth filing — happy to write clean repros + tracebacks and/or send fixes:
Which, if any, would you like as issues? This comment was written via Claude and verified by a human. |
|
While live-testing the model-driven email path here, I found a separate current- I split that into a narrow stabilization fix: #4195. It is not caused by this PR's fenced/native dispatch changes, but it affects live model-driven email validation for this PR. I also retested this PR on a branch stacked over the separate auth-disabled tool-exposure fix. On that stack:
Not run on the stacked branch: browser/UI E2E, mutating email actions, GitHub CI, and a direct retest of unstacked #3681 after the separate fix lands. |
|
I rechecked the current state after #4195 landed. #4195 is merged into The remaining blocker is that GitHub now marks #3681 stale/conflicting with current
After that, can you rerun the unstacked validation on the rebased branch? The useful minimum would be the focused parser/email/security tests, full pytest, and the live email-agent path now that #4195 is in For your last list of follow-ups: |
…-args-email-dispatch # Conflicts: # src/tool_schemas.py # tests/test_review_regressions.py
|
Rebased onto current This comment was written via Claude and verified by a human. |
|
Rechecked latest head Please rebase, port the full |
Resolve the conflict from the upstream tool_implementations.py -> src/agent_tools/admin_tools.py move of manage_settings (odysseus-dev#4423/odysseus-dev#3629): - src/tool_implementations.py: take upstream's facade. This branch's only change to that file was the manage_settings "email" tool-category expansion, which moved with do_manage_settings to admin_tools.py. - src/agent_tools/admin_tools.py: port the full `disable_tool email` coverage there, deriving the set from BUILTIN_EMAIL_TOOLS in both spellings (bare + mcp__email__*) instead of the hand-picked 3-tool subset. The search/web/research toggles keep current dev behavior (["web_search", "web_fetch"]). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Semantic conflict from the dev merge that textual auto-merge didn't flag:
dev added test_live_strip_email_tool_fences.py whose _tool_tags() helper
source-scrapes only the TOOL_TAGS literal `{...}`, which worked on dev
because the email tool names were listed inline there. This branch makes
TOOL_TAGS the single source — `{...} | BUILTIN_EMAIL_TOOLS` — so the email
names are no longer in the literal and the scraper missed them, leaving the
email-fence strip assertions failing even though TOOL_TAGS does contain them
at runtime.
Import the real TOOL_TAGS instead of scraping source, so the test mirrors
exactly what GET /api/tools serves (sorted(TOOL_TAGS)) and the live
EXEC_FENCE_RE derives from — robust to however the set is composed. The
source-level frontend/route guards in the same file are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Brought the branch up to current
One semantic conflict that textual auto-merge did not flag, caught on the test run: dev added Validation — focused parser/email/security/admin batch (22 files incl. This comment was written via Claude and verified by a human. |
Resolve the strip_tool_blocks conflict from dev's tool-call-parser ReDoS hardening (odysseus-dev#4941) landing on this branch's fenced-tool changes: - Keep this branch's executed-vs-illustrative fence handling (_TOOL_BLOCK_RE.sub(_strip_executed_fence, ...)) so example fences from native function-calling models stay visible/inert while dispatched fences are stripped — the executed<->stripped invariant. - Take dev's forward-only [TOOL_CALL]{...}[/TOOL_CALL] removal (_strip_delimited over _TOOL_CALL_OPEN_RE/_CLOSE_RE) in place of the old _TOOL_CALL_RE.sub(''), so untrusted output can't drive the O(n^2) lazy rescan. This mirrors the parse side, which already scans the same delimiters via _iter_delimited. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
The two edits were independent, so the resolution keeps both:
Validation: parser/strip/ReDoS set — This comment was written via Claude and verified by a human. |
alteixeira20
left a comment
There was a problem hiding this comment.
LGTM.
Reviewed the PR body, full discussion, changed files, staged merge-result diff, and focused validation. This fixes the local/fenced-tool path for inline JSON arguments and bare email tool names, while preserving the required safety gates around email/MCP tools.
Validated:
- PR discussion has no unresolved review threads, no review comments, and no formal blocking reviews.
- Expected head guard passed.
- Changed-file allowlist passed.
- Merge probe is clean against current dev.
- GitHub checks are green.
- git diff --check is clean.
- The 6 model/default-chat failures in tests/test_review_regressions.py reproduce on clean dev with the same core.log_safety import error, so they are not PR-caused.
- Focused PR-relevant validation passed: py_compile plus pytest on email registry, fenced inline args, live strip email fences, and review regressions with the known unrelated model/default-chat failures deselected.
Source-level review notes:
- parse_tool_blocks and strip_tool_blocks now share the same executed-vs-illustrative fence decision.
- bash/python Markdown fence metadata remains visible and inert.
- bare and mcp__email__ tool spellings are checked together by disabled-tool and ToolPolicy gates.
- non-admin users are blocked from all built-in email tools.
- plan mode explicitly classifies all built-in email tools.
- disable_tool email covers all bare and qualified email tool names.
- bare email calls reject non-object or invalid JSON args instead of silently falling back to default mailbox args.
… email tool names (odysseus-dev#3681) * fix(agent): execute fenced tool calls with inline args and bare email tool names Two bugs made local (Ollama) models unable to use email tools, leaving raw fences like ```list_email_accounts {}``` in the chat: 1. _TOOL_BLOCK_RE required a newline right after the fence tag, so a tool call with args on the same line ("```list_email_accounts {}") never matched and was never executed. The fence now matches with optional spaces/newline after the tag. 2. Even when parsed, bare email tool names had no dispatch branch in tool_execution.py and fell through to "Unknown tool type". They now route to the email MCP server as mcp__email__<name>, matching how function_call_to_tool_block already maps them for native callers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): block all bare email tool names for non-admins; harden fence-tag regex Review follow-up on odysseus-dev#3681 (thanks @vgalin): 1. Routing bare email names made 10 of the 14 email tools executable by non-admin owners — is_public_blocked_tool() runs on the bare name before dispatch, and NON_ADMIN_BLOCKED_TOOLS only listed 4. Define the full email tool set once (BUILTIN_EMAIL_TOOLS in tool_security.py) and derive the blocklist, the fence tags (TOOL_TAGS), the bare-name dispatch, and the native-call mapping from it so they can't drift. This also fixes 4 tools (search_emails, draft_email, draft_email_reply, ai_draft_email_reply) that were missing from the old tool_schemas copy and therefore unreachable even for native function-calling models. 2. The relaxed fence regex from the previous commit could prefix-match longer fence tags: ```python3 parsed as tool "python" with content "3\nprint(...)" and executed as code. Add a (?![\w-]) boundary after the tag. Tests: test_public_agent_policy_blocks_sensitive_tools now covers all 14 bare email names + the mcp__email__ form; new tests/test_fenced_inline_args.py pins inline-args parsing, the python3/hyphenated-tag non-matches, and strip/parse display mirroring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): gate bare and mcp-qualified email names together; stop executing Markdown info strings Review follow-up on odysseus-dev#3681 (thanks @RaresKeY): 1. P1: execute_tool_block() checked disabled_tools / the turn ToolPolicy only against the incoming block name, then the bare-email branch qualified it to mcp__email__<name> and called the MCP manager. Plan mode and the MCP settings toggle write the QUALIFIED name into the denylist, so a bare fence like ```list_emails``` sailed past a mcp__email__list_emails entry. Both gates now match on both spellings (bare <-> mcp__email__-qualified), in either direction. 2. P2: the relaxed fence regex accepted arbitrary same-line text after a recognized tag, which made ordinary Markdown info strings executable: ```python title="example.py" ran as a python tool call. Same-line content now only counts as tool input when it starts with { or [ (JSON args); anything else leaves the fence as display text, and strip_tool_blocks mirrors that (the fence stays visible). Tests: disabled-tools alias regression (qualified entry blocks bare name and vice versa, never reaching the MCP manager), ToolPolicy alias regression, python/bash title="..." non-execution + display retention, and inline JSON-array args still parsing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): reject brace-style fence metadata; cover the full email set in the friendly toggle Review follow-up round 3 on odysseus-dev#3681 (thanks @RaresKeY): 1. Brace-style fence metadata no longer executes. The previous narrowing still treated any same-line {/[ after a recognized tag as tool input, so ```bash {title="setup"} ran as a bash call. The fence header is now captured separately and judged by one predicate shared between parse_tool_blocks and strip_tool_blocks (_fenced_tool_call), so the execute and display decisions can't disagree: same-line content only counts as inline args when the tag is NOT a code tag (bash/python never take same-line args — that text is Markdown fence attributes) AND the inline text (plus any continuation lines) parses as standalone JSON. ```bash {title="setup"}, ```python {"title":"example.py"} and ```list_emails {title="x"} all stay visible and inert. 2. The friendly `disable_tool email` toggle covered 3 of the 14 email tools (mcp__email__{list_emails,read_email,send_email}); the other bare aliases this PR routes stayed executable after an operator disabled email. The alias now derives from BUILTIN_EMAIL_TOOLS in BOTH spellings — bare (function-schema hiding, bare-fence dispatch) and mcp__email__* (MCP schema hiding, qualified runtime blocks) — so the toggle and the runtime gate can't drift apart. Tests: brace/bracket metadata regressions for parse and strip symmetry (code tags, invalid-JSON inline on a JSON tool, multi-line inline JSON still parsing), and disable_tool/enable_tool email covering all 14 names in both spellings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(email): close remaining email-tool registry drift; classify every email tool for plan mode Deep self-review follow-up on odysseus-dev#3681. Three review rounds each found another hand-maintained copy of the email tool list that had drifted; this commit hunts down ALL remaining copies and pins them to BUILTIN_EMAIL_TOOLS. The same 5 tools (search_emails, draft_email, draft_email_reply, ai_draft_email_reply, download_attachment) were missing from every advertising surface, so they were dispatchable but never offered: - FUNCTION_TOOL_SCHEMAS: native function-calling models never saw them (the round-1 fix covered dispatch only); schemas added, mirroring the email server's inputSchema definitions. - TOOL_SECTIONS: fenced-block models were never told about them; prompt sections added. - tool_index: absent from the RAG embedding registry (never retrievable), the email keyword hints, and the scheduled assistant's always-available set — the latter two now derive from BUILTIN_EMAIL_TOOLS. - agent_loop._DOMAIN_TOOL_MAP["email"], tool_policy._COMMON_TOOL_NAMES, the assistant tool-selector UI groups (assistant.js), and the default Assistant crew seed (task_scheduler) now derive from / cover the set. Plan mode now classifies every email tool explicitly: - list_email_accounts and search_emails join PLAN_MODE_READONLY_TOOLS. Without this, list_email_accounts sat in the plan-mode bare denylist (schema-derived) while its qualified form passed the MCP read-only filter — and the round-2 bare/qualified alias gate would have blocked the qualified call too, regressing read-only email discovery in plan mode. - draft_email, draft_email_reply, ai_draft_email_reply, and download_attachment join the fail-closed mutator backstop (drafts create documents; download_attachment writes to disk). Tests: tests/test_email_registry_sync.py pins every registry (including the email server source and assistant.js) to BUILTIN_EMAIL_TOOLS and asserts the plan-mode partition, so the next email tool can't drift; a parse/strip mirror grid covers 192 fence shapes (tag x header x body) asserting executed <=> stripped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: move the email alias rule into tool_security; extract the assistant seed constant Code-quality pass over the PR's own changes: - The bare<->qualified email aliasing rule lived inline in the generic dispatcher (_execute_tool_block_impl). It is policy knowledge, so it moves next to BUILTIN_EMAIL_TOOLS as email_tool_policy_names(); the dispatcher just consumes it, and the rule gets its own unit test (including the mcp__email__<not-a-tool> and mcp__other__ non-alias cases). - The default Assistant's enabled_tools list was an inline literal inside the CrewMember seed, and its registry-sync test asserted a source-code substring. Extracted to DEFAULT_ASSISTANT_ENABLED_TOOLS so the test imports and checks the actual value. - _fenced_tool_call return type tightened to Optional[Tuple[str, str]]. No behavior change; suite green (3295 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * revert: move the email registry consolidation to a follow-up PR Per review feedback on scope, this PR stays narrow: fenced inline-args parsing, bare email tool routing, and the directly required safety gates. This commit reverts the registry/advertising consolidation from db29046 and 016ce47 (native schemas, prompt sections, RAG description index + keyword hints, assistant always-available set, guide-only known-names union, frontend tool-selector groups, default assistant seed, and their sync tests) — all of that moves to a dedicated follow-up PR together with the _EMAIL_TOOL_HINTS finding. Kept here because the narrow scope needs them: - email_tool_policy_names() in tool_security + its use in the execute_tool_block gates and its unit test (refactor of this PR's own round-2 alias fix), - list_email_accounts in PLAN_MODE_READONLY_TOOLS (the alias gate works both ways, and the schema-derived plan-mode bare denylist would otherwise block the qualified read-only call too), - the parse/strip mirror grid test (parser scope), - the narrow registry sync tests (email server <-> BUILTIN_EMAIL_TOOLS match, fence-tag coverage, non-admin blocklist coverage). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(email): execute empty email fences with empty args; reject non-object JSON args Two gaps found by replaying captured local-model traffic against the narrowed branch: 1. ```list_email_accounts``` with NO body — a shape gemma really emits for no-arg tools — was silently dropped (parse skips empty content), so the model concluded email was broken: the original odysseus-dev#337 symptom through a different door. Empty fences whose tag is a built-in email tool now dispatch with {} args and the tool's own validation answers (e.g. an empty send_email returns "to is required" instead of silence). Empty bash/python/other fences keep skipping, and strip stays mirrored (the fence was executed, so it is removed). 2. The fence parser accepts JSON arrays as inline args, but the email dispatch parsed only objects — an array silently became {} args. Non-object JSON now returns a correctable "arguments must be a JSON object" error before reaching the MCP server (same class as odysseus-dev#3966). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): classify all email tools for plan mode statically; reject invalid email JSON bodies Review follow-up round 5 on odysseus-dev#3681 (thanks @RaresKeY): 1. This PR makes every BUILTIN_EMAIL_TOOLS name fence-taggable, so each one must be explicitly classified for plan mode — the draft tools and download_attachment were in neither the read-only allowlist nor the static denylist, leaving their bare-alias plan-mode safety dependent on the MCP read-only inventory being present and current. search_emails joins PLAN_MODE_READONLY_TOOLS (explicit, not allowed-by-omission); draft_email, draft_email_reply, ai_draft_email_reply, and download_attachment join the fail-closed _PLAN_MODE_KNOWN_MUTATORS backstop. (Moved back from the odysseus-dev#4053 split: the partition is directly required for this PR to merge independently.) 2. The classic tag/body fence form reaches execution unvalidated (only INLINE args are JSON-checked by the parser), so a body like {account: "work"} silently became {} args and read the DEFAULT mailbox instead of the intended one. JSON-looking bodies that fail to parse now return a correctable "not valid JSON" error before reaching the MCP server. Tests: a partition invariant (every email tool is explicitly read-only or plan-mode-denied), a mutating-alias probe that uses only the static denylist with a fake MCP manager (no inventory layer), and the body-form invalid-JSON regression. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tool-dispatch): decode inline JSON args for legacy MCP tools; reject all non-object email bodies Review follow-up round 6 on odysseus-dev#3681 (thanks @RaresKeY) — both pre-existing on this branch, surfaced by the relaxed inline-args parser: 1. The relaxed parser accepts inline JSON for every non-code tag, but the legacy line-based arg builders (web_search/web_fetch/read_file/ write_file/generate_image/manage_memory) wrapped the whole JSON string as the query/url/path/prompt — so `web_search {"query": "x"}` executed as a search for the literal string `{"query": "x"}`. _build_mcp_args now uses a fenced JSON object directly when it carries the tool's primary arg key (query/url/path/prompt/action). Keyed off membership so it can't drift; an object without the primary key (e.g. a freeform JSON query, or bare object content for write_file) falls through to the line parser unchanged. Also fixes the same corruption for the classic newline-JSON form. 2. The bare-email dispatch only rejected bodies starting with { or [, so a non-empty non-JSON body like `account: work` still fell through to {} args and silently read the DEFAULT mailbox. Now ANY non-empty body must decode to a JSON object or it returns a correctable error; only a truly empty body keeps the no-arg path (```list_email_accounts```). Tests: inline-JSON arg decoding for the five legacy tools plus the freeform and missing-primary-key fallbacks; the email body rejection extended to cover the brace-looking and bare `key: value` shapes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tool-dispatch): drop dead manage_memory JSON-decode entry; pin the live-path invariant Self-audit catch on the round-6 fix. manage_memory was added to _MCP_JSON_PRIMARY_KEYS, but _build_mcp_args is only reached via _call_mcp_tool, which only runs for _MCP_TOOL_MAP tools — and manage_memory isn't one (its tag routes through dispatch_ai_tool -> do_manage_memory, which line-parses). So the round-6 decode for manage_memory was dead code: the unit test exercising _build_mcp_args passed while a real `manage_memory {"action": ...}` fence still parsed the whole JSON blob as the action. Remove the dead entry and add test_mcp_json_primary_keys_are_all_live, which asserts every JSON-primary tool is in _MCP_TOOL_MAP so a dead decode can't be added again. The same inline-JSON corruption for manage_memory and the other tools that route through positional dispatchers (create_session, ui_control, send_to_session, search_chats, the document tools, etc.) is pre-existing (dev corrupts their newline JSON form too) and tracked separately; the proper fix there is to route fenced JSON through function_call_to_tool_block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tool-dispatch): decode inline JSON in WriteFileTool (its live path); round-6 fix was on the dead MCP path Self-audit: round 6 claimed to fix inline JSON args for write_file via _build_mcp_args, but there is no filesystem MCP server, so write_file always runs through _direct_fallback -> WriteFileTool, never through _build_mcp_args. WriteFileTool — unlike its siblings ReadFileTool / WebSearchTool / WebFetchTool, which all decode JSON — took lines[0] as the path, so `write_file {"path": "/tmp/x", "content": "y"}` wrote to a file literally named with the JSON blob. The round-6 _build_mcp_args entry decoded correctly but on a path that never executes (same class as the manage_memory dead entry), and the round-6 unit test passed on that dead path. WriteFileTool now decodes a JSON object carrying "path" (matching ReadFileTool directly above it), and the comment on _MCP_JSON_PRIMARY_KEYS records that only generate_image has a live MCP server today — the other entries are defense-in-depth for the MCP path; the live fix for each server-less tool is in its handler. Test: test_write_file_inline_json_args drives the LIVE path (execute_tool_block with no MCP) and asserts the intended path is used — verified to fail without the handler fix. web_search/web_fetch/read_file were already correct (their handlers decode); write_file was the gap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(strip-fence): derive the live-strip TOOL_TAGS from the real set Semantic conflict from the dev merge that textual auto-merge didn't flag: dev added test_live_strip_email_tool_fences.py whose _tool_tags() helper source-scrapes only the TOOL_TAGS literal `{...}`, which worked on dev because the email tool names were listed inline there. This branch makes TOOL_TAGS the single source — `{...} | BUILTIN_EMAIL_TOOLS` — so the email names are no longer in the literal and the scraper missed them, leaving the email-fence strip assertions failing even though TOOL_TAGS does contain them at runtime. Import the real TOOL_TAGS instead of scraping source, so the test mirrors exactly what GET /api/tools serves (sorted(TOOL_TAGS)) and the live EXEC_FENCE_RE derives from — robust to however the set is composed. The source-level frontend/route guards in the same file are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: botinate <285686135+botinate@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
… email tool names (odysseus-dev#3681) * fix(agent): execute fenced tool calls with inline args and bare email tool names Two bugs made local (Ollama) models unable to use email tools, leaving raw fences like ```list_email_accounts {}``` in the chat: 1. _TOOL_BLOCK_RE required a newline right after the fence tag, so a tool call with args on the same line ("```list_email_accounts {}") never matched and was never executed. The fence now matches with optional spaces/newline after the tag. 2. Even when parsed, bare email tool names had no dispatch branch in tool_execution.py and fell through to "Unknown tool type". They now route to the email MCP server as mcp__email__<name>, matching how function_call_to_tool_block already maps them for native callers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): block all bare email tool names for non-admins; harden fence-tag regex Review follow-up on odysseus-dev#3681 (thanks @vgalin): 1. Routing bare email names made 10 of the 14 email tools executable by non-admin owners — is_public_blocked_tool() runs on the bare name before dispatch, and NON_ADMIN_BLOCKED_TOOLS only listed 4. Define the full email tool set once (BUILTIN_EMAIL_TOOLS in tool_security.py) and derive the blocklist, the fence tags (TOOL_TAGS), the bare-name dispatch, and the native-call mapping from it so they can't drift. This also fixes 4 tools (search_emails, draft_email, draft_email_reply, ai_draft_email_reply) that were missing from the old tool_schemas copy and therefore unreachable even for native function-calling models. 2. The relaxed fence regex from the previous commit could prefix-match longer fence tags: ```python3 parsed as tool "python" with content "3\nprint(...)" and executed as code. Add a (?![\w-]) boundary after the tag. Tests: test_public_agent_policy_blocks_sensitive_tools now covers all 14 bare email names + the mcp__email__ form; new tests/test_fenced_inline_args.py pins inline-args parsing, the python3/hyphenated-tag non-matches, and strip/parse display mirroring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): gate bare and mcp-qualified email names together; stop executing Markdown info strings Review follow-up on odysseus-dev#3681 (thanks @RaresKeY): 1. P1: execute_tool_block() checked disabled_tools / the turn ToolPolicy only against the incoming block name, then the bare-email branch qualified it to mcp__email__<name> and called the MCP manager. Plan mode and the MCP settings toggle write the QUALIFIED name into the denylist, so a bare fence like ```list_emails``` sailed past a mcp__email__list_emails entry. Both gates now match on both spellings (bare <-> mcp__email__-qualified), in either direction. 2. P2: the relaxed fence regex accepted arbitrary same-line text after a recognized tag, which made ordinary Markdown info strings executable: ```python title="example.py" ran as a python tool call. Same-line content now only counts as tool input when it starts with { or [ (JSON args); anything else leaves the fence as display text, and strip_tool_blocks mirrors that (the fence stays visible). Tests: disabled-tools alias regression (qualified entry blocks bare name and vice versa, never reaching the MCP manager), ToolPolicy alias regression, python/bash title="..." non-execution + display retention, and inline JSON-array args still parsing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): reject brace-style fence metadata; cover the full email set in the friendly toggle Review follow-up round 3 on odysseus-dev#3681 (thanks @RaresKeY): 1. Brace-style fence metadata no longer executes. The previous narrowing still treated any same-line {/[ after a recognized tag as tool input, so ```bash {title="setup"} ran as a bash call. The fence header is now captured separately and judged by one predicate shared between parse_tool_blocks and strip_tool_blocks (_fenced_tool_call), so the execute and display decisions can't disagree: same-line content only counts as inline args when the tag is NOT a code tag (bash/python never take same-line args — that text is Markdown fence attributes) AND the inline text (plus any continuation lines) parses as standalone JSON. ```bash {title="setup"}, ```python {"title":"example.py"} and ```list_emails {title="x"} all stay visible and inert. 2. The friendly `disable_tool email` toggle covered 3 of the 14 email tools (mcp__email__{list_emails,read_email,send_email}); the other bare aliases this PR routes stayed executable after an operator disabled email. The alias now derives from BUILTIN_EMAIL_TOOLS in BOTH spellings — bare (function-schema hiding, bare-fence dispatch) and mcp__email__* (MCP schema hiding, qualified runtime blocks) — so the toggle and the runtime gate can't drift apart. Tests: brace/bracket metadata regressions for parse and strip symmetry (code tags, invalid-JSON inline on a JSON tool, multi-line inline JSON still parsing), and disable_tool/enable_tool email covering all 14 names in both spellings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(email): close remaining email-tool registry drift; classify every email tool for plan mode Deep self-review follow-up on odysseus-dev#3681. Three review rounds each found another hand-maintained copy of the email tool list that had drifted; this commit hunts down ALL remaining copies and pins them to BUILTIN_EMAIL_TOOLS. The same 5 tools (search_emails, draft_email, draft_email_reply, ai_draft_email_reply, download_attachment) were missing from every advertising surface, so they were dispatchable but never offered: - FUNCTION_TOOL_SCHEMAS: native function-calling models never saw them (the round-1 fix covered dispatch only); schemas added, mirroring the email server's inputSchema definitions. - TOOL_SECTIONS: fenced-block models were never told about them; prompt sections added. - tool_index: absent from the RAG embedding registry (never retrievable), the email keyword hints, and the scheduled assistant's always-available set — the latter two now derive from BUILTIN_EMAIL_TOOLS. - agent_loop._DOMAIN_TOOL_MAP["email"], tool_policy._COMMON_TOOL_NAMES, the assistant tool-selector UI groups (assistant.js), and the default Assistant crew seed (task_scheduler) now derive from / cover the set. Plan mode now classifies every email tool explicitly: - list_email_accounts and search_emails join PLAN_MODE_READONLY_TOOLS. Without this, list_email_accounts sat in the plan-mode bare denylist (schema-derived) while its qualified form passed the MCP read-only filter — and the round-2 bare/qualified alias gate would have blocked the qualified call too, regressing read-only email discovery in plan mode. - draft_email, draft_email_reply, ai_draft_email_reply, and download_attachment join the fail-closed mutator backstop (drafts create documents; download_attachment writes to disk). Tests: tests/test_email_registry_sync.py pins every registry (including the email server source and assistant.js) to BUILTIN_EMAIL_TOOLS and asserts the plan-mode partition, so the next email tool can't drift; a parse/strip mirror grid covers 192 fence shapes (tag x header x body) asserting executed <=> stripped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: move the email alias rule into tool_security; extract the assistant seed constant Code-quality pass over the PR's own changes: - The bare<->qualified email aliasing rule lived inline in the generic dispatcher (_execute_tool_block_impl). It is policy knowledge, so it moves next to BUILTIN_EMAIL_TOOLS as email_tool_policy_names(); the dispatcher just consumes it, and the rule gets its own unit test (including the mcp__email__<not-a-tool> and mcp__other__ non-alias cases). - The default Assistant's enabled_tools list was an inline literal inside the CrewMember seed, and its registry-sync test asserted a source-code substring. Extracted to DEFAULT_ASSISTANT_ENABLED_TOOLS so the test imports and checks the actual value. - _fenced_tool_call return type tightened to Optional[Tuple[str, str]]. No behavior change; suite green (3295 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * revert: move the email registry consolidation to a follow-up PR Per review feedback on scope, this PR stays narrow: fenced inline-args parsing, bare email tool routing, and the directly required safety gates. This commit reverts the registry/advertising consolidation from db29046 and 016ce47 (native schemas, prompt sections, RAG description index + keyword hints, assistant always-available set, guide-only known-names union, frontend tool-selector groups, default assistant seed, and their sync tests) — all of that moves to a dedicated follow-up PR together with the _EMAIL_TOOL_HINTS finding. Kept here because the narrow scope needs them: - email_tool_policy_names() in tool_security + its use in the execute_tool_block gates and its unit test (refactor of this PR's own round-2 alias fix), - list_email_accounts in PLAN_MODE_READONLY_TOOLS (the alias gate works both ways, and the schema-derived plan-mode bare denylist would otherwise block the qualified read-only call too), - the parse/strip mirror grid test (parser scope), - the narrow registry sync tests (email server <-> BUILTIN_EMAIL_TOOLS match, fence-tag coverage, non-admin blocklist coverage). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(email): execute empty email fences with empty args; reject non-object JSON args Two gaps found by replaying captured local-model traffic against the narrowed branch: 1. ```list_email_accounts``` with NO body — a shape gemma really emits for no-arg tools — was silently dropped (parse skips empty content), so the model concluded email was broken: the original odysseus-dev#337 symptom through a different door. Empty fences whose tag is a built-in email tool now dispatch with {} args and the tool's own validation answers (e.g. an empty send_email returns "to is required" instead of silence). Empty bash/python/other fences keep skipping, and strip stays mirrored (the fence was executed, so it is removed). 2. The fence parser accepts JSON arrays as inline args, but the email dispatch parsed only objects — an array silently became {} args. Non-object JSON now returns a correctable "arguments must be a JSON object" error before reaching the MCP server (same class as odysseus-dev#3966). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): classify all email tools for plan mode statically; reject invalid email JSON bodies Review follow-up round 5 on odysseus-dev#3681 (thanks @RaresKeY): 1. This PR makes every BUILTIN_EMAIL_TOOLS name fence-taggable, so each one must be explicitly classified for plan mode — the draft tools and download_attachment were in neither the read-only allowlist nor the static denylist, leaving their bare-alias plan-mode safety dependent on the MCP read-only inventory being present and current. search_emails joins PLAN_MODE_READONLY_TOOLS (explicit, not allowed-by-omission); draft_email, draft_email_reply, ai_draft_email_reply, and download_attachment join the fail-closed _PLAN_MODE_KNOWN_MUTATORS backstop. (Moved back from the odysseus-dev#4053 split: the partition is directly required for this PR to merge independently.) 2. The classic tag/body fence form reaches execution unvalidated (only INLINE args are JSON-checked by the parser), so a body like {account: "work"} silently became {} args and read the DEFAULT mailbox instead of the intended one. JSON-looking bodies that fail to parse now return a correctable "not valid JSON" error before reaching the MCP server. Tests: a partition invariant (every email tool is explicitly read-only or plan-mode-denied), a mutating-alias probe that uses only the static denylist with a fake MCP manager (no inventory layer), and the body-form invalid-JSON regression. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tool-dispatch): decode inline JSON args for legacy MCP tools; reject all non-object email bodies Review follow-up round 6 on odysseus-dev#3681 (thanks @RaresKeY) — both pre-existing on this branch, surfaced by the relaxed inline-args parser: 1. The relaxed parser accepts inline JSON for every non-code tag, but the legacy line-based arg builders (web_search/web_fetch/read_file/ write_file/generate_image/manage_memory) wrapped the whole JSON string as the query/url/path/prompt — so `web_search {"query": "x"}` executed as a search for the literal string `{"query": "x"}`. _build_mcp_args now uses a fenced JSON object directly when it carries the tool's primary arg key (query/url/path/prompt/action). Keyed off membership so it can't drift; an object without the primary key (e.g. a freeform JSON query, or bare object content for write_file) falls through to the line parser unchanged. Also fixes the same corruption for the classic newline-JSON form. 2. The bare-email dispatch only rejected bodies starting with { or [, so a non-empty non-JSON body like `account: work` still fell through to {} args and silently read the DEFAULT mailbox. Now ANY non-empty body must decode to a JSON object or it returns a correctable error; only a truly empty body keeps the no-arg path (```list_email_accounts```). Tests: inline-JSON arg decoding for the five legacy tools plus the freeform and missing-primary-key fallbacks; the email body rejection extended to cover the brace-looking and bare `key: value` shapes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tool-dispatch): drop dead manage_memory JSON-decode entry; pin the live-path invariant Self-audit catch on the round-6 fix. manage_memory was added to _MCP_JSON_PRIMARY_KEYS, but _build_mcp_args is only reached via _call_mcp_tool, which only runs for _MCP_TOOL_MAP tools — and manage_memory isn't one (its tag routes through dispatch_ai_tool -> do_manage_memory, which line-parses). So the round-6 decode for manage_memory was dead code: the unit test exercising _build_mcp_args passed while a real `manage_memory {"action": ...}` fence still parsed the whole JSON blob as the action. Remove the dead entry and add test_mcp_json_primary_keys_are_all_live, which asserts every JSON-primary tool is in _MCP_TOOL_MAP so a dead decode can't be added again. The same inline-JSON corruption for manage_memory and the other tools that route through positional dispatchers (create_session, ui_control, send_to_session, search_chats, the document tools, etc.) is pre-existing (dev corrupts their newline JSON form too) and tracked separately; the proper fix there is to route fenced JSON through function_call_to_tool_block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tool-dispatch): decode inline JSON in WriteFileTool (its live path); round-6 fix was on the dead MCP path Self-audit: round 6 claimed to fix inline JSON args for write_file via _build_mcp_args, but there is no filesystem MCP server, so write_file always runs through _direct_fallback -> WriteFileTool, never through _build_mcp_args. WriteFileTool — unlike its siblings ReadFileTool / WebSearchTool / WebFetchTool, which all decode JSON — took lines[0] as the path, so `write_file {"path": "/tmp/x", "content": "y"}` wrote to a file literally named with the JSON blob. The round-6 _build_mcp_args entry decoded correctly but on a path that never executes (same class as the manage_memory dead entry), and the round-6 unit test passed on that dead path. WriteFileTool now decodes a JSON object carrying "path" (matching ReadFileTool directly above it), and the comment on _MCP_JSON_PRIMARY_KEYS records that only generate_image has a live MCP server today — the other entries are defense-in-depth for the MCP path; the live fix for each server-less tool is in its handler. Test: test_write_file_inline_json_args drives the LIVE path (execute_tool_block with no MCP) and asserts the intended path is used — verified to fail without the handler fix. web_search/web_fetch/read_file were already correct (their handlers decode); write_file was the gap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(strip-fence): derive the live-strip TOOL_TAGS from the real set Semantic conflict from the dev merge that textual auto-merge didn't flag: dev added test_live_strip_email_tool_fences.py whose _tool_tags() helper source-scrapes only the TOOL_TAGS literal `{...}`, which worked on dev because the email tool names were listed inline there. This branch makes TOOL_TAGS the single source — `{...} | BUILTIN_EMAIL_TOOLS` — so the email names are no longer in the literal and the scraper missed them, leaving the email-fence strip assertions failing even though TOOL_TAGS does contain them at runtime. Import the real TOOL_TAGS instead of scraping source, so the test mirrors exactly what GET /api/tools serves (sorted(TOOL_TAGS)) and the live EXEC_FENCE_RE derives from — robust to however the set is composed. The source-level frontend/route guards in the same file are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: botinate <285686135+botinate@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
… email tool names (odysseus-dev#3681) * fix(agent): execute fenced tool calls with inline args and bare email tool names Two bugs made local (Ollama) models unable to use email tools, leaving raw fences like ```list_email_accounts {}``` in the chat: 1. _TOOL_BLOCK_RE required a newline right after the fence tag, so a tool call with args on the same line ("```list_email_accounts {}") never matched and was never executed. The fence now matches with optional spaces/newline after the tag. 2. Even when parsed, bare email tool names had no dispatch branch in tool_execution.py and fell through to "Unknown tool type". They now route to the email MCP server as mcp__email__<name>, matching how function_call_to_tool_block already maps them for native callers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): block all bare email tool names for non-admins; harden fence-tag regex Review follow-up on odysseus-dev#3681 (thanks @vgalin): 1. Routing bare email names made 10 of the 14 email tools executable by non-admin owners — is_public_blocked_tool() runs on the bare name before dispatch, and NON_ADMIN_BLOCKED_TOOLS only listed 4. Define the full email tool set once (BUILTIN_EMAIL_TOOLS in tool_security.py) and derive the blocklist, the fence tags (TOOL_TAGS), the bare-name dispatch, and the native-call mapping from it so they can't drift. This also fixes 4 tools (search_emails, draft_email, draft_email_reply, ai_draft_email_reply) that were missing from the old tool_schemas copy and therefore unreachable even for native function-calling models. 2. The relaxed fence regex from the previous commit could prefix-match longer fence tags: ```python3 parsed as tool "python" with content "3\nprint(...)" and executed as code. Add a (?![\w-]) boundary after the tag. Tests: test_public_agent_policy_blocks_sensitive_tools now covers all 14 bare email names + the mcp__email__ form; new tests/test_fenced_inline_args.py pins inline-args parsing, the python3/hyphenated-tag non-matches, and strip/parse display mirroring. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): gate bare and mcp-qualified email names together; stop executing Markdown info strings Review follow-up on odysseus-dev#3681 (thanks @RaresKeY): 1. P1: execute_tool_block() checked disabled_tools / the turn ToolPolicy only against the incoming block name, then the bare-email branch qualified it to mcp__email__<name> and called the MCP manager. Plan mode and the MCP settings toggle write the QUALIFIED name into the denylist, so a bare fence like ```list_emails``` sailed past a mcp__email__list_emails entry. Both gates now match on both spellings (bare <-> mcp__email__-qualified), in either direction. 2. P2: the relaxed fence regex accepted arbitrary same-line text after a recognized tag, which made ordinary Markdown info strings executable: ```python title="example.py" ran as a python tool call. Same-line content now only counts as tool input when it starts with { or [ (JSON args); anything else leaves the fence as display text, and strip_tool_blocks mirrors that (the fence stays visible). Tests: disabled-tools alias regression (qualified entry blocks bare name and vice versa, never reaching the MCP manager), ToolPolicy alias regression, python/bash title="..." non-execution + display retention, and inline JSON-array args still parsing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): reject brace-style fence metadata; cover the full email set in the friendly toggle Review follow-up round 3 on odysseus-dev#3681 (thanks @RaresKeY): 1. Brace-style fence metadata no longer executes. The previous narrowing still treated any same-line {/[ after a recognized tag as tool input, so ```bash {title="setup"} ran as a bash call. The fence header is now captured separately and judged by one predicate shared between parse_tool_blocks and strip_tool_blocks (_fenced_tool_call), so the execute and display decisions can't disagree: same-line content only counts as inline args when the tag is NOT a code tag (bash/python never take same-line args — that text is Markdown fence attributes) AND the inline text (plus any continuation lines) parses as standalone JSON. ```bash {title="setup"}, ```python {"title":"example.py"} and ```list_emails {title="x"} all stay visible and inert. 2. The friendly `disable_tool email` toggle covered 3 of the 14 email tools (mcp__email__{list_emails,read_email,send_email}); the other bare aliases this PR routes stayed executable after an operator disabled email. The alias now derives from BUILTIN_EMAIL_TOOLS in BOTH spellings — bare (function-schema hiding, bare-fence dispatch) and mcp__email__* (MCP schema hiding, qualified runtime blocks) — so the toggle and the runtime gate can't drift apart. Tests: brace/bracket metadata regressions for parse and strip symmetry (code tags, invalid-JSON inline on a JSON tool, multi-line inline JSON still parsing), and disable_tool/enable_tool email covering all 14 names in both spellings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(email): close remaining email-tool registry drift; classify every email tool for plan mode Deep self-review follow-up on odysseus-dev#3681. Three review rounds each found another hand-maintained copy of the email tool list that had drifted; this commit hunts down ALL remaining copies and pins them to BUILTIN_EMAIL_TOOLS. The same 5 tools (search_emails, draft_email, draft_email_reply, ai_draft_email_reply, download_attachment) were missing from every advertising surface, so they were dispatchable but never offered: - FUNCTION_TOOL_SCHEMAS: native function-calling models never saw them (the round-1 fix covered dispatch only); schemas added, mirroring the email server's inputSchema definitions. - TOOL_SECTIONS: fenced-block models were never told about them; prompt sections added. - tool_index: absent from the RAG embedding registry (never retrievable), the email keyword hints, and the scheduled assistant's always-available set — the latter two now derive from BUILTIN_EMAIL_TOOLS. - agent_loop._DOMAIN_TOOL_MAP["email"], tool_policy._COMMON_TOOL_NAMES, the assistant tool-selector UI groups (assistant.js), and the default Assistant crew seed (task_scheduler) now derive from / cover the set. Plan mode now classifies every email tool explicitly: - list_email_accounts and search_emails join PLAN_MODE_READONLY_TOOLS. Without this, list_email_accounts sat in the plan-mode bare denylist (schema-derived) while its qualified form passed the MCP read-only filter — and the round-2 bare/qualified alias gate would have blocked the qualified call too, regressing read-only email discovery in plan mode. - draft_email, draft_email_reply, ai_draft_email_reply, and download_attachment join the fail-closed mutator backstop (drafts create documents; download_attachment writes to disk). Tests: tests/test_email_registry_sync.py pins every registry (including the email server source and assistant.js) to BUILTIN_EMAIL_TOOLS and asserts the plan-mode partition, so the next email tool can't drift; a parse/strip mirror grid covers 192 fence shapes (tag x header x body) asserting executed <=> stripped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: move the email alias rule into tool_security; extract the assistant seed constant Code-quality pass over the PR's own changes: - The bare<->qualified email aliasing rule lived inline in the generic dispatcher (_execute_tool_block_impl). It is policy knowledge, so it moves next to BUILTIN_EMAIL_TOOLS as email_tool_policy_names(); the dispatcher just consumes it, and the rule gets its own unit test (including the mcp__email__<not-a-tool> and mcp__other__ non-alias cases). - The default Assistant's enabled_tools list was an inline literal inside the CrewMember seed, and its registry-sync test asserted a source-code substring. Extracted to DEFAULT_ASSISTANT_ENABLED_TOOLS so the test imports and checks the actual value. - _fenced_tool_call return type tightened to Optional[Tuple[str, str]]. No behavior change; suite green (3295 passed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * revert: move the email registry consolidation to a follow-up PR Per review feedback on scope, this PR stays narrow: fenced inline-args parsing, bare email tool routing, and the directly required safety gates. This commit reverts the registry/advertising consolidation from db29046 and 016ce47 (native schemas, prompt sections, RAG description index + keyword hints, assistant always-available set, guide-only known-names union, frontend tool-selector groups, default assistant seed, and their sync tests) — all of that moves to a dedicated follow-up PR together with the _EMAIL_TOOL_HINTS finding. Kept here because the narrow scope needs them: - email_tool_policy_names() in tool_security + its use in the execute_tool_block gates and its unit test (refactor of this PR's own round-2 alias fix), - list_email_accounts in PLAN_MODE_READONLY_TOOLS (the alias gate works both ways, and the schema-derived plan-mode bare denylist would otherwise block the qualified read-only call too), - the parse/strip mirror grid test (parser scope), - the narrow registry sync tests (email server <-> BUILTIN_EMAIL_TOOLS match, fence-tag coverage, non-admin blocklist coverage). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(email): execute empty email fences with empty args; reject non-object JSON args Two gaps found by replaying captured local-model traffic against the narrowed branch: 1. ```list_email_accounts``` with NO body — a shape gemma really emits for no-arg tools — was silently dropped (parse skips empty content), so the model concluded email was broken: the original odysseus-dev#337 symptom through a different door. Empty fences whose tag is a built-in email tool now dispatch with {} args and the tool's own validation answers (e.g. an empty send_email returns "to is required" instead of silence). Empty bash/python/other fences keep skipping, and strip stays mirrored (the fence was executed, so it is removed). 2. The fence parser accepts JSON arrays as inline args, but the email dispatch parsed only objects — an array silently became {} args. Non-object JSON now returns a correctable "arguments must be a JSON object" error before reaching the MCP server (same class as odysseus-dev#3966). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(security): classify all email tools for plan mode statically; reject invalid email JSON bodies Review follow-up round 5 on odysseus-dev#3681 (thanks @RaresKeY): 1. This PR makes every BUILTIN_EMAIL_TOOLS name fence-taggable, so each one must be explicitly classified for plan mode — the draft tools and download_attachment were in neither the read-only allowlist nor the static denylist, leaving their bare-alias plan-mode safety dependent on the MCP read-only inventory being present and current. search_emails joins PLAN_MODE_READONLY_TOOLS (explicit, not allowed-by-omission); draft_email, draft_email_reply, ai_draft_email_reply, and download_attachment join the fail-closed _PLAN_MODE_KNOWN_MUTATORS backstop. (Moved back from the odysseus-dev#4053 split: the partition is directly required for this PR to merge independently.) 2. The classic tag/body fence form reaches execution unvalidated (only INLINE args are JSON-checked by the parser), so a body like {account: "work"} silently became {} args and read the DEFAULT mailbox instead of the intended one. JSON-looking bodies that fail to parse now return a correctable "not valid JSON" error before reaching the MCP server. Tests: a partition invariant (every email tool is explicitly read-only or plan-mode-denied), a mutating-alias probe that uses only the static denylist with a fake MCP manager (no inventory layer), and the body-form invalid-JSON regression. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tool-dispatch): decode inline JSON args for legacy MCP tools; reject all non-object email bodies Review follow-up round 6 on odysseus-dev#3681 (thanks @RaresKeY) — both pre-existing on this branch, surfaced by the relaxed inline-args parser: 1. The relaxed parser accepts inline JSON for every non-code tag, but the legacy line-based arg builders (web_search/web_fetch/read_file/ write_file/generate_image/manage_memory) wrapped the whole JSON string as the query/url/path/prompt — so `web_search {"query": "x"}` executed as a search for the literal string `{"query": "x"}`. _build_mcp_args now uses a fenced JSON object directly when it carries the tool's primary arg key (query/url/path/prompt/action). Keyed off membership so it can't drift; an object without the primary key (e.g. a freeform JSON query, or bare object content for write_file) falls through to the line parser unchanged. Also fixes the same corruption for the classic newline-JSON form. 2. The bare-email dispatch only rejected bodies starting with { or [, so a non-empty non-JSON body like `account: work` still fell through to {} args and silently read the DEFAULT mailbox. Now ANY non-empty body must decode to a JSON object or it returns a correctable error; only a truly empty body keeps the no-arg path (```list_email_accounts```). Tests: inline-JSON arg decoding for the five legacy tools plus the freeform and missing-primary-key fallbacks; the email body rejection extended to cover the brace-looking and bare `key: value` shapes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tool-dispatch): drop dead manage_memory JSON-decode entry; pin the live-path invariant Self-audit catch on the round-6 fix. manage_memory was added to _MCP_JSON_PRIMARY_KEYS, but _build_mcp_args is only reached via _call_mcp_tool, which only runs for _MCP_TOOL_MAP tools — and manage_memory isn't one (its tag routes through dispatch_ai_tool -> do_manage_memory, which line-parses). So the round-6 decode for manage_memory was dead code: the unit test exercising _build_mcp_args passed while a real `manage_memory {"action": ...}` fence still parsed the whole JSON blob as the action. Remove the dead entry and add test_mcp_json_primary_keys_are_all_live, which asserts every JSON-primary tool is in _MCP_TOOL_MAP so a dead decode can't be added again. The same inline-JSON corruption for manage_memory and the other tools that route through positional dispatchers (create_session, ui_control, send_to_session, search_chats, the document tools, etc.) is pre-existing (dev corrupts their newline JSON form too) and tracked separately; the proper fix there is to route fenced JSON through function_call_to_tool_block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tool-dispatch): decode inline JSON in WriteFileTool (its live path); round-6 fix was on the dead MCP path Self-audit: round 6 claimed to fix inline JSON args for write_file via _build_mcp_args, but there is no filesystem MCP server, so write_file always runs through _direct_fallback -> WriteFileTool, never through _build_mcp_args. WriteFileTool — unlike its siblings ReadFileTool / WebSearchTool / WebFetchTool, which all decode JSON — took lines[0] as the path, so `write_file {"path": "/tmp/x", "content": "y"}` wrote to a file literally named with the JSON blob. The round-6 _build_mcp_args entry decoded correctly but on a path that never executes (same class as the manage_memory dead entry), and the round-6 unit test passed on that dead path. WriteFileTool now decodes a JSON object carrying "path" (matching ReadFileTool directly above it), and the comment on _MCP_JSON_PRIMARY_KEYS records that only generate_image has a live MCP server today — the other entries are defense-in-depth for the MCP path; the live fix for each server-less tool is in its handler. Test: test_write_file_inline_json_args drives the LIVE path (execute_tool_block with no MCP) and asserts the intended path is used — verified to fail without the handler fix. web_search/web_fetch/read_file were already correct (their handlers decode); write_file was the gap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(strip-fence): derive the live-strip TOOL_TAGS from the real set Semantic conflict from the dev merge that textual auto-merge didn't flag: dev added test_live_strip_email_tool_fences.py whose _tool_tags() helper source-scrapes only the TOOL_TAGS literal `{...}`, which worked on dev because the email tool names were listed inline there. This branch makes TOOL_TAGS the single source — `{...} | BUILTIN_EMAIL_TOOLS` — so the email names are no longer in the literal and the scraper missed them, leaving the email-fence strip assertions failing even though TOOL_TAGS does contain them at runtime. Import the real TOOL_TAGS instead of scraping source, so the test mirrors exactly what GET /api/tools serves (sorted(TOOL_TAGS)) and the live EXEC_FENCE_RE derives from — robust to however the set is composed. The source-level frontend/route guards in the same file are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: botinate <285686135+botinate@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
… resolve code quality items (#1) * Code quality: security, deps, error handling, tests, frontend, database Phase 0 — Security (5/5): - Fix bare except in require_admin() (middleware.py) - Replace shell=True with list-form subprocess (builtin_actions.py) - Add .catch() to 3 dynamic imports (app.js) - Encrypt Webhook.secret with EncryptedText - Verify XSS safety in chat.js Phase 1 — Deps/Config (7/7): - Pin all 28 deps; move test deps to requirements-dev.txt - Set asyncio_loop_scope = module, trim CORS whitespace - Pin Docker to python:3.13-slim - Replace __import__ with standard imports Phase 2 — Error Handling (3/3): - Create safe_create_task() helper; fix 13 fire-and-forget tasks - Lock _response_cache with threading.Lock() - Narrow bare excepts in subprocess_tools, builtin_actions, agent_loop, task_scheduler (24 instances fixed) Phase 3 — Tests (2/2): - Fix test_session_manager explicit assertion - Remove src.database MagicMock fallback in conftest.py Phase 4 — Frontend (5/5): - Scope MutationObservers to #app; add beforeunload cleanup - Create shared API_BASE module (14 files updated) - Fix admin.js undefined API_BASE bug - Fix chatRenderer.js window.API_BASE fallback - Fix _defaultChat race, rename guard in app.js Phase 5 — Database (3/3): - Defer init_db() from module import to app startup - Add .limit() safety caps to 11 unbounded queries - Extract 26 ORM models to core/orm_models.py (709 lines) - Add index=True to 6 FK columns Phase 6 — Cross-cutting (3/4): - Add Docker socket security warning - Fix dead-host cache: move _clear_host_dead after status check - Cache EncryptedText import to avoid per-call overhead - Add -> None to note_model_activity * making ulises multilang * finished multilang base & tests * fix(security): scope owner-less email accounts to a mailbox match in route guards (#5238) The HTTP email route guard `_assert_owns_account` and the explicit-account_id path in `_get_email_config` gated cross-tenant access with `if row.owner and row.owner != owner` -- which skips the check entirely when the account row is owner-less (owner NULL or ""). `email_accounts` is the one owner-scoped table left out of the legacy-owner migration backfill (core/database.py), so such rows persist on multi-user deploys: an account configured while auth was disabled, or an imported legacy row. Any authenticated user could then pass that account's id to read/send/update-credentials/delete another tenant's mailbox and read its decrypted IMAP/SMTP creds. Both sibling paths already enforce the intended contract -- the same-file `_owner_or_matching_legacy_account` fallback and the MCP `_account_visible_to_owner` gate (whose comment says it mirrors "the HTTP email route fallback") only expose an owner-less account when its own mailbox (imap_user / from_address) is the caller's. Factor that row-level predicate into `_account_visible_to_owner` and use it in both guards, so owner-less accounts are visible only on a mailbox match. Owned accounts, the legacy-claim path, and single-user mode (owner == "") are unchanged. Complements #5234 (which fixes the same class on the MCP tool layer); this is the HTTP route layer it does not touch. * fix(agent): fall back to keyword tool selection when retrieval times out The retrieval-timeout branch hard-coded ALWAYS_AVAILABLE, silently skipping the deterministic keyword hints whenever the embedding backend was slow (e.g. a remote endpoint cold-loading its model). Queries that named email or calendar outright lost those tools and the model concluded the integrations did not exist. Let the timeout fall through to the existing keyword fallback instead — same baseline, plus the hints. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(search): pin httpx connection to resolved IP to block DNS rebinding (#704) * fix(search): pin DNS-validated fetch connections Rebase the DNS-rebinding SSRF fix onto current dev after search content moved behind the services.search.content canonical module. Integrate the pinned httpcore NetworkBackend/BaseTransport approach with the current size-capped Client.stream fetch path, preserving Host/SNI semantics while forcing TCP connect to the already validated resolved IP. Keep src.search.content as the compatibility wrapper and preserve existing OG-image http(s) behavior; this avoids reintroducing the unrelated scope changes that previously blocked review. Add the explicit httpcore>=1.0,<2.0 requirement used by the public httpcore NetworkBackend and ConnectionPool APIs. * test(search): restore and rebase DNS rebinding regressions Keep the current security regression coverage that the stale PR branch had deleted, including auth-disabled localhost bypass and Ollama cookbook hardening tests. Carry forward the DNS-rebinding coverage for private resolve blocking, pinned TCP connect behavior, Host header preservation, redirect revalidation, and the BaseTransport/public-httpcore static guard. Update redirect tests to mock the current Client.stream-based capped fetch path rather than the older httpx.stream/get path. * test(search): adapt size-cap fetch tests to pinned client stream The DNS-rebinding repair moved _get_public_url from the module-level httpx.stream shortcut to httpx.Client(...).stream(...) so the fetch can use the pinned transport. Keep the existing size-cap test fakes by routing Client.stream through the monkeypatched httpx.stream only when a test has installed that fake; otherwise fall back to a real Client. This fixes the CI failures in tests/test_web_fetch_size_caps.py without touching unrelated upload-handler atomicity behavior, which is already flaky on clean origin/dev. --------- Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com> * fix: resolve RAG manager search signature TypeError (#4994) * fix: resolve RAG manager search signature TypeError and adjust similarity threshold * fix: revert similarity threshold change to keep PR focused * test(rag): remove trailing whitespace --------- Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com> * fix(security): scope send_to_session to an exact session owner * fix(tools): handle non-dict JSON values in _parse_tool_args (closes #5043) (#5064) * fix(search): extract non-ASCII capitalized names in _extract_entities * fix(agent): skip deny-listed sensitive files in glob (#5094) * fix(security): match the sensitive-file deny-list case-insensitively (#5097) * fix(integrations): don't append a trailing slash when api_call path is '/' * fix(calendar): honor list_events date range aliases (#3283) * fix: auto-spam move/delete targets the wrong message (seqnum vs UID) (#1874) * fix(session): use utcnow_naive across session routes (#1116) (#5003) * fix(cookbook): stop Ollama runner from executing the install one-liner (#3926) * fix(security): apply the webhook SSRF guard to the reminder ntfy sender (#5142) * fix(security): pin webhook delivery to the SSRF-validated IP (DNS rebinding) (#5147) * fix(security): validate integration api_call URLs with the outbound SSRF guard (#5145) * fix(security): match grep's rg sensitive-file exclusions case-insensitively (#5189) * Fix Cookbook download runner for Python 3.11 * fix: add grace period to document tidy to prevent deleting new documents (#5036) * fix(email): enforce MCP account owner scope (#5234) * fix(security): prevent ReDoS in LLM-output tool/think parsers (#4704) * fix(tasks): normalize task endpoint URL to /chat/completions before model call (#4619) * fix(security): prevent ReDoS in XML and args tool-call parsers (#4941) * fix(chat): sanitize web search query to strip markdown and code blocks (#4863) * fix(security): batch A — 7 security fixes * fix(bugfix): batch B — 16 critical bugfixes * fix(chat): batch C — web search denial + explicit intent * test(ci): batch D — test splits + CI guidance * feat(refactor): batch E — features, refactors, catalog * fix(infra): batch F — 16 infrastructure + bugfixes * feat(docs): batch G — models, documents, docs updates * fix(ux): batch H — email, gallery, cookbook UX fixes * feat(models): batch I — models hardening + OCR captions * ui: batch J1 — cookbook/llama model UI spacing * ui: batch J2 — cookbook engine tint, MTP, vllm spacing * feat(background): batch J3 — background tasks, CLI sanitization, info-budget fixes * chore(release): bump version to 1.0.2 * fix(ui): lower cookbook serve top row (cherry-pick fc6a6dc) * fix(chat): honor explicit web search denial * fix(mobile): stack the model-comparison grid into one column on phones (#4979) The comparison grid hard-codes 2-4 equal columns with no phone breakpoint, so at 390px two models get ~178px columns and four get ~88px columns. Each column is a full scrolling chat, so content is unreadably over-wrapped and clipped. On phones (<=768px), stack the panes into a single scrollable column. Desktop is unaffected. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(docs): correct broken backup-restore link in setup.md Fixes #4926 - the link used docs/backup-restore.md from within docs/setup.md, which resolved to docs/docs/backup-restore.md (404). Changed to same-directory relative path. * docs(setup): add a self-host troubleshooting cookbook of common traps (#4834) ROADMAP "Self-host troubleshooting cookbook" asks to document the weird 30-second fixes that otherwise become 30-minute searches. Adds a "Common self-host traps" subsection under Troubleshooting covering: the UTF-8 BOM .env gotcha (app.py loads with utf-8-sig), macOS AirPlay holding port 7000 (the start script uses 7860), the plain-HTTP Tailscale/LAN clipboard limitation, self-hosted ntfy delivery (NTFY_BIND/NTFY_BASE_URL + the ntfy Android Instant-delivery toggle), Dovecot cleartext-auth on LAN mail stacks, and Radicale full-collection-URL sync. Docs only; grounded in existing repo behavior (.env.example NTFY_* block, app.py utf-8-sig loader, start-macos.sh port choice). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tasks): normalize task endpoint URL to /chat/completions before model call (#4619) Upstream bug (present in pewdiepie-archdaemon/odysseus main): the task executor passes task.endpoint_url VERBATIM to the model HTTP call, unlike the chat path which stores build_chat_url(normalize_base(base)) on the session. A task carrying an explicit bare OpenAI-compatible base such as "http://host:11434/v1" therefore POSTs to a 404 ("page not found"); the agent loop swallows the empty body into "The model returned an empty response" and marks the run success, so nothing surfaces the failure. Tasks that omit an endpoint dodge this only because _resolve_defaults() cribs an already-full URL from a recent chat session. The API/token path (e.g. an external client that POSTs /api/tasks with endpoint_url=".../v1") hits it every time. Fix: route every resolved task endpoint through _normalize_chat_endpoint() at the three resolution sites (_execute_llm_task, the persona/research session path, and _execute_research_task). The helper is idempotent (strips any existing chat suffix, re-appends the correct one) and leaves native-Ollama (/api...) and already-concrete URLs untouched, so other providers are unaffected. Proven via isolated repro: ".../v1" -> 404 -> empty; ".../v1/chat/completions" -> 200 -> real gemma4:31b output. Regression test asserts the bare-/v1 -> full-chat-URL mapping, idempotency, and the native-Ollama/empty passthroughs. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(setup): load .env so a pre-seeded admin password is honored on native installs (#4787) setup.py read ODYSSEUS_ADMIN_USER / ODYSSEUS_ADMIN_PASSWORD via os.getenv() but never loaded .env, so on native Linux/macOS installs a password pre-seeded in .env (documented in docs/setup.md and .env.example) was silently ignored and a random one generated, breaking the first login. Docker was unaffected because compose passes the vars into the container env. Call load_dotenv(BASE_DIR/.env, encoding="utf-8-sig") at the top of main(), mirroring app.py (utf-8-sig tolerates a Notepad UTF-8 BOM). load_dotenv does not override already-exported OS vars, so the existing precedence is kept. python-dotenv is already a required dependency. Adds a regression test that pre-seeds credentials only in .env (not the shell) and asserts the stored bcrypt hash matches the pre-seeded password. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(llm): detect mistral.ai provider and support reasoning_effort (#4698) * fix(llm): detect mistral.ai provider and support reasoning_effort Four coupled bugs broke Mistral thinking model support: 1. _detect_provider() had no mistral.ai host check, so all Mistral endpoints fell through to the generic 'openai' provider string. _provider_display_name() correctly identified them as 'Mistral', making any 'if provider == "Mistral"' check elsewhere dead code. 2. reasoning_effort parameter was never sent in the request payload, so Mistral never activated thinking mode even when the user configured a thinking-capable model (mistral-small-latest, mistral-medium-latest, magistral-*). 3. Mistral returns content as a typed array ([{"type":"thinking",...},{"type":"text",...}]) when reasoning is on, not as a plain string. Both the streaming and non-streaming parsers expected strings and silently dropped the thinking content. 4. _THINKING_MODEL_PATTERNS didn't include magistral or mistral-* model prefixes, so the frontend wouldn't tag reasoning output as thinking even after the above were fixed. Fix: - Add mistral.ai to _detect_provider() host checks - Add a _normalize_mistral_content() helper that splits the typed array into (text, thinking) strings - Inject payload["reasoning_effort"] = "high" when provider is Mistral and _supports_thinking(model) is true, in both stream_llm and llm_call_async payload construction - Wire the normalizer into both response parsers - Extend _THINKING_MODEL_PATTERNS to include magistral, mistral-small, mistral-medium, mistral-large Tested on Docker install with mistral-small-latest + reasoning_effort=high. Reasoning streams correctly into the thinking panel after the fix. Fixes #4678 * fix(llm): address review — lowercase provider id, configurable effort, tests Addresses vdmkenny's review on PR #4698: 1. Removed duplicate 'if provider == "mistral"' block in stream_llm — two back-to-back copies, one was dead-redundant. 2. Dropped personal-context comment ('free-tier limits are generous for this user') and made reasoning_effort configurable via env var ODYSSEUS_MISTRAL_REASONING_EFFORT (high / medium / low / none). Default remains 'high' for backward compat with the tested behavior. 3. Recased provider id from 'Mistral' to 'mistral' to match the lowercase convention used by every other provider id in the file (openai, anthropic, ollama, copilot, ...). _provider_display_name() still returns the Title-Case 'Mistral' for UI labels — only the runtime id used in 'if provider == ...' checks was recased. 4. Added tests/test_llm_core_mistral_content.py with 13 tests pinning _normalize_mistral_content()'s contract: string passthrough, the Mistral array format (thinking + text blocks), and edge cases (empty, garbage, None, wrong types, missing fields, string-vs-array inner thinking field). Also fixed a gap the review didn't catch: the non-streaming paths (llm_call sync + llm_call_async) were missing the reasoning_effort injection entirely. Added the same injection to both, so Deep Research and agent tool calls also activate Mistral thinking. All 13 new tests pass. Existing reasoning/streaming/ollama-thinking tests still pass (38 tests, no regressions). Fixes #4678 * fix(chat): restore missing _explicit_web_intent definition (#5290) chat_stream() references `_explicit_web_intent` in three places (disabled-tools gating, global-disabled web allowance, and the per-turn tool filter) but the assignment was dropped during a branch merge. Every chat request raised NameError: name '_explicit_web_intent' is not defined at routes/chat_routes.py, surfacing to the client as a bare "Internal Server Error" before any LLM call was made — chat was fully broken on dev and main. Restore the original definition, computed from the already-derived tool intent, immediately before its first use: _explicit_web_intent = bool(_tool_intent and _tool_intent.category == "web") Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 2d8177035b6a1dd3e9211c6cb5ba68443b826504) * Parse local function_model tool wrappers * Show fallback model in picker * Nudge serve engine control up * Move core serve memory fields farther left * Move vllm block size left again * Nudge vllm attention field right * Raise unified llama context estimate * fix(llm-core): prevent cache-affinity fields from reaching Cerebras Recognize api.cerebras.ai as a Cerebras cloud provider so llama.cpp/LM Studio cache-affinity fields are not attached even when endpoint_kind is misconfigured as local. Add regression coverage for provider detection, self-hosted classification, and payload field exclusion. * fix(models): accept bare-list /models responses (Together AI) (#4761) * fix(api): handle varying response formats for model IDs from compatible providers merge conflict for pr-2204 resolved * fix(modal): keep body-portaled dropdowns above their tool modal at any stack depth (#4720) (#4724) * fix(memory): keep the Brain memory item menu above the modal at any stack depth The memory item "⋮" dropdown is portaled to <body> with a hardcoded z-index of 10001. Tool modals, however, get a monotonically increasing z-index from modalManager's bring-to-front counter (_modalTopZ), which climbs unbounded as modals are opened/restored over a session. Once that counter passes 10001, the Brain modal stacks above the body-portaled dropdown, so the menu renders behind the panel — visible only where it spills past the modal's edge (#4720). Derive the dropdown's z-index from the owning modal's current z-index (+1), keeping 10001 as a floor for the common low-counter case, so the menu always sits just above its modal however high the counter has climbed. Verified with document.elementFromPoint at the dropdown's location: with a high modal z-index the old build returns the modal at every sampled point (menu behind); the fixed build returns the dropdown (menu on top). The default low-counter case is unchanged (z stays 10001). * refactor(modal): route body-portaled dropdowns through a shared topPortalZ() helper The hardcoded z-index:10001 the Brain memory menu used (#4720) is the same literal shared by ~16 body-portaled dropdowns across calendar, cookbook, cookbookServe, documentLibrary, emailLibrary, gallery, notes, emojiPicker and memory — each renders behind its owning tool modal once modalManager's bring-to-front counter climbs past the literal over a long session. Promote the per-dropdown fix into a single topPortalZ() helper in toolWindowZOrder.js — the existing source of truth for tool-window z, already imported by modalManager's _bringToFront and notes.js — returning max(topToolWindowZ(), dock-chip floor) + 1, so a portaled dropdown always sits just above the live tool-window stack however high the counter has climbed. Route all 16 sites through it. The slashCommands tour tooltips and the cookbookServe VRAM dialog are intentionally left out (neither is a modal-owned portaled dropdown). Add tests/test_portal_dropdown_z_js.py covering the helper, including the #4720 scenario (modal counter at 99999 -> dropdown at 100000). Existing test_notes_z_order_js.py stays green. * fix(llm): detect mistral.ai provider and support reasoning_effort (#4698) * fix(llm): detect mistral.ai provider and support reasoning_effort Four coupled bugs broke Mistral thinking model support: 1. _detect_provider() had no mistral.ai host check, so all Mistral endpoints fell through to the generic 'openai' provider string. _provider_display_name() correctly identified them as 'Mistral', making any 'if provider == "Mistral"' check elsewhere dead code. 2. reasoning_effort parameter was never sent in the request payload, so Mistral never activated thinking mode even when the user configured a thinking-capable model (mistral-small-latest, mistral-medium-latest, magistral-*). 3. Mistral returns content as a typed array ([{"type":"thinking",...},{"type":"text",...}]) when reasoning is on, not as a plain string. Both the streaming and non-streaming parsers expected strings and silently dropped the thinking content. 4. _THINKING_MODEL_PATTERNS didn't include magistral or mistral-* model prefixes, so the frontend wouldn't tag reasoning output as thinking even after the above were fixed. Fix: - Add mistral.ai to _detect_provider() host checks - Add a _normalize_mistral_content() helper that splits the typed array into (text, thinking) strings - Inject payload["reasoning_effort"] = "high" when provider is Mistral and _supports_thinking(model) is true, in both stream_llm and llm_call_async payload construction - Wire the normalizer into both response parsers - Extend _THINKING_MODEL_PATTERNS to include magistral, mistral-small, mistral-medium, mistral-large Tested on Docker install with mistral-small-latest + reasoning_effort=high. Reasoning streams correctly into the thinking panel after the fix. Fixes #4678 * fix(llm): address review — lowercase provider id, configurable effort, tests Addresses vdmkenny's review on PR #4698: 1. Removed duplicate 'if provider == "mistral"' block in stream_llm — two back-to-back copies, one was dead-redundant. 2. Dropped personal-context comment ('free-tier limits are generous for this user') and made reasoning_effort configurable via env var ODYSSEUS_MISTRAL_REASONING_EFFORT (high / medium / low / none). Default remains 'high' for backward compat with the tested behavior. 3. Recased provider id from 'Mistral' to 'mistral' to match the lowercase convention used by every other provider id in the file (openai, anthropic, ollama, copilot, ...). _provider_display_name() still returns the Title-Case 'Mistral' for UI labels — only the runtime id used in 'if provider == ...' checks was recased. 4. Added tests/test_llm_core_mistral_content.py with 13 tests pinning _normalize_mistral_content()'s contract: string passthrough, the Mistral array format (thinking + text blocks), and edge cases (empty, garbage, None, wrong types, missing fields, string-vs-array inner thinking field). Also fixed a gap the review didn't catch: the non-streaming paths (llm_call sync + llm_call_async) were missing the reasoning_effort injection entirely. Added the same injection to both, so Deep Research and agent tool calls also activate Mistral thinking. All 13 new tests pass. Existing reasoning/streaming/ollama-thinking tests still pass (38 tests, no regressions). Fixes #4678 * fix: Images cannot be seen by model that is vision capable (#4726) * fix: Images cannot be seen by model that is vision capable * fix: skip http(s) image_url for Ollama (images[] is base64-only) --------- Co-authored-by: michaelxer <michaelxer@users.noreply.github.com> * fix(chat): strip executed email tool fences from the live stream (#3993) (#4275) * fix(chat): strip executed email tool fences from the live stream (#3993) The backend strips every fenced tool block from persisted text (the regex in src/tool_parsing.py is built from the full TOOL_TAGS set, which includes the email tools), so a reloaded session renders cleanly. The live frontend path uses a separate hardcoded EXEC_FENCE_RE in static/js/chatRenderer.js that only listed web_search/read_file/write_file/create_document/edit_document/ update_document — so executed email tool fences (list_emails, etc.) lingered as raw code blocks in the live assistant bubble until the user reloaded. Add the nine email tool tags to EXEC_FENCE_RE so the live render settles into the same clean layout as the history reload. bash/python stay excluded on purpose: those are languages a user may legitimately have asked the model to show as code, not tool invocations. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(chat): single-source live exec-fence tool list from TOOL_TAGS (#3993) Per review: EXEC_FENCE_RE was a second, hand-maintained copy of the executable-tool list, so any tool not in it — and every future tool added to TOOL_TAGS — would leave its executed fence lingering in the live bubble until reload (the original #3993 bug, recurring one tool at a time). EXEC_FENCE_RE is now built from an explicit EXEC_TOOL_TAGS list that mirrors TOOL_TAGS (src/agent_tools/__init__.py) minus bash/python, which stay excluded as legitimate code-example languages. A new regression test (test_exec_fence_re_covers_all_executable_tools) extracts both lists from source and fails if they drift, so the whole class is caught in CI instead of by a user — the "minimum acceptable middle ground" from the review, made exact (set equality, not just coverage). Verified: pytest tests/test_live_strip_email_tool_fences.py (5 passed); node --check static/js/chatRenderer.js; and a node run of the built regex confirms email/generate_image/manage_memory/ls fences strip while bash/python/sh are preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(chat): build live exec-fence list from /api/tools at runtime (#3993) Make TOOL_TAGS the single source for live exec-fence stripping. chatRenderer.js no longer hard-codes a tool list; it fetches the backend's authoritative set once from GET /api/tools (sorted(TOOL_TAGS)) and builds EXEC_FENCE_RE from it at load, minus bash/python. No second list to drift, and a future tool added to TOOL_TAGS is covered automatically — without touching the streaming path. Until the fetch resolves EXEC_FENCE_RE is null and exec fences aren't stripped (a sub-second window before the first stream); the backend already strips persisted history, so a reload always renders clean. Drop test_exec_fence_re_covers_all_executable_tools (no hand-maintained list to guard) and add source-level guards: the frontend keeps no hard-coded list and fetches /api/tools, and the endpoint serves the full sorted(TOOL_TAGS). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CVCKth4g8pWh7pwFDVm4iL * fix(chat): warn on /api/tools fetch failure instead of swallowing it (#3993) A fresh-context review flagged that loadExecFenceRegex's catch silently discarded errors: if the one-shot fetch fails, EXEC_FENCE_RE stays null for the whole session and live exec fences go unstripped until reload, with zero signal. console.warn it, and correct the comment to describe the failure mode honestly (was understated as just a sub-second startup window). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CVCKth4g8pWh7pwFDVm4iL --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(routes): log and cleanly 500 on unreadable HTML page (#4637) * fix(routes): serve 404 instead of 500 when an HTML page file is missing _serve_html_with_nonce opened the HTML file with no error handling, and callers such as /backgrounds and /login pass their paths in with no existence check, so a missing or unreadable file raised an unhandled OSError that surfaced as a 500. Wrap the read and raise HTTPException(404) instead; the normal render path (CSP-nonce substitution) is unchanged. Fixes #4594 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(routes): distinguish missing page (404) from read failure (500) The previous fix caught a broad OSError and returned 404 for every failure, which masks real server-side problems (permission errors, I/O failures) as "not found" and lets them slip past error alerting. Split FileNotFoundError (genuine 404) from other OSError, which now logs the exception and returns a generic 500 — without leaking the OS error string or file path into the response body. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(routes): treat unreadable bundled HTML page as logged 500, not 404 Per PR #4637 review: every caller of the page-render helper serves a fixed, server-owned template (index/login/backgrounds), never a client-supplied path. So a missing or unreadable file is a server fault (broken deployment), not a client "not found" — a 404 there mislabels a server error and hides a missing core template from 5xx alerting, contradicting the OSError->500 rationale this PR is built on. Collapse both branches into a single logged, leak-free 500. Move the helper to src.app_helpers.serve_html_with_nonce so the behavior can be unit-tested without importing the whole app (app.py is the slim orchestrator; the test harness stubs src.database, so importing app in tests is not viable). Add tests pinning missing/unreadable -> 500 (not 404) and nonce injection on the happy path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(catalog): add Gemma 4 12B/QAT entries and RTX 3050 bandwidth (#4728) Add official Gemma 4 12B-it plus QAT-INT4/INT8 catalog entries (with their GGUF sources), QAT quantization support across the quant tables and the prequantized-prefix list, and the missing RTX 3050 / 3050 Ti memory bandwidth so speed estimates stop falling back to the generic cuda value. * fix debugging on windows (#4679) * fix: Real-ESRGAN install + Cookbook deps-panel crash on the Python 3.14 image (#4694) * fix(docker): make Real-ESRGAN installable on the Python 3.14 image realesrgan's deps basicsr/gfpgan/facexlib (unmaintained since 2022) read their version in setup.py via `exec(...); locals()['__version__']`, which raises KeyError on Python 3.13+ — PEP 667 made locals() in a function an independent snapshot that exec() can no longer mutate. That fails the Cookbook "install realesrgan" sdist build on the python:3.14 base. Add a `realesrgan-wheels` builder stage that fetches the pinned sdists, patches get_version() to exec into an explicit namespace dict, and builds wheels; the final stage installs them --no-deps so a later `pip install realesrgan` resolves from wheels instead of rebuilding the broken sdists. torch stays a runtime pull to keep the base image lean. Also add the runtime libs opencv-python (cv2) needs — libgl1, libglib2.0-0t64, libxcb1 — which the slim base omits; without them the install succeeds but `import cv2` dies with `libxcb.so.1: cannot open shared object file`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cookbook): don't let a package's sys.exit() on import hang the deps panel The local optional-dependency probe imports each package in-process and catches ImportError / Exception. But a package can call sys.exit() at import time — e.g. rembg does `sys.exit(1)` when no onnxruntime backend loads. SystemExit is a BaseException, not Exception, so it escaped the probe, propagated out of the list_packages endpoint, and hung the whole Dependencies panel / worker (the UI loads forever). Catch (Exception, SystemExit) so one broken optional package is reported as not-usable instead of taking down the panel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(routes): 500 (not 404) when the app-shell index.html is missing (#4791) Follow-up to #4637. serve_index — the handler for / and the SPA deep-link routes (/notes, /calendar, /cookbook, /email, /memory, /gallery, /tasks, /library) — pre-checked os.path.exists and raised its own HTTPException(404, "index.html not found") when the bundle was missing. So a missing core template returned 404 before serve_html_with_nonce's 500 could fire, the one inconsistency left after #4637. index.html is a fixed, app-bundled template; a missing one is a broken deployment (server fault), not a client "not found", so it should surface as a logged 500 in 5xx alerting rather than a 404. Keep the static->root fallback, drop the redundant existence guard and the dead-end 404, and let the shared helper handle the missing case. Verified against the running app: / and /notes return 200 with the bundle present and a logged 500 when index.html is absent. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(setup): load .env so a pre-seeded admin password is honored on native installs (#4787) setup.py read ODYSSEUS_ADMIN_USER / ODYSSEUS_ADMIN_PASSWORD via os.getenv() but never loaded .env, so on native Linux/macOS installs a password pre-seeded in .env (documented in docs/setup.md and .env.example) was silently ignored and a random one generated, breaking the first login. Docker was unaffected because compose passes the vars into the container env. Call load_dotenv(BASE_DIR/.env, encoding="utf-8-sig") at the top of main(), mirroring app.py (utf-8-sig tolerates a Notepad UTF-8 BOM). load_dotenv does not override already-exported OS vars, so the existing precedence is kept. python-dotenv is already a required dependency. Adds a regression test that pre-seeds credentials only in .env (not the shell) and asserts the stored bcrypt hash matches the pre-seeded password. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: email poller marks calendar extraction processed on LLM failure (#4622) Move calendar processed-marker insert into the LLM success path (else branch). Previously, the INSERT ran even after a transient LLM failure, causing the poller to skip retrying calendar extraction on subsequent runs. Minimal change: only touches the try/except/else control flow in _auto_summarize_pass_single() — preserves existing formatting and line endings. * feat(ui): add toggle for padding around chat area (#4691) * feat: Allow admins to choose if they want to share defaults (#4752) * First bare fix * Adding the option toggle * toggle function fix * Final fix, added missing /auth/ * Extended toggle text & added tests * Comments change * Description toggle change * br tag fix * description change based on suggestion * fix(agent): parse misfenced read_file calls (#4799) * fix: use atomic write in APIKeyManager.save() to prevent credential data loss (#4591) (#4597) * fix: use atomic write in APIKeyManager.save() to prevent data loss Opening api_keys.json with 'w' truncates the file before writing, so a crash, disk-full, or mid-write error leaves all stored provider API keys corrupted. Switch to atomic write (temp file + fsync + os.replace) so the original file is always intact on any failure. Fixes #4591 * chore: trigger CI re-run * chore: update PR description * chore: fix how-to-test section for description check --------- Co-authored-by: michaelxer <michaelxer@users.noreply.github.com> * feat(discovery): detect llama.cpp servers and label local providers (#4729) * feat(discovery): detect llama.cpp servers and label local providers Scan port 8080 (llama-server) and 11435 (APFEL) during discovery, fingerprint llama.cpp via its native /props endpoint, and label well-known local serving ports (8080 llama.cpp, 8000 vLLM, 1234 LM Studio, 11434 Ollama) consistently in both the Python provider helper and the JS endpoint UI. Adds a llama.cpp hint to the /setup slash command. * fix(discovery): don't infer the serving tool from the port alone Per review: vLLM, SGLang, llama.cpp and plain OpenAI-compatible servers all share 8000/8080, so labeling by port mislabels real setups (a vLLM box on 8080 shown as llama.cpp). Drop the port->tool assertions from _provider_label and providerLabel; the authoritative signal is the /props fingerprint done during discovery, which is unchanged. Loopback now reads a neutral 'local endpoint' / 'Local'. Tests updated to assert the neutral labels. * refactor(tools): migrate config/integration admin tools to the registry (#4742) Part of #3629 (the `admin_tools.py` bullet). Moves the config/integration admin tools off the legacy elif dispatch chain in tool_implementations.py onto the agent_tools registry: manage_endpoints, manage_mcp, manage_webhooks, manage_tokens, manage_settings The do_* implementations (and manage_mcp's command-allowlist / RCE guard: _validate_mcp_command, _mcp_allowed_commands, and the _MCP_* constants) move verbatim into the new src/agent_tools/admin_tools.py. They register through a single ADMIN_TOOL_HANDLERS map that TOOL_HANDLERS.update()s, and the five elif branches plus their imports are dropped from tool_execution.py, so these tools now flow through _direct_fallback like the other migrated clusters. The names are re-exported from src.agent_tools for back-compat. Dedup: - _parse_tool_args was duplicated in tool_implementations.py and document_tools.py. It now lives once in src.tool_utils (which imports nothing from the project beyond src.constants, so this introduces no cycle) and both call sites import it from there. The orphaned `import json` in document_tools is removed with it. - The five tools share one _owner_adapter(fn) factory that threads ctx["owner"] into the owner-taking do_* signature, instead of five near-identical wrappers. Tests: new tests/test_admin_tools_registry.py pins the registration, the re-export back-compat, the owner-threading adapter, and the single-source _parse_tool_args (across admin_tools and document_tools). Existing MCP / settings / webhook suites are repointed at the new module. * refactor(exceptions): dedupe src/exceptions via core re-export (#4785) src/exceptions.py was a byte-for-byte duplicate of the canonical core/exceptions.py. Replace its class bodies with a re-export shim (mirroring the core/constants.py -> src/constants.py pattern) so the exception classes are defined in exactly one place. Also fix the stale "# src/exceptions.py" header comment in core/exceptions.py. No behavior change: both import paths resolve to the same class objects (verified by identity), so `except SessionNotFoundError` works regardless of which module it was imported from. Ran py_compile and pytest tests/test_app.py (12 passed). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tasks): normalize task endpoint URL to /chat/completions before model call (#4619) Upstream bug (present in pewdiepie-archdaemon/odysseus main): the task executor passes task.endpoint_url VERBATIM to the model HTTP call, unlike the chat path which stores build_chat_url(normalize_base(base)) on the session. A task carrying an explicit bare OpenAI-compatible base such as "http://host:11434/v1" therefore POSTs to a 404 ("page not found"); the agent loop swallows the empty body into "The model returned an empty response" and marks the run success, so nothing surfaces the failure. Tasks that omit an endpoint dodge this only because _resolve_defaults() cribs an already-full URL from a recent chat session. The API/token path (e.g. an external client that POSTs /api/tasks with endpoint_url=".../v1") hits it every time. Fix: route every resolved task endpoint through _normalize_chat_endpoint() at the three resolution sites (_execute_llm_task, the persona/research session path, and _execute_research_task). The helper is idempotent (strips any existing chat suffix, re-appends the correct one) and leaves native-Ollama (/api...) and already-concrete URLs untouched, so other providers are unaffected. Proven via isolated repro: ".../v1" -> 404 -> empty; ".../v1/chat/completions" -> 200 -> real gemma4:31b output. Regression test asserts the bare-/v1 -> full-chat-URL mapping, idempotency, and the native-Ollama/empty passthroughs. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(model-routes): harden _probe_endpoint against malformed model-list responses (#4789) * fix(model-routes): harden _probe_endpoint against malformed model-list responses _probe_endpoint parsed model lists with data.get(...) at four sites without checking that data is a dict, and built the list with a truthiness-only filter. A /models (or /api/tags) endpoint returning HTTP 200 with valid but non-dict JSON ([], "x", null, 123) made data.get(...) raise AttributeError, and a non-string id like 123 passed the filter and then hit .startswith() / .lower() in the Z.AI/Kimi curated merge and _is_chat_model(). Both errors are swallowed by the broad except Exception, but the comprehension dies mid-list so the ENTIRE probed model list is discarded and the endpoint silently degrades — masking a misconfigured/non-compliant upstream as "no models". - Guard each data.get(...) with isinstance(data, dict) so a non-dict body falls through the existing `or []` default. - Restrict the OpenAI and Ollama model-list comprehensions to non-empty str values, protecting the .startswith() merges and both _is_chat_model calls. - Add an isinstance guard at the top of _is_chat_model (defense in depth for all four call sites). No behavior change for well-formed {"data":[...]} / {"models":[...]} responses. Adds regression tests (non-dict body via caplog, mixed/all non-string ids, _is_chat_model boundary) that fail before the fix and pass after. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(model-routes): extract _openai_model_ids / _ollama_model_names helpers Per review on #4789: the malformed-response guards were inlined four times in _probe_endpoint (two OpenAI-id comprehensions, two Ollama-name comprehensions). Pull each into a small, directly-testable helper so the security-relevant parsing lives in one place and a future malformed-shape fix doesn't have to be applied in four spots (CONTRIBUTING flags repeated logic for this reason). Behavior is unchanged. Adds direct unit tests for both helpers (non-dict body, non-string ids, non-dict entries, name>model precedence). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cookbook): only block model launch on real port collisions (#4760) * Fix #4507: only block model launch on real port collisions Quick-run hardcoded port 8000 and never called _nextAvailablePort(), so every launch collided. Both pre-launch guards (serve panel + quick-run) were count-based and fired regardless of port. - quick-run now auto-assigns a free port (8080 for llama.cpp) - both guards parse the new port and only prompt on a real overlap, stopping only the colliding serve - dialog reports the actual port instead of a hardcoded 8000 * refactor(cookbook): share _taskPort for port parsing; auto-assign llama.cpp port Addresses review on #4760: - _taskPort regex now matches --port= as well as --port (space) - _nextAvailablePort and both launch guards reuse _taskPort instead of inline regex - quick-run llama.cpp no longer pins 8080, so two can run concurrently * fix(cookbook): _taskPort also parses -p; add port-parsing tests Addresses review on #4760: - _taskPort now matches -p <n> too, so it's the complete single reader (was missing the short flag that other readers already handle) - add tests/test_cookbook_port_parsing_js.py covering the port forms, shared-reader reuse, and llama.cpp auto-assign * test(cookbook): extract pure port helpers and test behavior Addresses review on #4760: the prior tests only asserted source strings. - extract portOf() and nextFreePort() into static/js/cookbookPorts.js - cookbookRunning.js imports them; _taskPort and _nextAvailablePort delegate - tests run the helpers via node and assert real behavior: all port forms (--port, --port=, -p, -p=), next-free-port skipping taken ports, and the same-port-clash / different-port-coexist outcome --------- Co-authored-by: samy <samy@odysseus.boukouro.com> * fix(ui): route tasks.js + skills.js dropdowns through topPortalZ() (#4768) Fixes #4767. #4724 routed 16 body-portaled dropdowns through the shared topPortalZ() helper so they always render just above the currently-raised tool modal, but two were missed and still used a hardcoded z-index, so they hit the same #4720 bug once a modal's bring-to-front counter climbed past the literal: - tasks.js _showTaskDropdown(): inline z-index:100000 on .task-dropdown - skills.js kebab menu (.skill-kebab-menu): z-index:100002 in style.css Both now set zIndex from topPortalZ() after they are appended to the body, matching the other migrated sites. The dead CSS z-index on .skill-kebab-menu is removed (the inline value always wins). test_portal_dropdown_z_js.py gains a source guard asserting both files use topPortalZ() and that no hardcoded 100000/100002 portal literal survives in either file or style.css. * do_list_models in ai_interaction.py dropped --------- Co-authored-by: Max Hsu <maxmilian@users.noreply.github.com> Co-authored-by: aubrey <kyuhex@gmail.com> Co-authored-by: Michael <52305679+michaelxer@users.noreply.github.com> Co-authored-by: michaelxer <michaelxer@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ahmed Dlshad <ahmed.dlshad.m@gmail.com> Co-authored-by: Joel Alejandro Escareño Fernández <52678667+TheAlexz@users.noreply.github.com> Co-authored-by: Kalin Stoyanov <kgs.void@gmail.com> Co-authored-by: Pedro Barbosa <devpedrobarbosa@gmail.com> Co-authored-by: Solanki Sumit <125974181+YAMRAJ13y@users.noreply.github.com> Co-authored-by: Rudra Sarker <78224940+rudra496@users.noreply.github.com> Co-authored-by: Skoh <101289702+SkohTV@users.noreply.github.com> Co-authored-by: Jakub Grula <ramsters110@gmail.com> Co-authored-by: Dividesbyzer0 <54127744+zoomdbz@users.noreply.github.com> Co-authored-by: Kenny Van de Maele <kenny@kvandemaele.be> Co-authored-by: Magiomakes <114195802+Magiomakes@users.noreply.github.com> Co-authored-by: Samy <12219635+touzenesmy@users.noreply.github.com> Co-authored-by: samy <samy@odysseus.boukouro.com> * fix(upload): remove trailing whitespace * fix(security): redact credential-bearing URLs and PII from logs (#4750) * fix(security): redact credential-bearing URLs and PII from logs Several log statements emitted sensitive data in clear text: - model_routes / chat_routes / contacts_routes logged endpoint URLs raw. Admin-configured URLs can embed credentials in userinfo or query (e.g. https://user:pass@host, ?api_key=...). Route them through a shared core.log_safety.redact_url() that drops userinfo/query/fragment. - note_routes / task_scheduler logged operator email addresses (smtp_user, recipient). Replaced with presence booleans, which keeps the diagnostic ("why didn't this send") without writing PII to logs. model_routes already had a local redactor on its HTTPStatusError branch; the generic except branch was missed, so reuse the existing helper there. Clears CodeQL py/clear-text-logging-sensitive-data alerts 264, 317, 324, 325, 343, 344, 528. * fix(security): re-bracket IPv6 hosts and single-source the URL redactor Address review on #4750: - redact_url now re-brackets IPv6 literals so host:port stays unambiguous (https://[2001:db8::1]:8443/v1, not the bracket-less ambiguous form). - point model_routes._redact_url_for_log at the shared helper so the two redactors are single-sourced (also picks up the IPv6 fix). * fix(chat): honor explicit web search denial * fix(chat): require explicit web search enable (cherry picked from commit 54f1d015b578832abbf608e4d367097a1b3def0e) * fix(tasks): gate cookbook serve task execution (#5235) * fix(security): scope owner-less email accounts to a mailbox match in route guards (#5238) The HTTP email route guard `_assert_owns_account` and the explicit-account_id path in `_get_email_config` gated cross-tenant access with `if row.owner and row.owner != owner` -- which skips the check entirely when the account row is owner-less (owner NULL or ""). `email_accounts` is the one owner-scoped table left out of the legacy-owner migration backfill (core/database.py), so such rows persist on multi-user deploys: an account configured while auth was disabled, or an imported legacy row. Any authenticated user could then pass that account's id to read/send/update-credentials/delete another tenant's mailbox and read its decrypted IMAP/SMTP creds. Both sibling paths already enforce the intended contract -- the same-file `_owner_or_matching_legacy_account` fallback and the MCP `_account_visible_to_owner` gate (whose comment says it mirrors "the HTTP email route fallback") only expose an owner-less account when its own mailbox (imap_user / from_address) is the caller's. Factor that row-level predicate into `_account_visible_to_owner` and use it in both guards, so owner-less accounts are visible only on a mailbox match. Owned accounts, the legacy-claim path, and single-user mode (owner == "") are unchanged. Complements #5234 (which fixes the same class on the MCP tool layer); this is the HTTP route layer it does not touch. * refactor(routes): move contacts domain into routes/contacts/ subpackage (#5227) Slice 2e of the route-domain reorganization (#4082/#4071, per specs/architecture-runtime-inventory.md §6.3). Moves contacts_routes.py into routes/contacts/, leaving a backward-compat sys.modules shim at the old path. Pure file reorganization, no behavior change. The shim uses sys.modules replacement (same pattern as the merged gallery `import routes.contacts_routes`, `from routes.contacts_routes import X`, `importlib.import_module(...)`, the string-targeted `monkeypatch.setattr("routes.contacts_routes.SETTINGS_FILE", ...)` used by test_carddav_password_encryption.py, and the `import ... as cr` + `setattr(cr, ...)` pattern in test_contacts_add_null_name.py all operate on the same module object the application uses. This also keeps the mutable module state `_contact_cache` identical across import paths. The canonical module does NOT depend on the shim — routes/contacts/ contacts_routes.py imports only from core/, src/, and stdlib (zero internal routes/ coupling). The inbound edge from routes/email_helpers.py (imports _fetch_contacts) keeps working through the shim. Zero source-introspection landmines — no test reads this file by path. Adds tests/test_contacts_routes_shim.py to pin the sys.modules shim contract (same-object + string-targeted monkeypatch reach-through). Verified: compileall clean; full suite 4485 passed, 3 skipped. * fix(security): pin webhook delivery to the SSRF-validated IP (DNS rebinding) (#5147) validate_webhook_url resolves the host to accept/reject, but the delivery connect (httpx.AsyncClient.post) re-resolved independently — a DNS record flipping between the two lookups (rebinding) could slip an internal IP (127.0.0.1 / 169.254.169.254 / LAN) past the check and receive the signed payload. The module docstring already flagged this as only a "partial defense". Resolve + validate once via _validated_public_ips, then pin the delivery TCP connect to that approved IP with an async _PinnedAsyncTransport built on the public httpcore/httpx APIs (mirrors the sync search-fetch pin from validation and vhost routing still target the original hostname; only the socket destination is pinned. Delivery now uses a per-request pinned client instead of one shared client, so close() is a no-op kept for API compatibility. Adds end-to-end tests that drive the real transport against loopback servers, proving the connect follows the pin rather than re-resolving the URL host. Fixes #5146 Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * fix(security): apply sensitive-file deny-list to grep tool (#5011) (#5013) The grep tool bypassed the sensitive-file deny-list that read_file, write_file, and edit_file all respect. Two code paths fixed: 1. ripgrep path: adds --glob exclusion patterns for each entry in _SENSITIVE_FILE_PATTERNS (id_rsa, known_hosts, authorized_keys, etc.) 2. Pure-Python os.walk fallback: checks _is_sensitive_path() before opening each file, skipping files that match the deny-list Fixes #5011 Co-authored-by: michaelxer <michaelxer@users.noreply.github.com> * fix(chat): sanitize web search query to strip markdown and code blocks (#4863) Layer a defensive cleanup on top of the generated-query web-search flow so the final selected query is sanitized before reaching comprehensive_web_search. - remove fenced code blocks from the final search query - preserve inline code as plain text - collapse whitespace and cap query length - cover generated-query success plus LLM failure/empty fallback paths Partially addresses #4547. * fix(calendar): honor list_events date range aliases (#3283) * fix(calendar): honor list_events date range aliases * fix(calendar): reject partially resolved loose range queries * fix(integrations): don't append a trailing slash when api_call path is '/' _join_integration_url built urljoin(base + '/', '') for a bare '/' path — the minimum execute_api_call accepts — so every request against a POST-to-base integration went to base_url + '/'. Discord webhook URLs 404 ('Unknown Webhook') on the trailing-slash variant, which made the integration look broken even though the stored base URL was correct. Resolve a bare '/' (or empty) path to the base URL itself and keep all other paths joining exactly as before, including deliberate trailing slashes inside non-empty paths (linkding /api/tags/, Home Assistant /api/). The reminder webhook sender and the discord_webhook connectivity test already posted to the bare base URL; execute_api_call was the remaining path that re-added the slash. Fixes #5138 * test: split service health tests (#4972) * test: split service health tests * test(service-health): preserve focus selector * fix(tools): handle non-dict JSON values in _parse_tool_args (closes #5043) (#5064) When an LLM generates a valid JSON string that parses to a native non-dict type (like a list, int, or string), _parse_tool_args previously returned that object. Callers expecting a dictionary would then crash with AttributeError or KeyError when attempting to look up action keys. - Update _parse_tool_args in src/tool_utils.py to explicitly type-check the parsed JSON object and return {} for non-dict objects. - Add test coverage in tests/test_admin_tools_registry.py for lists, ints, and strings. * test(integrations): drop redundant trailing-slash assertion The exact-equality assert on the line above (requested_url == WEBHOOK_BASE) already implies the URL has no trailing slash, so the endswith check adds nothing. * Move core serve memory fields further left * Move vllm attention farther right * Color cookbook context fit notes * fix(parser): parse Gemma 3/4 custom tool calling tokens (#5033) * fix: parse Gemma 3/4 custom tool calling tokens in parser * test: cover Gemma tool call parsing --------- Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com> * fix(agent): preserve bare email tool parity (#5075) * fix(agent): confine glob literal lookups to the search root (#5010) GlobTool resolves its search root through _resolve_search_root (which confines it to the workspace or default allowlist), but the literal fast-path joined the model-supplied pattern onto that root without re-confining it. os.path.join lets an absolute pattern or one containing ../ escape the root, and normpath collapsed the .. segments, so glob returned the absolute path of arbitrary host files once they existed -- an existence/path oracle that bypasses the confinement read_file, write_file, grep, and ls all enforce. Keep the literal lookup inside the root via a commonpath containment check; an escaping literal falls through to the os.walk matcher, which only ever yields paths under the root. Wildcard matching was already confined. * fix(agent): skip deny-listed sensitive files in glob (#5094) * fix(llm): normalize OpenAI-compatible chat URLs Normalize OpenAI-compatible chat URL shapes so base /v1 endpoints route to /v1/chat/completions while already-full chat endpoints remain idempotent. Preserve native local Ollama routing for bare localhost:11434 endpoints, keep localhost:11434/v1 as OpenAI-compatible, and add focused regression coverage for provider detection, chat target URLs, and model listing from /v1. Part of #541. * fix(search): use generated query for chat mode web search #4547 (#4557) * fix(search): use generated query for chat mode web search #4547 * style(search): tidy query generation call --------- Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com> * fix(email): validate IMAP/SMTP ports instead of crashing with 500 (#4464) The email-account endpoints coerced user-supplied ports with a bare int(data.get("imap_port") or 993), so a non-numeric port (e.g. "imap") raised ValueError and surfaced as an HTTP 500 in the create, update, and test-config endpoints. Add a _coerce_port(value, default) -> (port, error) helper and use it in all three endpoints, returning the endpoints standard {"ok": False, "error": ...} response (matching the existing "name required" validation) instead of crashing. A blank or missing port still falls back to the default (993/465). * fix(security): prevent ReDoS in LLM-output tool/think parsers (#4704) * fix(security): prevent ReDoS in LLM-output tool/think parsers The regexes that parse untrusted model output in text_helpers.py and tool_parsing.py are delimiter-bounded with a lazy [\s\S]*? (or an ambiguous (\s+[^>]*)?). Applied with re.sub/re.finditer over a whole response, they degrade to O(n^2) when the closing delimiter is absent: the engine rescans to end-of-string from every opener. Model output is untrusted, so a prompt-injected or malicious model can stall the agent loop with many unclosed openers (measured ~25s on a 60KB <thought flood). - text_helpers.py: replace ambiguous <thought(\s+[^>]*)?> with <thought([^>]*)> (identical capture, no \s+/[^>]* overlap); skip the Gemma <|channel>...<channel|> subs when no <channel|> closer is present. - tool_parsing.py: gate _TOOL_CALL_RE, _XML_TOOL_CALL_RE and _TOOL_CODE_RE (in parse_tool_blocks and strip_tool_blocks) on a cheap presence check for their closing delimiter. With no closer the regex cannot match, so skipping is equivalent; only the wasted O(n^2) rescan is removed. Resolves CodeQL py/polynomial-redos #230, #231, #232, #233, #235, #236, (its greedy [\s\S]*\Z is linear) and left untouched. * fix(security): close ReDoS gaps in tool/think parsers from review Addresses two review findings on the closer-guard approach: - Whole-string "closer exists?" checks were bypassable: a stale closer before an opener flood, or a closer with no reachable inner `}`, kept the guard true while every opener still rescanned to end-of-string (O(n^2)). Replace the substring guards with `_iter_delimited`, a forward-only scan that pairs each opener with a *later* closer and stops once none is reachable (O(n)). `parse_tool_blocks` and `strip_tool_blocks` (via `_strip_delimited`) both use it for the [TOOL_CALL], <tool_call>/<function_call>, and <tool_code> formats. Verified equivalent to the original regexes on well-formed inputs. - `<thought([^>]*)>` dropped the tag-name boundary and corrupted unrelated tags (`<thoughtful>` -> `<thinkful>`). Use `<thought(\s[^>]*)?>`: the single fixed `\s` keeps the pattern linear (no `\s+`/`[^>]*` overlap) while restoring the boundary; capture is byte-for-byte identical for real `<thought ...>` openers. Adds regressions for stale-closer-before-opener, closer-present-without- inner-brace, and the <thoughtful>/<thoughts> passthrough. * fix(security): close Gemma channel ReDoS guard flagged in review vdmkenny noted the same bypassable whole-string guard remained in text_helpers.py: `if "<channel|>" in out.lower()` gating the Gemma thought/response channel subs. A stale `<channel|>` before a `<|channel>thought` opener flood keeps the guard true while every opener still rescans to end-of-string (measured ~7.3s at 4k openers). Replace it with `_sub_delimited`, the same forward-only scan used for the tool-call parsers: pair each opener with a later closer, stop when none is reachable (O(n)). Verified output-equivalent to the original capture regexes on well-formed multi-channel inputs; the stale-closer case now runs in <2ms. Adds a regression for stale-closer-before-opener on the Gemma path. * fix(security): harden strip_think() think-tag ReDoS flagged in review The earlier fixes hardened normalize_thinking_markup and the delimiter scanners, but the production entrypoint strip_think() still ran _THINK_CLOSED_RE / _THINK_ATTR_RE / _THINK_OPEN_RE (and the stray-tag _THINK_TAG_RE) over untrusted model output. Those kept the same ReDoS shapes: the lazy `<open>[\s\S]*?</close>` rescanned to end-of-string from every opener, and `(?:\s+[^>]*)?` / `[^>]*` attribute scans ran to end-of-string from every opener on a "many openers, no closer" flood. On the prior head, malformed `<think` / `<thinking` / `<thought` floods took 6-14s through strip_think(). The shipped `<thought>` normalization had the same residual: the single-opener case was linear but an opener flood was still O(n^2) (~4.4s). - Replace the lazy multi-pass _THINK_CLOSED_RE loop with the existing forward-only _sub_delimited scan (pair each opener with the first reachable closer, stop when none is reachable). One pass collapses sequential and nested blocks as before. - Bound every opener/stray-tag attribute scan at `<` (`[^<>]` not `[^>]`) so a no-`>` opener flood can't drive a single match attempt to end-of-string. Identical capture for well-formed think/thought tags. - email_helpers._strip_think: compute had_think from the single linear _THINK_TAG_RE instead of the lazy closed/open `.search()` calls, which had the same O(n^2) on the email reply/summary/extraction paths. All flood variants now finish in <10ms (were 6-14s). Output verified byte-for-byte identical to the prior implementation over a 34-case corpus (nested, mismatched, attr, uppercase, Gemma, prose, prompt-echo). Adds strip_think() timing regressions for malformed openers, opener floods (all three tag names), the closed-opener flood, and the malformed-closer flood. * docs: trim verbose comments in think-tag ReDoS fix * Support mobile enter for queued agent prompts * Keep open document context for section edits * Add bulk email attachment downloads * Add hover labels to mini sidebar buttons * fix(cookbook): treat local Windows as Windows for serve commands (#3975) * fix(cookbook): prefer native llama-server on local Windows * fix(cookbook): harden local llama-server launch commands * fix(cookbook): build serve commands for selected target * fix(cookbook): only block model launch on real port collisions (#4760) * Fix #4507: only block model launch on real port collisions Quick-run hardcoded port 8000 and never called _nextAvailablePort(), so every launch collided. Both pre-launch guards (serve panel + quick-run) were count-based and fired regardless of port. - quick-run now auto-assigns a free port (8080 for llama.cpp) - both guards parse the new port and only prompt on a real overlap, stopping only the colliding serve - dialog reports the actual port instead of a hardcoded 8000 * refactor(cookbook): share _taskPort for port parsing; auto-assign llama.cpp port Addresses review on #4760: - _taskPort regex now matches --port= as well as --port (space) - _nextAvailablePort and both launch guards reuse _taskPort instead of inline regex - quick-run llama.cpp no longer pins 8080, so two can run concurrently * fix(cookbook): _taskPort also parses -p; add port-parsing tests Addresses review on #4760: - _taskPort now matches -p <n> too, so it's the complete single reader (was missing the short flag that other readers already handle) - add tests/test_cookbook_port_parsing_js.py covering the port forms, shared-reader reuse, and llama.cpp auto-assign * test(cookbook): extract pure port helpers and test behavior Addresses review on #4760: the prior tests only asserted source strings. - extract portOf() and nextFreePort() into static/js/cookbookPorts.js - cookbookRunning.js imports them; _taskPort and _nextAvailablePort delegate - tests run the helpers via node and assert real behavior: all port forms (--port, --port=, -p, -p=), next-free-port skipping taken ports, and the same-port-clash / different-port-coexist outcome --------- Co-authored-by: samy <samy@odysseus.boukouro.com> * fix(agent): execute fenced tool calls with inline args and route bare email tool names (#3681) * fix(agent): execute fenced tool calls with inline args and bare email tool names Two bugs made local (Ollama) models unable to use email tools, leaving raw fences like ```list_email_accounts {}``` in the chat: 1. _TOOL_BLOCK_RE required a newline right after the fence tag, so a tool call with args on the same line ("```list_email_accounts {}") …
Summary
Local models that use the fenced-block tool channel (Ollama, gemma, etc.) could not call any of the email tools in chat/agent mode. The model emits
```list_email_accounts {}, nothing executes, the raw fence leaks into the chat as text, and the model concludes it has no email access. Two bugs caused this: (1) the fence regex insrc/tool_parsing.pyrequired a newline directly after the tool name, so a call with arguments on the same line never matched; (2) even when a fence did parse, bare email tool names had no dispatch branch insrc/tool_execution.pyand fell through to "Unknown tool type".Review follow-up (thanks @vgalin): routing bare email names had a security gap —
is_public_blocked_tool()checks the bare name before dispatch, andNON_ADMIN_BLOCKED_TOOLSlisted only 4 of the 14 email tools, so non-admin owners could have executed the other 10 (delete_email,bulk_email, …). The full email tool set is now defined once (BUILTIN_EMAIL_TOOLSinsrc/tool_security.py) and the non-admin blocklist, fence tags, bare-name dispatch, and native-call mapping all derive from it, so the sets can't drift apart. This also fixed the native-call mapping for 4 tools (search_emails,draft_email,draft_email_reply,ai_draft_email_reply) missing from the oldtool_schemas.pycopy; advertising them to models (schemas, prompt sections, RAG index) is part of the follow-up #4053. Additionally, the relaxed fence regex from the first commit could prefix-match longer fence tags (```python3would have executed3\nprint(...)as apythontool call — the same false-positive class as #3604); a(?![\w-])boundary now prevents that.Second review round (thanks @RaresKeY): two more gaps fixed. (1) The runtime denylist (
disabled_tools) and the turnToolPolicywere checked against the incoming block name only, while plan mode and the MCP settings toggle write the qualifiedmcp__email__...name — a barelist_emailsfence bypassed a qualified block entry. Both gates now match bare and qualified spellings, in either direction. (2) The relaxed fence regex accepted any same-line text after a recognized tag, which made Markdown info strings executable (python title="example.py"ran as apythontool call). Same-line content now only counts as tool input when it starts with{/[(JSON args); anything else stays display text, with parse/strip mirrored.Third round (thanks @RaresKeY): (1) the same-line rule is now tag-aware and JSON-validated through one predicate shared by parse and strip —
bash/pythonnever take same-line args, other tags require valid standalone JSON — so brace-shaped fence attributes likebash {title="setup"}stay visible and inert. (2) The friendlydisable_tool emailtoggle derives fromBUILTIN_EMAIL_TOOLSin both bare andmcp__email__*spellings. Scope: per review, the wider registry/advertising consolidation has been moved out to #4053 (reverted here inee35896); this PR keeps only the parser fix, the bare-email routing, and the directly required safety gates. Final hardening from corpus replay: empty email fences (list_email_accountswith no body) now dispatch with empty args instead of vanishing, and non-object JSON args return a correctable error.Target branch
dev, notmain.Linked Issue
Part of #337 — fixes the email-in-chat failure reported there (UI test button works, but the model claims no email access). The calendar half of that issue needs retesting after this lands; the regex fix applies to every fenced tool with inline args (including
manage_calendar), but that is unconfirmed.Also related: #2447 (this fixes the fenced inline-args variant; the bare unfenced variant is still open), #1789 (possibly the same dispatch gap for user-added MCP servers), #3141 (closed; same symptom surface on Windows/Ollama).
Type of Change
Checklist
devdocker compose uporuvicorn app:app) and verified the change works end-to-end. (Manual UAT on macOS with a local Ollama gemma model — results in the comment below. One failing scenario (chat-phrase email kill-switch) decomposes entirely into pre-existing defects outside this PR's files, tracked separately.)How to Test
python -m pytest tests/test_fenced_inline_args.py tests/test_review_regressions.py -q— covers inline-args parsing, the```python3/hyphenated-tag non-matches, parse/strip display mirroring, and non-admin blocking of all 14 bare email names plus themcp__email__...form.uvicorn app:app), configure an email account (Settings → Email; confirm the test button lists your inbox).```list_email_accounts {}fence and no account data. After: the tool executes and the model answers with the configured account(s).list_emails) to confirm coverage beyond the accounts tool.BLOCKED("restricted to admin users"), bare ormcp__email__-qualified.Visual / UI changes — REQUIRED if you touched anything that renders
N/A — no UI/rendering changes; only backend tool parsing/dispatch/policy (
src/tool_parsing.py,src/tool_execution.py,src/tool_security.py,src/tool_schemas.py,src/agent_tools/__init__.py) and tests.This PR was written via Claude and verified by a human.
🤖 Generated with Claude Code