From 1684c9380a8725ccb07c93cad4b532d47919ddf9 Mon Sep 17 00:00:00 2001 From: Nick Launces <1409277+nicklaunches@users.noreply.github.com> Date: Tue, 1 Sep 2026 22:08:02 -0700 Subject: [PATCH] Give a Bot the deployment's mailbox through four governed tools A Bot could reach Drive, Notion and a dozen custom MCP servers, and could not read the mail sent to the deployment it runs in. Mailbox is a new catalogue entry at /admin/plugins/mailbox with four tools over IMAP and SMTP: list_messages, read_message, search_messages and send_message, the last of which threads a reply from the uid of the message it answers and files a copy of what it sent in the account's Sent folder, since SMTP delivers a message and does not file one. The connector is a builtin because there is nothing to authenticate to a vendor for. It runs in-process, on hosts and accounts the deployment configures with MAILBOX_IMAP_HOST, MAILBOX_SMTP_HOST and MAILBOX_USERS, which are needed together so a half-configured mailbox is refused at boot rather than at the first login in front of somebody. MAILBOX_USERS is a list, because a shared host is how support@, sales@ and billing@ usually arrive; the first is the default account a call that named none works in. The passwords are not environment variables: each is a vault credential of kind mcp, provider mailbox, key id the address, read at the moment a call needs it so a rotation lands on the next call rather than the next restart. That access model is why CatalogueAuth.builtin now carries reachedAs. Routines touches the asking person's own rows and is reached as them; the mailbox belongs to the deployment and is opened on a password the deployment holds, so a row naming the asker would put a person's id on access that was never theirs. The transport asks reachedAsFor for the value and the audit row gets it. Everything else is the path every connector already takes. The grant is checked per tool, the policy is evaluated with the tool's effect (send_message as a write, the other three as reads), and the audit row is written, before any mail server is dialled. MAILBOX_ALLOWED_RECIPIENT_DOMAINS bounds recipients on top of that, because a policy rule sees a tool call's name and effect and never its arguments, and a Bot holding the read tools and an unconstrained send_message can be talked into mailing the inbox out by an email addressed to it. The folder and account guards are what live runs taught. A smaller model given both arguments put an address in the folder one, was answered "Character not allowed in mailbox name", and retried with the local part. So the argument is named folder and says it is neither an address nor half of one; a configured address there is adopted as the account with a note rather than refused, an unconfigured one and a local part are refused with what to do instead, and a folder that is genuinely missing is answered with the folders that do exist. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01WaHWJ1niprhBc5NzJ9pxme --- .env.example | 40 + CHANGELOG.md | 42 + bun.lock | 85 +- docs/README.md | 1 + docs/configuration.md | 7 + docs/mailbox.md | 240 +++ server/package.json | 5 + server/src/config.ts | 205 ++ server/src/index.ts | 46 + server/src/mailbox/client.ts | 866 ++++++++ server/src/plugins/builtin-mailbox.ts | 1048 ++++++++++ server/src/plugins/catalogue.ts | 48 +- server/src/plugins/store.ts | 21 +- server/src/plugins/transport.ts | 22 +- server/tests/builtin-mailbox.test.ts | 1819 +++++++++++++++++ server/tests/config.test.ts | 175 ++ server/tests/plugin-catalogue.test.ts | 74 +- .../plugin-reached-as.integration.test.ts | 171 ++ 18 files changed, 4890 insertions(+), 25 deletions(-) create mode 100644 docs/mailbox.md create mode 100644 server/src/mailbox/client.ts create mode 100644 server/src/plugins/builtin-mailbox.ts create mode 100644 server/tests/builtin-mailbox.test.ts create mode 100644 server/tests/plugin-reached-as.integration.test.ts diff --git a/.env.example b/.env.example index 201282873..b4debe2a7 100644 --- a/.env.example +++ b/.env.example @@ -330,3 +330,43 @@ AGENT_TOOL_TOKEN= # Do not accept a default in production. WORKER_SHARED_SECRET= +# The deployment's mailbox, which a Bot reads and answers through the Mailbox connector. All three +# are needed together: set one and the server refuses to start naming the others, rather than booting +# with half a mailbox that fails at the first login in front of somebody. Leave all three unset and +# the connector is still there, still grantable, and every call answers with what to set. +# +# MAILBOX_USERS is a comma-separated list of addresses, all on the hosts above, which is the shape a +# shared host gives you: support@, sales@ and billing@ are three mailboxes on one IMAP and one SMTP +# server. The FIRST is the default, the one a tool call that named no account works in, so the order +# is a decision rather than a detail. One address is a perfectly good list. +# +# MAILBOX_USER, the singular this feature shipped with, is still read as a list of one, so an +# existing deployment keeps working unedited. Setting both refuses to start: they are two answers to +# the same question. +# +# THE PASSWORDS ARE NOT HERE, and that is deliberate. They are the only secrets in this feature, so +# they live where this deployment's other secrets live: the encrypted vault. Store one per account +# at /admin/credentials with kind `mcp`, provider `mailbox` and key id the address itself, and +# rotate them there. A password in this file is a password in a repository, a compose file and every +# process list that ever read it. +MAILBOX_IMAP_HOST= +MAILBOX_SMTP_HOST= +MAILBOX_USERS= +# MAILBOX_USERS=support@example.com,sales@example.com +# +# Both default to the implicit-TLS ports, 993 and 465, so the connection is encrypted before the +# password is sent rather than negotiating for it in the clear. Set them only for a server that +# listens elsewhere. +# MAILBOX_IMAP_PORT=993 +# MAILBOX_SMTP_PORT=465 +# +# Where a Bot may send mail, as a comma-separated list of domains. Unset or empty means anywhere, +# which is the default and the behaviour every deployment had before this existed. +# +# Worth setting, because the policy engine cannot do this job: a rule sees a tool call's name and +# effect, never its arguments, so the only rule you can write about send_message covers all of it or +# none of it. Meanwhile the reading tools bring text somebody else wrote into a model's context, so +# a Bot that can read and send without limits is one persuasive message away from mailing the inbox +# to whoever asked for it. This bounds where anything can go; an approval rule on send_message in +# your boundaries decides whether it goes at all. Deployments that care should have both. +# MAILBOX_ALLOWED_RECIPIENT_DOMAINS=example.com,partner.example diff --git a/CHANGELOG.md b/CHANGELOG.md index d446e0438..a830b7a62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,48 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged. ## Unreleased +### A Bot can read and answer the deployment's own mailbox + +A new catalogue entry, Mailbox, appears at `/admin/plugins/mailbox`. It gives a Bot four tools +against the deployment's own mail: `list_messages`, `read_message`, `search_messages` and +`send_message`, the last of which threads a reply when it is given the uid of the message being +answered, and files a copy of what it sent in the account's Sent folder so webmail and the Bot both +find the outgoing mail. The mailbox belongs to the deployment rather than to whoever is asking, so +everybody granted the tools reads the same mail. Decide that before granting it, because it is the +whole of the access model. + +Configure it with `MAILBOX_IMAP_HOST`, `MAILBOX_SMTP_HOST` and `MAILBOX_USERS`, which are needed +together: set one of the three and the server refuses to start naming the others, rather than +booting with half a mailbox that fails at the first login in front of somebody. `MAILBOX_USERS` is +a comma-separated list, so several accounts on one shared host are one deployment, and the first +listed is the default that a call naming no `account` works in. `MAILBOX_USER` is still read as a +list of one for a deployment that already had it, and setting both refuses to start. +`MAILBOX_IMAP_PORT` and `MAILBOX_SMTP_PORT` default to the implicit-TLS ports, 993 and 465. Leave +the three required variables unset and the connector is still listed and still grantable, and every +call answers with the sentence naming what to set. + +The passwords are not environment variables. Each account's password is a row in the encrypted +credential vault at `/admin/credentials`, kind `mcp`, provider `mailbox`, key id the address itself, +so a deployment with three mailboxes holds three rows and rotates or revokes each on its own. A +password is read at the moment a call needs it, so a rotation takes effect on the next tool call +rather than on the next restart, and a revocation stops that account within a call. Nothing prints +one: IMAP command logging is off, and a mail server's failure sentence is scrubbed of the plaintext +and of the base64 forms before it reaches an audit row, a transcript or a model. + +`MAILBOX_ALLOWED_RECIPIENT_DOMAINS` bounds where mail may go, refusing a recipient outside the list +before any connection is opened. Unset means anywhere, which is the behaviour every deployment had +before this existed. It is there because a policy rule cannot do this job: a rule sees a tool call's +name and its effect, never its arguments, so the only thing it can express about `send_message` is +whether it happens at all. An approval rule decides that mail goes; the allowlist decides where it +may go. A deployment that cares should set both. + +Grants are per tool and not per account, so "may read the mail" and "may answer it" are two separate +decisions, while "may read support@ but not billing@" is not one this deployment can express: if an +account must stay out of a Bot's reach, do not configure it here. Every call takes the same route as +any other connector's. The Bot's grant is checked, the policy is evaluated with the tool's effect +(`send_message` as a write, the other three as reads), and an audit row is written, before any mail +server is dialled. + ### Coworkers are made in a wizard and managed in a dialog Creating a coworker is now a three-step wizard — who it is, who may see it, then where it runs, diff --git a/bun.lock b/bun.lock index 415b9ce62..d4f03be1a 100644 --- a/bun.lock +++ b/bun.lock @@ -72,6 +72,9 @@ "cron-parser": "^5", "drizzle-orm": "^0.45.2", "hono": "^4.10.0", + "imapflow": "^1.7.8", + "mailparser": "^3.9.20", + "nodemailer": "^9.1.1", "postgres": "^3.4.9", "rxjs": "7.8.1", "yaml": "^2.9.0", @@ -79,6 +82,8 @@ }, "devDependencies": { "@copilotkit/aimock": "1.39.0", + "@types/mailparser": "^3.4.6", + "@types/nodemailer": "^7.0.4", "drizzle-kit": "^0.31.10", "eventsource": "3.0.7", }, @@ -589,6 +594,8 @@ "@segment/analytics-node": ["@segment/analytics-node@2.3.0", "", { "dependencies": { "@lukeed/uuid": "^2.0.0", "@segment/analytics-core": "1.8.2", "@segment/analytics-generic-utils": "1.2.0", "buffer": "^6.0.3", "jose": "^5.1.0", "node-fetch": "^2.6.7", "tslib": "^2.4.1" } }, "sha512-fOXLL8uY0uAWw/sTLmezze80hj8YGgXXlAfvSS6TUmivk4D/SP0C0sxnbpFdkUzWg2zT64qWIZj26afEtSnxUA=="], + "@selderee/plugin-htmlparser2": ["@selderee/plugin-htmlparser2@0.12.0", "", { "dependencies": { "domelementtype": "~2.3.0", "domhandler": "~5.0.3" }, "peerDependencies": { "selderee": "~0.12.0" } }, "sha512-oELmoyA6ML9jDRMV3kgcMQFKxUfBU0yFVn6yTctVaLT5ygXnxH52I3TZEgV9EhXJC68/uFvE5Daj1/25c0Xa/A=="], + "@shadcn/react": ["@shadcn/react@0.3.0", "", { "peerDependencies": { "@types/react": ">=19", "react": ">=19" }, "optionalPeers": ["@types/react", "react"] }, "sha512-iKN0NuYe850VDHlxEfvppFrpa7CMV3zArTHusXlaSmeFeQhohGbIqclv6WwPTs/44I8EkXQpcuRt6sXEVKkWqQ=="], "@shikijs/core": ["@shikijs/core@3.23.0", "", { "dependencies": { "@shikijs/types": "3.23.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4", "hast-util-to-html": "^9.0.5" } }, "sha512-NSWQz0riNb67xthdm5br6lAkvpDJRTgB36fxlo37ZzM2yq0PQFFzbd8psqC2XMPgCzo1fW6cVi18+ArJ44wqgA=="], @@ -803,12 +810,16 @@ "@types/katex": ["@types/katex@0.16.8", "", {}, "sha512-trgaNyfU+Xh2Tc+ABIb44a5AYUpicB3uwirOioeOkNPPbmgRNtcWyDeeFRzjPZENO9Vq8gvVqfhaaXWLlevVwg=="], + "@types/mailparser": ["@types/mailparser@3.4.6", "", { "dependencies": { "@types/node": "*", "iconv-lite": "^0.6.3" } }, "sha512-wVV3cnIKzxTffaPH8iRnddX1zahbYB1ZEoAxyhoBo3TBCBuK6nZ8M8JYO/RhsCuuBVOw/DEN/t/ENbruwlxn6Q=="], + "@types/mdast": ["@types/mdast@4.0.4", "", { "dependencies": { "@types/unist": "*" } }, "sha512-kGaNbPh1k7AFzgpud/gMdvIm5xuECykRR+JnWKQno9TAXVa6WIVCGTPvYGekIDL4uwCZQSYbUxNBSb1aUo79oA=="], "@types/ms": ["@types/ms@2.1.0", "", {}, "sha512-GsCCIZDE/p3i96vtEqx+7dBUGXrc7zeSK3wwPHIaRThS+9OhWIXRqzs4d6k1SVU8g91DrNRWxWUGhp5KXQb2VA=="], "@types/node": ["@types/node@26.2.0", "", { "dependencies": { "undici-types": "~8.3.0" } }, "sha512-5IviulTZeRNp2vAJ514cc/HUlY5nZ9fCbq9DMyC52BrhFZACo3nI0R7qBxhQmo/d27NFe96ur/b7Wwxklda+kg=="], + "@types/nodemailer": ["@types/nodemailer@7.0.12", "", { "dependencies": { "@types/node": "*" } }, "sha512-80vKwiIsVSyFA1rRovH59jNPLBOuc6dRZIHEu40gXTkBkZnQv8vog1xSGEb9j5q/tdMAs5ivvDR2pLTU0hGHXA=="], + "@types/prop-types": ["@types/prop-types@15.7.15", "", {}, "sha512-F6bEyamV9jKGAFBEmlQnesRPGOQqS2+Uwi0Em15xenOxHaf2hv6L8YCVn3rPdPJOiJfPiCnLIRyvwVaqMY3MIw=="], "@types/qs": ["@types/qs@6.15.1", "", {}, "sha512-GZHUBZR9hckSUhrxmp1nG6NwdpM9fCunJwyThLW1X3AyHgd9IlHb6VANpQQqDr2o/qQp6McZ3y/IA2rVzKzSbw=="], @@ -869,6 +880,8 @@ "@xmldom/xmldom": ["@xmldom/xmldom@0.9.12", "", {}, "sha512-5AXjrcMClTryPe9LgZrygpB1lj7s0S9E0+W+AHaVKAVyHanafK86iPSvG5xHVSp/jC+VH1UXu0TAEmY279xH7A=="], + "@zone-eu/mailsplit": ["@zone-eu/mailsplit@5.4.16", "", { "dependencies": { "libbase64": "1.3.0", "libmime": "5.4.3", "libqp": "2.1.1" } }, "sha512-zQ9iXvlT3Wi/hazeC1MdI4rQc1UJwJ6IQ6QzSZ5KDxLZZWQSazWLOzImLFluXadKShJ9WJvI1xH+AyVS8b9azg=="], + "abort-controller": ["abort-controller@3.0.0", "", { "dependencies": { "event-target-shim": "^5.0.0" } }, "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg=="], "accepts": ["accepts@1.3.8", "", { "dependencies": { "mime-types": "~2.1.34", "negotiator": "0.6.3" } }, "sha512-PYAthTa2m2VKxuvSD3DPC/Gy+U+sOA1LAuT8mkmRuvw+NACSaeXEQ+NHcVF7rONl6qcaxV3Uuemwawk+7+SJLw=="], @@ -1129,6 +1142,8 @@ "deepmerge": ["deepmerge@4.3.1", "", {}, "sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A=="], + "deepmerge-ts": ["deepmerge-ts@8.0.2", "", {}, "sha512-uqbvqLUMrc6p0MO+WBRtTxY55hmyh94WRwI5a++PZe54X+bfVh59FSN7uWCBCW1CCVjzjnrwzfI8zidE2obMMw=="], + "default-browser": ["default-browser@5.5.1", "", { "dependencies": { "bundle-name": "^4.1.0", "default-browser-id": "^5.0.0" } }, "sha512-m1pAzaJgZ/gssEqlOhJkPJp8Xly7QyW6xcrkUa2KKcDeDSEMP7X8xipU3snUcfisTQx0w1AGae+9UtJSfVnXGw=="], "default-browser-id": ["default-browser-id@5.0.1", "", {}, "sha512-x1VCxdX4t+8wVfd1so/9w+vQ4vx7lKd2Qp5tDRutErwmR85OgmfX7RlLRMWafRMY7hbEiXIbudNrjOAPa/hL8Q=="], @@ -1157,8 +1172,16 @@ "dom-accessibility-api": ["dom-accessibility-api@0.5.16", "", {}, "sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg=="], + "dom-serializer": ["dom-serializer@2.0.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.2", "entities": "^4.2.0" } }, "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg=="], + + "domelementtype": ["domelementtype@2.3.0", "", {}, "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw=="], + + "domhandler": ["domhandler@5.0.3", "", { "dependencies": { "domelementtype": "^2.3.0" } }, "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w=="], + "dompurify": ["dompurify@3.4.14", "", { "optionalDependencies": { "@types/trusted-types": "^2.0.7" } }, "sha512-dVoH9z+MY+C9IilgGCk3YfFqjLi3fChm2OiKJMzh6axrJ5qwxqWaZamgmHrpv22CN/KdbZJuGEGgfQoL00LTdg=="], + "domutils": ["domutils@3.2.2", "", { "dependencies": { "dom-serializer": "^2.0.0", "domelementtype": "^2.3.0", "domhandler": "^5.0.3" } }, "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw=="], + "dot-prop": ["dot-prop@6.0.1", "", { "dependencies": { "is-obj": "^2.0.0" } }, "sha512-tE7ztYzXHIeyvc7N+hR3oi7FIbf/NIjVP9hmAt3yMXzrQ072/fpjGLx2GxNxGxUl5V73MEqYzioOMoVhGMJ5cA=="], "dotenv": ["dotenv@17.4.2", "", {}, "sha512-nI4U3TottKAcAD9LLud4Cb7b2QztQMUEfHbvhTH09bqXTxnSie8WnjPALV/WMCrJZ6UV/qHJ6L03OqO3LcdYZw=="], @@ -1181,6 +1204,8 @@ "encodeurl": ["encodeurl@2.0.0", "", {}, "sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg=="], + "encoding-japanese": ["encoding-japanese@2.3.0", "", {}, "sha512-eQyh1vzHz13DUkZcJO+0IOAoKXRQwKV5IBffeuYsWZyRLGiSzfzXObCqWvqFXdX0UU8qOk+lBXbkUhMCpdJe4Q=="], + "end-of-stream": ["end-of-stream@1.4.5", "", { "dependencies": { "once": "^1.4.0" } }, "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg=="], "enhanced-resolve": ["enhanced-resolve@5.24.5", "", { "dependencies": { "graceful-fs": "^4.2.4", "tapable": "^2.3.3" } }, "sha512-L1l8TNvomm6UVW5B253AGxQagSQr+vGwhMlrrfRS2qmhx46AMpMVJKQYLvWYbysTMY8VoicOvzHzoHMbyzB+4A=="], @@ -1367,14 +1392,20 @@ "hastscript": ["hastscript@9.0.1", "", { "dependencies": { "@types/hast": "^3.0.0", "comma-separated-tokens": "^2.0.0", "hast-util-parse-selector": "^4.0.0", "property-information": "^7.0.0", "space-separated-tokens": "^2.0.0" } }, "sha512-g7df9rMFX/SPi34tyGCyUBREQoKkapwdY/T04Qn9TDWfHhAYt4/I0gMVirzK5wEzeUqIjEB+LXC/ypb7Aqno5w=="], + "he": ["he@1.2.0", "", { "bin": { "he": "bin/he" } }, "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw=="], + "help-me": ["help-me@5.0.0", "", {}, "sha512-7xgomUX6ADmcYzFik0HzAxh/73YlKR9bmFzf51CZwR+b6YtzU2m0u49hQCqV6SvlqIqsaxovfwdvbnsw3b/zpg=="], "hono": ["hono@4.13.4", "", {}, "sha512-AGEwKIyRMHRv1t8Wjwa3LHxQ61X5CqrdFT+4BRNTpqS5aJNnpl5WLjADb7vFlJzI/8uK7T5QLVApCMQKNa3LgQ=="], + "html-to-text": ["html-to-text@10.0.1", "", { "dependencies": { "@selderee/plugin-htmlparser2": "~0.12.0", "deepmerge-ts": "^8.0.1", "dom-serializer": "^2.0.0", "htmlparser2": "^10.1.0", "selderee": "~0.12.0" } }, "sha512-GiVhRI1BatGARSCmlXWNCjDT0cWrwBWoeduLoV0WSKAgaV/wa+hUWy5LiQLUs4UwiUrE52ZCMfBGiKD87TDPrg=="], + "html-url-attributes": ["html-url-attributes@3.0.1", "", {}, "sha512-ol6UPyBWqsrO6EJySPz2O7ZSr856WDrEzM5zMqp+FJJLGMW35cLYmmZnl0vztAZxRUoNZJFTCohfjuIJ8I4QBQ=="], "html-void-elements": ["html-void-elements@3.0.0", "", {}, "sha512-bEqo66MRXsUGxWHV5IP0PUiAWwoEjba4VCzg0LjFJBpchPaTfyfCKTG6bc5F8ucKec3q5y6qOdGyYTSBEvhCrg=="], + "htmlparser2": ["htmlparser2@10.1.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.3", "domutils": "^3.2.2", "entities": "^7.0.1" } }, "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ=="], + "http-errors": ["http-errors@2.0.1", "", { "dependencies": { "depd": "~2.0.0", "inherits": "~2.0.4", "setprototypeof": "~1.2.0", "statuses": "~2.0.2", "toidentifier": "~1.0.1" } }, "sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ=="], "http-proxy-agent": ["http-proxy-agent@7.0.2", "", { "dependencies": { "agent-base": "^7.1.0", "debug": "^4.3.4" } }, "sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig=="], @@ -1383,12 +1414,14 @@ "human-signals": ["human-signals@8.0.1", "", {}, "sha512-eKCa6bwnJhvxj14kZk5NCPc6Hb6BdsU9DZcOnmQKSnO1VKrfV0zCvtttPZUsBvjmNDn8rpcJfpwSYnHBjc95MQ=="], - "iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "iconv-lite": ["iconv-lite@0.6.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw=="], "ieee754": ["ieee754@1.2.1", "", {}, "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA=="], "ignore": ["ignore@5.3.2", "", {}, "sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g=="], + "imapflow": ["imapflow@1.7.8", "", { "dependencies": { "@zone-eu/mailsplit": "5.4.16", "encoding-japanese": "2.3.0", "iconv-lite": "0.7.3", "libbase64": "1.3.0", "libmime": "5.4.3", "libqp": "2.1.1", "pino": "10.3.1", "socks": "2.8.9" } }, "sha512-dJoCIdZOJh26Rn2PdwEzwj0bRDgGBxxX38pio534FagIHVuR2l0SAfLr6sJo32YfuJHiSs/U2ntNDHnMg3/Hlg=="], + "import-fresh": ["import-fresh@3.3.1", "", { "dependencies": { "parent-module": "^1.0.0", "resolve-from": "^4.0.0" } }, "sha512-TR3KfrTZTYLPB6jUjfx6MF9WcWrHL9su5TObK4ZkYgBdWKPOFoSoQIdEuTuR82pmtxH2spWG9h6etwfr1pLBqQ=="], "import-meta-resolve": ["import-meta-resolve@4.2.0", "", {}, "sha512-Iqv2fzaTQN28s/FwZAoFq0ZSs/7hMAHJVX+w8PZl3cY19Pxk6jFFalxQoIfW2826i/fDLXv8IiEZRIT0lDuWcg=="], @@ -1503,8 +1536,16 @@ "layout-base": ["layout-base@1.0.2", "", {}, "sha512-8h2oVEZNktL4BH2JCOI90iD1yXwL6iNW7KcCKT2QZgQJR2vbqDsldCTPRU9NifTCqHZci57XvQQ15YTu+sTYPg=="], + "leac": ["leac@0.7.0", "", {}, "sha512-qMrZeyEekgdRQ9o6a4NAB2EQZrv827GJdn1vnapwSJ90hWRB4TzUSunvacPkxQ2TnNqHNI1/zSt0hlo0crG8Jw=="], + + "libbase64": ["libbase64@1.3.0", "", {}, "sha512-GgOXd0Eo6phYgh0DJtjQ2tO8dc0IVINtZJeARPeiIJqge+HdsWSuaDTe8ztQ7j/cONByDZ3zeB325AHiv5O0dg=="], + + "libmime": ["libmime@5.4.3", "", { "dependencies": { "encoding-japanese": "2.3.0", "iconv-lite": "0.7.3", "libbase64": "1.3.0", "libqp": "2.1.1" } }, "sha512-di9BoDabBUMqjeD/wGj+hHpSgdqAph5ui7w6OdY6NpzU6O6VFLQsMOg9tqCjm/zf9OHzAM9EZxSOF7uIb8O8Hw=="], + "libphonenumber-js": ["libphonenumber-js@1.13.11", "", {}, "sha512-ETER2kMaIFTI/Nh1a8Gk03dUF/SL0VZqtI+CcVHZxp5WIHYwNS7S+uiYZDYCvLy3lOR4/DAD5jf0h5WkePPpqg=="], + "libqp": ["libqp@2.1.1", "", {}, "sha512-0Wd+GPz1O134cP62YU2GTOPNA7Qgl09XwCqM5zpBv87ERCXdfDtyKXvV7c9U22yWJh44QZqBocFnXN11K96qow=="], + "lightningcss": ["lightningcss@1.32.0", "", { "dependencies": { "detect-libc": "^2.0.3" }, "optionalDependencies": { "lightningcss-android-arm64": "1.32.0", "lightningcss-darwin-arm64": "1.32.0", "lightningcss-darwin-x64": "1.32.0", "lightningcss-freebsd-x64": "1.32.0", "lightningcss-linux-arm-gnueabihf": "1.32.0", "lightningcss-linux-arm64-gnu": "1.32.0", "lightningcss-linux-arm64-musl": "1.32.0", "lightningcss-linux-x64-gnu": "1.32.0", "lightningcss-linux-x64-musl": "1.32.0", "lightningcss-win32-arm64-msvc": "1.32.0", "lightningcss-win32-x64-msvc": "1.32.0" } }, "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ=="], "lightningcss-android-arm64": ["lightningcss-android-arm64@1.32.0", "", { "os": "android", "cpu": "arm64" }, "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg=="], @@ -1533,6 +1574,8 @@ "lines-and-columns": ["lines-and-columns@1.2.4", "", {}, "sha512-7ylylesZQ/PV29jhEDl3Ufjo6ZX7gCqJr5F7PKrqc93v7fzSymt1BpwEU8nAUXs8qzzvqhbjhK5QZg6Mt/HkBg=="], + "linkify-it": ["linkify-it@5.0.2", "", { "dependencies": { "uc.micro": "^2.0.0" } }, "sha512-ONTm2jCMAVZjgQa/Fy1kScXsuOoF5NPTsoFBdE1KVIZ2vAh/r9+Bqo+0jINCBYnavTPQZz38QzFTme79ENoN3Q=="], + "lit": ["lit@3.3.3", "", { "dependencies": { "@lit/reactive-element": "^2.1.0", "lit-element": "^4.2.0", "lit-html": "^3.3.0" } }, "sha512-fycuvZg/hkpozL00lm1pEJH5nN/lr9ZXd6mJI2HSN4+Bzc+LDNdEApJ6HFbPkdFNHLvOplIIuJvxkS4XUxqirw=="], "lit-element": ["lit-element@4.2.2", "", { "dependencies": { "@lit-labs/ssr-dom-shim": "^1.5.0", "@lit/reactive-element": "^2.1.0", "lit-html": "^3.3.0" } }, "sha512-aFKhNToWxoyhkNDmWZwEva2SlQia+jfG0fjIWV//YeTaWrVnOxD89dPKfigCUspXFmjzOEUQpOkejH5Ly6sG0w=="], @@ -1581,6 +1624,8 @@ "magic-string": ["magic-string@0.30.21", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.5" } }, "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ=="], + "mailparser": ["mailparser@3.9.20", "", { "dependencies": { "@zone-eu/mailsplit": "5.4.16", "encoding-japanese": "2.3.0", "he": "1.2.0", "html-to-text": "10.0.1", "iconv-lite": "0.7.3", "libmime": "5.4.3", "linkify-it": "5.0.2", "nodemailer": "9.1.1", "punycode.js": "2.3.1", "tlds": "1.261.0" } }, "sha512-PZ9RD6B7SkmyQ9rj8JvYNS18rL6pWoNkSw9KGVuJQrrdouFxFIB05ugnR3ubInlnjLXV3KHXhGyz8MlTz1VGzw=="], + "markdown-table": ["markdown-table@3.0.4", "", {}, "sha512-wiYz4+JrLyb/DqW2hkFJxP7Vd7JuTDm77fvbM8VfEQdmSMqcImWeeRbHwZjBjIFki/VaMK2BhFi7oUUZeM5bqw=="], "marked": ["marked@17.0.6", "", { "bin": { "marked": "bin/marked.js" } }, "sha512-gB0gkNafnonOw0obSTEGZTT86IuhILt2Wfx0mWH/1Au83kybTayroZ/V6nS25mN7u8ASy+5fMhgB3XPNrOZdmA=="], @@ -1739,6 +1784,8 @@ "node-rsa": ["node-rsa@1.1.1", "", { "dependencies": { "asn1": "^0.2.4" } }, "sha512-Jd4cvbJMryN21r5HgxQOpMEqv+ooke/korixNNK3mGqfGJmy0M77WDDzo/05969+OkMy3XW1UuZsSmW9KQm7Fw=="], + "nodemailer": ["nodemailer@9.1.1", "", {}, "sha512-izw9mVKFix6YSnC9eLgV6g1opl9DUlRio9ZNcq+Wu9Ujn2UwF+8Nl0B8nz22kEC+CTZCvinkxwJ0DeFbb6NwcQ=="], + "npm-run-path": ["npm-run-path@6.0.0", "", { "dependencies": { "path-key": "^4.0.0", "unicorn-magic": "^0.3.0" } }, "sha512-9qny7Z9DsQU8Ou39ERsPU4OZQlSTP47ShQzuKZ6PRXpYLtIFgl/DEBYEXKlvcEa+9tHVcK8CF81Y2V72qaZhWA=="], "object-assign": ["object-assign@4.1.1", "", {}, "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg=="], @@ -1791,6 +1838,8 @@ "parse5": ["parse5@7.3.0", "", { "dependencies": { "entities": "^6.0.0" } }, "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw=="], + "parseley": ["parseley@0.13.1", "", { "dependencies": { "leac": "^0.7.0", "peberminta": "^0.10.0" } }, "sha512-uNBJZzmb60l6p6VWLTmevizNAGnE0xoSf1n0B4q3ntegDNzcS68NRCcBDZTcyXHxt2XhBChsCuqj4M+nChvE/A=="], + "parseurl": ["parseurl@1.3.3", "", {}, "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ=="], "partial-json": ["partial-json@0.1.7", "", {}, "sha512-Njv/59hHaokb/hRUjce3Hdv12wd60MtM9Z5Olmn+nehe0QDAsRtRbJPvJ0Z91TusF0SuZRIvnM+S4l6EIP8leA=="], @@ -1809,6 +1858,8 @@ "pathe": ["pathe@2.0.3", "", {}, "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w=="], + "peberminta": ["peberminta@0.10.0", "", {}, "sha512-80B2AsU+I4Qdb0ZAPSfe9UwvGzwkM37IKIFEvdS3D/3Ndgv2bsuJ0bfG1+iEYO+l7Gfd4EUJmuRyq7efLgRMzQ=="], + "phoenix": ["phoenix@1.8.12", "", {}, "sha512-svUniHb83aGh1XpYWHe4fkbfrGd6xa26TWO/3clkuNplXwgjxh7RI7GOWxf28VAgDWwQ2/yN8PxIa5dqjXOcAA=="], "picocolors": ["picocolors@1.1.1", "", {}, "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="], @@ -1863,6 +1914,8 @@ "pump": ["pump@3.0.4", "", { "dependencies": { "end-of-stream": "^1.1.0", "once": "^1.3.1" } }, "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA=="], + "punycode.js": ["punycode.js@2.3.1", "", {}, "sha512-uxFIHU0YlHYhDQtV4R9J6a52SLx28BCjT+4ieh7IGbgwVJWO+km431c4yRlREUAsAmt/uMjQUyQHNEPf0M39CA=="], + "qs": ["qs@6.15.3", "", { "dependencies": { "es-define-property": "^1.0.1", "side-channel": "^1.1.1" } }, "sha512-O9gl3zCl5h5blw1KGUzQKhA5oUXSl8rwUIM5o0S3nCXMliSvy5Dzx7/DJcI+SwgICv+IneSZwhBh1oSyEHA71A=="], "queue-microtask": ["queue-microtask@1.2.3", "", {}, "sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A=="], @@ -1979,6 +2032,8 @@ "secure-json-parse": ["secure-json-parse@2.7.0", "", {}, "sha512-6aU+Rwsezw7VR8/nyvKTx8QpWH9FrcYiXXlqC4z5d5XQBDRqtbfsRjnwGyqbi3gddNtWHuEk9OANUotL26qKUw=="], + "selderee": ["selderee@0.12.0", "", { "dependencies": { "parseley": "~0.13.1" } }, "sha512-b1YMh3+DHZp59DLna3qVwQ5iOla/nrI6mLBNW02XxU77M3046Df6VLkoaJyFz20VsGIG5kkp+FK0kg4K4HnUFw=="], + "semver": ["semver@6.3.1", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA=="], "send": ["send@0.19.2", "", { "dependencies": { "debug": "2.6.9", "depd": "2.0.0", "destroy": "1.2.0", "encodeurl": "~2.0.0", "escape-html": "~1.0.3", "etag": "~1.8.1", "fresh": "~0.5.2", "http-errors": "~2.0.1", "mime": "1.6.0", "ms": "2.1.3", "on-finished": "~2.4.1", "range-parser": "~1.2.1", "statuses": "~2.0.2" } }, "sha512-VMbMxbDeehAxpOtWJXlcUS5E8iXh6QmN+BkRX1GARS3wRaXEEgzCcB10gTQazO42tpNIya8xIyNx8fll1OFPrg=="], @@ -2081,6 +2136,8 @@ "tinyglobby": ["tinyglobby@0.2.17", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g=="], + "tlds": ["tlds@1.261.0", "", { "bin": { "tlds": "bin.js" } }, "sha512-QXqwfEl9ddlGBaRFXIvNKK6OhipSiLXuRuLJX5DErz0o0Q0rYxulWLdFryTkV5PkdZct5iMInwYEGe/eR++1AA=="], + "tldts": ["tldts@7.4.11", "", { "dependencies": { "tldts-core": "^7.4.11" }, "bin": { "tldts": "bin/cli.js" } }, "sha512-aBiNayCfTQxuIJBm06M+xR14cYaYlDlSXZbgsnKzKNxDKUVq7KFwTjwBSsb7m9Y5xO8WfPnBc63WaYFMTGlvqw=="], "tldts-core": ["tldts-core@7.4.11", "", {}, "sha512-CW3WN2rIIE/Of21mulhgnGOwoDyEFNygyIBOONSdyAuSATgMMUCpLeUlB+E8sAwA5xRV9hYPl+kyZ9citHCaKg=="], @@ -2115,6 +2172,8 @@ "typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="], + "uc.micro": ["uc.micro@2.1.0", "", {}, "sha512-ARDJmphmdvUk6Glw7y9DQ2bFkKBHwQHLi2lsaH6PPmz/Ka9sFOBsBluozhDltWmnv9u/cF6Rt87znRTPV+yp/A=="], + "undici": ["undici@7.29.0", "", {}, "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw=="], "undici-types": ["undici-types@8.3.0", "", {}, "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ=="], @@ -2361,12 +2420,12 @@ "d3-dsv/commander": ["commander@7.2.0", "", {}, "sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw=="], - "d3-dsv/iconv-lite": ["iconv-lite@0.6.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw=="], - "d3-sankey/d3-array": ["d3-array@2.12.1", "", { "dependencies": { "internmap": "^1.0.0" } }, "sha512-B0ErZK/66mHtEsR1TkPEEkwdy+WDesimkM5gpZr5Dsg54BiTA5RXtYW5qTLIAcekaS9xfZrzBLF/OAkB3Qn1YQ=="], "d3-sankey/d3-shape": ["d3-shape@1.3.7", "", { "dependencies": { "d3-path": "1" } }, "sha512-EUkvKjqPFUAZyOlhY5gzCxCeI0Aep04LwIRpsZ/mLFelJiUfnK56jo5JMDSE7yyP2kLSb6LtF+S5chMk7uqPqw=="], + "dom-serializer/entities": ["entities@4.5.0", "", {}, "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw=="], + "dot-prop/is-obj": ["is-obj@2.0.0", "", {}, "sha512-drqDG3cbczxxEJRoOXcOjtdp1J/lyp1mNn0xaznRs8+muBhgQcrnbspox5X5fOw0HnMnbfDzvnEMEtqDEJEo8w=="], "drizzle-kit/esbuild": ["esbuild@0.25.12", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.25.12", "@esbuild/android-arm": "0.25.12", "@esbuild/android-arm64": "0.25.12", "@esbuild/android-x64": "0.25.12", "@esbuild/darwin-arm64": "0.25.12", "@esbuild/darwin-x64": "0.25.12", "@esbuild/freebsd-arm64": "0.25.12", "@esbuild/freebsd-x64": "0.25.12", "@esbuild/linux-arm": "0.25.12", "@esbuild/linux-arm64": "0.25.12", "@esbuild/linux-ia32": "0.25.12", "@esbuild/linux-loong64": "0.25.12", "@esbuild/linux-mips64el": "0.25.12", "@esbuild/linux-ppc64": "0.25.12", "@esbuild/linux-riscv64": "0.25.12", "@esbuild/linux-s390x": "0.25.12", "@esbuild/linux-x64": "0.25.12", "@esbuild/netbsd-arm64": "0.25.12", "@esbuild/netbsd-x64": "0.25.12", "@esbuild/openbsd-arm64": "0.25.12", "@esbuild/openbsd-x64": "0.25.12", "@esbuild/openharmony-arm64": "0.25.12", "@esbuild/sunos-x64": "0.25.12", "@esbuild/win32-arm64": "0.25.12", "@esbuild/win32-ia32": "0.25.12", "@esbuild/win32-x64": "0.25.12" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-bbPBYYrtZbkt6Os6FiTLCTFxvq4tt3JKall1vRwshA3fdVztsLAatFaZobhkBC8/BrPetoa0oksYoKXoG4ryJg=="], @@ -2437,18 +2496,26 @@ "hastscript/property-information": ["property-information@7.2.0", "", {}, "sha512-IAtzIB6sUiWaJYrX9smp3V46pBGbBeLFRGdh25kg1334VcBlD8HzhPeNIWQH9zhGmo2itIe25EHt9dQP7G5hmg=="], + "imapflow/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + + "imapflow/pino": ["pino@10.3.1", "", { "dependencies": { "@pinojs/redact": "^0.4.0", "atomic-sleep": "^1.0.0", "on-exit-leak-free": "^2.1.0", "pino-abstract-transport": "^3.0.0", "pino-std-serializers": "^7.0.0", "process-warning": "^5.0.0", "quick-format-unescaped": "^4.0.3", "real-require": "^0.2.0", "safe-stable-stringify": "^2.3.1", "sonic-boom": "^4.0.1", "thread-stream": "^4.0.0" }, "bin": { "pino": "bin.js" } }, "sha512-r34yH/GlQpKZbU1BvFFqOjhISRo1MNx1tWYsYvmj6KIRHSPMT2+yHOEb1SG6NMvRoHRF0a07kCOox/9yakl1vg=="], + "is-inside-container/is-docker": ["is-docker@3.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ=="], "jsonwebtoken/semver": ["semver@7.8.5", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA=="], "katex/commander": ["commander@8.3.0", "", {}, "sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww=="], + "libmime/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "log-symbols/chalk": ["chalk@5.6.2", "", {}, "sha512-7NzBL0rN6fMUW+f7A6Io4h40qQlG+xGmtMxfbnH/K7TAtt8JQWVQK+6g0UXKMeVJoyV5EkkNsErQ8pVD3bLHbA=="], "log-symbols/is-unicode-supported": ["is-unicode-supported@1.3.0", "", {}, "sha512-43r2mRvz+8JRIKnWJ+3j8JtjRKZ6GmjzfaE/qiBJnikNnYv/6bagRJ1kUhNk8R5EX/GkobD+r+sfxCPJsiKBLQ=="], "lru-memoizer/lru-cache": ["lru-cache@11.5.2", "", {}, "sha512-4pfM1Ff0x50o0tQwb5ucw/RzNyD0/YJME6IVcStalZuMWxdt3sR3huStTtxz4PUmvZfRguvDejasvQ2kifR11g=="], + "mailparser/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "mdast-util-definitions/@types/mdast": ["@types/mdast@3.0.15", "", { "dependencies": { "@types/unist": "^2" } }, "sha512-LnwD+mUEfxWMa1QpDraczIn6k0Ee3SMicuYSSzS6ZYl2gKS09EClnJYGd8Du6rfc5r/GZEk5o1mRb8TaTj03sQ=="], "mdast-util-definitions/unist-util-visit": ["unist-util-visit@4.1.2", "", { "dependencies": { "@types/unist": "^2.0.0", "unist-util-is": "^5.0.0", "unist-util-visit-parents": "^5.1.1" } }, "sha512-MSd8OUGISqHdVvfY9TPhyK2VdUrPgxkUtWSuMHF6XAAFuL4LokseigBnZtPnJMu+FbynTkFNnFlyjxpVKujMRg=="], @@ -2493,6 +2560,8 @@ "prop-types/react-is": ["react-is@16.13.1", "", {}, "sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ=="], + "raw-body/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "react-markdown/remark-parse": ["remark-parse@10.0.2", "", { "dependencies": { "@types/mdast": "^3.0.0", "mdast-util-from-markdown": "^1.0.0", "unified": "^10.0.0" } }, "sha512-3ydxgHa/ZQzG8LvC7jTXccARYDcRld3VfcgIIFs7bI6vbRSxJJmzgLEIIoYKyrfhaY+ujuWaf/PJiMZXoiCXgw=="], "react-markdown/remark-rehype": ["remark-rehype@10.1.0", "", { "dependencies": { "@types/hast": "^2.0.0", "@types/mdast": "^3.0.0", "mdast-util-to-hast": "^12.1.0", "unified": "^10.0.0" } }, "sha512-EFmR5zppdBp0WQeDVZ/b66CWJipB2q2VLNFMabzDSGR66Z2fQii83G5gTBbgGEnEEA0QRussvrFHxk1HWGJskw=="], @@ -2787,6 +2856,10 @@ "hastscript/@types/hast/@types/unist": ["@types/unist@3.0.3", "", {}, "sha512-ko/gIFJRv177XgZsZcBwnqJN5x/Gien8qNOn0D5bQU/zAzVf9Zt3BlcUiLqhV9y4ARk0GbT3tnUiPNgnTXzc/Q=="], + "imapflow/pino/pino-abstract-transport": ["pino-abstract-transport@3.0.0", "", { "dependencies": { "split2": "^4.0.0" } }, "sha512-wlfUczU+n7Hy/Ha5j9a/gZNy7We5+cXp8YL+X+PG8S0KXxw7n/JXA3c46Y0zQznIJ83URJiwy7Lh56WLokNuxg=="], + + "imapflow/pino/thread-stream": ["thread-stream@4.2.0", "", { "dependencies": { "real-require": "^1.0.0" } }, "sha512-e2zZ96wSChazBsbENf/Pcm/4swHt2cEKQ92rhUjkL9GCKiTDJIaTBenjE/m9DXi0QBmTMDkFDdOomUy20A1tDQ=="], + "mdast-util-definitions/unist-util-visit/unist-util-is": ["unist-util-is@5.2.1", "", { "dependencies": { "@types/unist": "^2.0.0" } }, "sha512-u9njyyfEh43npf1M+yGKDGVPbY/JWEemg5nH05ncKPfi+kBbKBJoTdsogMu33uhytuLlv9y0O7GH7fEdwLdLQw=="], "mdast-util-definitions/unist-util-visit/unist-util-visit-parents": ["unist-util-visit-parents@5.1.3", "", { "dependencies": { "@types/unist": "^2.0.0", "unist-util-is": "^5.0.0" } }, "sha512-x6+y8g7wWMyQhL1iZfhIPhDAs7Xwbn9nRosDXl7qoPTSCy0yNxnKc+hWokFifWQIDGi154rdUqKvbCa4+1kLhg=="], @@ -2839,6 +2912,8 @@ "@modelcontextprotocol/sdk/express/body-parser/content-type": ["content-type@2.1.0", "", {}, "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag=="], + "@modelcontextprotocol/sdk/express/body-parser/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "@modelcontextprotocol/sdk/express/type-is/content-type": ["content-type@2.1.0", "", {}, "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag=="], "@modelcontextprotocol/sdk/express/type-is/media-typer": ["media-typer@1.1.1", "", {}, "sha512-yz3xRaG20c6/BOzvYoDaGtPmGscs7YivItZEEqe6GbwNfHuxu9YNmvnEkMzKldAGY4/80pRcQRZSEnhquk9XuQ=="], @@ -2847,10 +2922,14 @@ "@slack/bolt/express/body-parser/content-type": ["content-type@2.1.0", "", {}, "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag=="], + "@slack/bolt/express/body-parser/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "@slack/bolt/express/type-is/content-type": ["content-type@2.1.0", "", {}, "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag=="], "@slack/bolt/express/type-is/media-typer": ["media-typer@1.1.1", "", {}, "sha512-yz3xRaG20c6/BOzvYoDaGtPmGscs7YivItZEEqe6GbwNfHuxu9YNmvnEkMzKldAGY4/80pRcQRZSEnhquk9XuQ=="], + "imapflow/pino/thread-stream/real-require": ["real-require@1.0.0", "", {}, "sha512-P4nbQYQfePJxRSmY+v/KINxVucm4NF3p3s7pJveMTtom52FR4YGltUQLB8idDXwDDWW+eYrWDFbuzUnjoWHF7g=="], + "react-markdown/remark-parse/mdast-util-from-markdown/mdast-util-to-string": ["mdast-util-to-string@3.2.0", "", { "dependencies": { "@types/mdast": "^3.0.0" } }, "sha512-V4Zn/ncyN1QNSqSBxTrMOLpjr+IKdHl2v3KVLoWmDPscP4r9GcCi71gjgvUV1SFSKh92AjAG4peFuBl2/YgCJg=="], "react-markdown/remark-parse/mdast-util-from-markdown/micromark": ["micromark@3.2.0", "", { "dependencies": { "@types/debug": "^4.0.0", "debug": "^4.0.0", "decode-named-character-reference": "^1.0.0", "micromark-core-commonmark": "^1.0.1", "micromark-factory-space": "^1.0.0", "micromark-util-character": "^1.0.0", "micromark-util-chunked": "^1.0.0", "micromark-util-combine-extensions": "^1.0.0", "micromark-util-decode-numeric-character-reference": "^1.0.0", "micromark-util-encode": "^1.0.0", "micromark-util-normalize-identifier": "^1.0.0", "micromark-util-resolve-all": "^1.0.0", "micromark-util-sanitize-uri": "^1.0.0", "micromark-util-subtokenize": "^1.0.0", "micromark-util-symbol": "^1.0.0", "micromark-util-types": "^1.0.1", "uvu": "^0.5.0" } }, "sha512-uD66tJj54JLYq0De10AhWycZWGQNUvDI55xPgk2sQM5kn1JYlhbCMTtEeT27+vAhW2FBQxLlOmS3pmA7/2z4aA=="], diff --git a/docs/README.md b/docs/README.md index 6b7c66687..80d6ca5d9 100644 --- a/docs/README.md +++ b/docs/README.md @@ -7,6 +7,7 @@ Start with the root [README](../README.md), then use these references: - [Development](development.md): local setup, migrations, ports, and quality checks. - [Coworkers](coworkers.md): durable Bot profiles, channels, visibility, deletion, and external AG-UI registration. - [Routines](routines.md): standing instructions a Bot runs on a schedule, the worker that fires them, and who they run as. +- [Mailbox](mailbox.md): the deployment's own mailbox, the four tools a Bot reads and answers it with, and where the password lives. - Plugins, one connector per page — what an administrator registers, what each person consents to, and what the failures mean: - [Google Drive](plugins/google-drive.md) - [Notion](plugins/notion.md) diff --git a/docs/configuration.md b/docs/configuration.md index 738d1a1d8..6fe138f01 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -57,6 +57,13 @@ at `agent-langgraph` on a laptop. | `AUDIT_RETENTION_DAYS` | unset | Whole number of days to keep audit rows; older ones are removed. Unset keeps the trail forever. | | `WORKER_SHARED_SECRET` | unset; `start.sh` uses a fixed local default | The secret the routines worker presents to fire a due routine. Without it the server refuses every handoff, whether or not a worker exists to send one. | | `OPENBOT_GENERATIVE_UI` | unset (capability off) | `true` or `1` lets a Bot answer with an interface it wrote itself. | +| `MAILBOX_IMAP_HOST` | unset | IMAP server for the deployment's mailbox. Needed together with the SMTP host and the user; see [mailbox.md](mailbox.md). | +| `MAILBOX_SMTP_HOST` | unset | SMTP server the mailbox sends through. | +| `MAILBOX_USERS` | unset | Comma-separated addresses on those hosts, one per mailbox account. The first is the default account. Passwords are not environment variables: each is a vault credential keyed by the address. | +| `MAILBOX_USER` | unset | The singular this feature shipped with, still read as a list of one. Setting it together with `MAILBOX_USERS` refuses to start. | +| `MAILBOX_IMAP_PORT` | `993` | Implicit TLS. Set only for a server that listens elsewhere. | +| `MAILBOX_SMTP_PORT` | `465` | Implicit TLS. Set only for a server that listens elsewhere. | +| `MAILBOX_ALLOWED_RECIPIENT_DOMAINS` | unset (anywhere) | Comma-separated domains a Bot may send mail to. A policy rule cannot do this, because rules see a tool's name and effect and not its arguments; see [mailbox.md](mailbox.md). | **`OPENBOT_GENERATIVE_UI`** turns on generated interfaces. Set it, and a Bot may answer by writing the markup, styles and script for an interface and streaming it into the transcript, where it renders diff --git a/docs/mailbox.md b/docs/mailbox.md new file mode 100644 index 000000000..fe04efe89 --- /dev/null +++ b/docs/mailbox.md @@ -0,0 +1,240 @@ +# Mailbox + +The Mailbox connector gives a Bot the deployment's mail: it can list what has arrived, open a message, search +for one, and send mail, including a threaded reply. "Check the support mailbox and tell me what came +in overnight" is a mailbox question, and so is "reply to Dana and say we will have it by Friday." + +The mailbox belongs to the deployment. It is not the mailbox of whoever is asking, and no person +connects an account to it. That is the difference from Google Drive or Notion, where each person +consents for themselves and sees only their own; here everybody granted the tools reads the same +mail. Decide that before granting it, because it is the whole of the access model. + +A deployment may configure several accounts on one pair of hosts, which is what a shared host gives +you: `support@`, `sales@` and `billing@` are three mailboxes on one IMAP and one SMTP server, with a +password each. Every tool takes an optional `account` to say which one to work in; leaving it out +works in the first configured, the default. + +**Grants are per tool, not per account.** A Bot granted `list_messages` can list every configured +account, and one granted `send_message` can send from any of them. There is no per-account grant and +no way to give a Bot one mailbox out of three. If an account must stay out of a Bot's reach, do not +configure it on this deployment. + +## The prerequisite + +Two things, and both are an administrator's: + +1. **Configuration**, which says where the mailbox is and which accounts are on it (below). +2. **Grants**, which say which Bots may use it. Mailbox is a catalogue entry like any other, at + `/admin/plugins/mailbox`, and enabling the entry hands no Bot anything. Each of the four tools is + granted per Bot, so "may read the mail" and "may answer it" are two separate decisions. + +`send_message` is the connector's only write tool, and it is a write in the strongest sense this +product has: it reaches people who never agreed to talk to a Bot, and there is nothing to recall once +it has run. Grant it deliberately, and consider a policy rule that requires approval for it the same +way one would for any other irreversible action. + +## Configuration + +| Variable | Default | What it is | +| -------------------- | ------- | ----------------------------------------------------------------- | +| `MAILBOX_IMAP_HOST` | unset | The IMAP server messages are read from. | +| `MAILBOX_SMTP_HOST` | unset | The SMTP server mail is sent through. | +| `MAILBOX_USERS` | unset | Comma-separated addresses on those hosts. The first is the default account. | +| `MAILBOX_USER` | unset | The singular this shipped with, read as a list of one. Both set refuses to start. | +| `MAILBOX_IMAP_PORT` | `993` | Implicit TLS. Set only for a server that listens elsewhere. | +| `MAILBOX_SMTP_PORT` | `465` | Implicit TLS. Set only for a server that listens elsewhere. | +| `MAILBOX_ALLOWED_RECIPIENT_DOMAINS` | unset (anywhere) | Comma-separated domains a Bot may send to. See [Bounding where mail can go](#bounding-where-mail-can-go). | + +The three hosts-and-users variables are needed together. Set one or two and the server refuses to +start, naming the ones that are missing, rather than booting with half a mailbox that fails at the +first login, at run time, in front of somebody, with nothing but an authentication error from a +server that will not say which half was wrong. Leave all three unset and the connector is still +listed, still grantable, and every tool call answers with the sentence naming what to set. + +`MAILBOX_USERS` is a list, so `support@example.com,sales@example.com` is two accounts and +`support@example.com` is one. The order matters: the first is the default. Addresses are lower-cased +and deduplicated, and an entry that is not an address refuses to start, naming it, rather than +becoming an account a model can select and nothing can unlock. `MAILBOX_USER` is still read as a +list of one for a deployment that already had it; setting both refuses to start, because they are +two answers to the same question. + +Both ports default to the implicit-TLS ones rather than the STARTTLS ones, so the connection is +encrypted before the password is sent rather than negotiating for it in the clear. + +### The passwords are not environment variables + +They are the only secrets in this feature, so they live where this deployment's other secrets live: +the encrypted credential vault. Store **one credential per account** at `/admin/credentials` as: + +- **kind** `mcp`, the vault's name for "the one token this deployment holds for this server", which + is exactly what a mailbox password is: one secret, the deployment's, used for every Bot granted the + tools, never anybody's own grant. +- **provider** `mailbox` +- **key id** the address exactly as it is configured, lower-cased: `support@example.com`. + +So a deployment with three accounts holds three rows, each rotated and revoked on its own. An +account with no stored password is refused by name, naming that account and the key id to store it +under, while the accounts that do have one keep working. + +Rotate in the same place. A password is read from the vault at the moment a call needs it and thrown +away after, so a rotation takes effect on the next tool call rather than on the next restart, and +revoking it stops that account within a call. + +Nothing prints one. IMAP logging is off at the client, which matters because one of the commands it +would log is the authentication one; and a failure sentence from a mail server that quoted the login +back is scrubbed before it reaches an audit row, a transcript or a model. The scrub covers the +base64 forms as well as the plaintext, because neither client sends the password as typed: IMAP +authenticates with `AUTH=PLAIN` or `AUTH=LOGIN`, both base64 on the wire, so a quoted command +carries an encoding of it rather than the password itself. + +## The four tools + +Every one of them takes two arguments about where to look, and they are different things: + +- **`account`** is an email address, one of the configured ones, and it says which mailbox to open. + It defaults to the first configured. An account this deployment does not have is refused before + the vault is read and before anything is dialled, and the refusal lists the ones that exist. +- **`folder`** is an IMAP folder inside that account, such as `INBOX`, `Sent` or `Archive`, and it + defaults to `INBOX`. + +**A configured address in `folder` is adopted rather than refused.** If the value is one of this +deployment's own accounts, and `account` is either unset or the same address, it is taken as the +account, the folder falls back to `INBOX`, and the answer opens with one line saying so: "[folder +took the address support@example.com; it was used as account, reading INBOX.]" There is exactly one +mailbox that value can mean and the model named it, so refusing would spend a whole turn teaching +vocabulary before any work happens, and live runs show the mistake on the first mailbox call of a +turn. The note teaches the same lesson on the way past. + +Two neighbouring cases are still refused before anything is dialled, because neither is unambiguous: + +- **An address in `folder` that is not a configured account.** There is nothing to adopt, so the + refusal says to pass it as `account` instead. +- **`folder` holding one configured address while `account` names a different one.** Two arguments + naming two mailboxes is a model that has lost track of which it is reading. The refusal names + both. + +**The part before the @ of a configured account is refused too**, and deliberately not adopted: +`support` is not an address, and a folder genuinely called `support` can exist. + +None of this is hypothetical. A smaller model given a `mailbox` argument and an `account` argument +put the address in the first one, was answered "Character not allowed in mailbox name" by the IMAP +server, and never tried the second; refused that, it retried with `support` and then `webmaster`, +which are the local parts of two configured accounts, and was answered "Mailbox doesn't exist: +support". The argument is named `folder`, says in its own description that it is neither an address +nor half of one, and is listed after `account` so that is the argument a model meets first. + +A folder that genuinely is not there is answered with the folders that are: "No folder named +Newsletters in support@example.com. Folders here: INBOX, Sent, Archive. The account is chosen by +`account`, not by folder; leave folder unset for INBOX." The listing costs one `LIST` on the +connection that was already open, inside the same deadline, and it is what turns the vendor's +"Mailbox doesn't exist" into something a model can act on rather than retry against. + +Every answer names both, so a turn that reads two accounts cannot merge them and a model reading the +result learns the vocabulary: "showing 10 of 236 messages in folder INBOX of support@example.com, +newest first." + +- **`list_messages`**: the newest messages in a mailbox, newest first: uid, date, sender, subject + and whether it has been read. No bodies. `limit` defaults to 10 and is capped at 50. +- **`read_message`**: one message by `uid`, with its headers and its text. +- **`search_messages`**: messages whose subject, sender or text match `query`, newest first. + `limit` defaults to 20. The match is the mail server's own IMAP `SEARCH`: a plain substring, with + no ranking and no boolean syntax. +- **`send_message`**: sends `to`, `subject` and `body` from the deployment's mailbox, and files a + copy in the account's Sent folder. Give `in_reply_to` as the uid of a message and the reply + threads: the original is fetched, its `Message-ID` becomes `In-Reply-To`, its own `References` + chain plus that id becomes `References`, and `Re: ` goes in front of the subject if it is not + already there. Without it, the message opens a new thread however the subject is worded. + +The sender is always the selected account, and the confirmation says which. There is no `from` +field, so a Bot cannot send as somebody else. + +### The copy in Sent + +SMTP delivers a message; it does not file one. So a send that did nothing else would leave the +account's Sent folder empty, webmail showing nothing sent, and a Bot unable to find its own outgoing +mail. That is not hypothetical: a person checking the mailbox concluded three messages had never +been sent, and all three had been delivered. + +So the message is built once as raw bytes, those bytes are what SMTP delivers, and the same bytes +are appended to the account's Sent folder, marked `\Seen`. Delivered and stored are then byte for +byte the same message, down to the `Message-ID`, which is what makes a send verifiable: the +confirmation names the folder ("A copy is in folder Sent.") and the message id, so a Bot can list +that folder and find what it just sent. + +The folder is resolved by its IMAP special-use flag first, which is the answer that survives a +localised server, and by the names `Sent`, `Sent Items` and `Sent Messages` on a server that marks +nothing. An account where neither finds one files no copy rather than guessing, since appending to +the wrong folder would put outgoing mail where a person reads it as incoming. + +**A copy that could not be filed is never reported as a send that failed.** Filing is a separate +operation against a separate server and happens after delivery, so it can fail on a message that has +already gone. The confirmation then says the mail was sent, names the reason no copy exists, and +says not to send it again. A model told only that something failed would send the message a second +time, and mail cannot be recalled. + +**Uids are per folder, and a folder belongs to one account.** A uid from a listing of `Archive` +names a different message in `INBOX`, and a uid from `support@`'s INBOX names a different message in +`sales@`'s, so every tool that takes one also takes the `folder` and the `account` it came from. A +uid that is not there is refused by name rather than guessed at, and for `send_message` nothing is +sent. + +**Every result is bounded.** At most 512 KB of a message is read off the wire, so one mail with a +large attachment cannot become a gigabyte in this process; a message body is then cut at 8,000 +characters and says so, with the full length, so a model can tell a message that has said what it +came to say from one that has not; and the whole result is capped again at the same 20,000 +characters every other connector's is. A listing or a search that had more behind it says +"showing 10 of 4321" rather than presenting a page as the whole mailbox. Truncation is always +visible, never silent. + +**Every call has a deadline.** Sixty seconds of wall clock per network operation, on top of the +thirty-second inactivity timeouts, and the socket is closed when it expires. The inactivity timeouts +alone would let a server that drips one byte at a time hold a turn open forever. + +**Nothing is changed by reading.** Opening a message does not mark it read, move it or delete it, and +there is no tool that does. A connection is opened, used and closed per call; no session is kept. + +## Bounding where mail can go + +`MAILBOX_ALLOWED_RECIPIENT_DOMAINS` is a comma-separated list of domains (`example.com,partner.example`; +a leading `@` is accepted and dropped). Set it and `send_message` refuses any recipient outside the +list before it opens a connection, naming the domain that was refused and saying nothing was sent. +Unset or empty means anywhere, so a deployment that has not set it behaves exactly as before. + +**Why this exists rather than a policy rule.** The policy engine sees a tool call's name and its +effect. It does not see the arguments, so no rule can say "may email the company and nobody else": +the only thing a rule can express about `send_message` is whether it happens at all. That leaves a +deployment two controls, and they do different jobs: + +1. **An approval rule on `send_message`** in your boundaries. Wholesale, per send, with a person in + the loop. This is the control that decides whether mail goes out. +2. **This allowlist.** Bounds where anything can go, with nobody in the loop. + +**The shape worth protecting against.** The read tools pull text that somebody else wrote into a +model's context, and that text can contain instructions. A Bot holding the read tools and an +unconstrained `send_message` can therefore be talked into mailing the mailbox out, by an email +addressed to it, with no person involved at any point. That is why `send_message` should be +approval-gated wherever the mailbox holds anything worth keeping, and why the allowlist is worth +setting even when it is gated: it is the half that still holds if a rule is edited or a mode is +switched to dry-run. + +## Governance + +Nothing about a Mailbox tool call is special. It goes through `plugins/store.ts` like every other +connector: the Bot's grant is checked, the policy is evaluated with the tool's effect (`send_message` +as a write, the other three as reads), an audit row is written, and only then is any mail server +dialled. There is no second path to the mailbox and no bypass. + +The grant is per tool. It is not per account, and the policy engine cannot make it one, for the same +reason it cannot bound recipients: a rule sees a tool call's name and effect, never its arguments. +A Bot granted the mailbox tools reaches every configured account. + +The audit trail records every call, and for a failure it keeps the mail server's own sentence, which +is usually the most useful thing available: "Invalid credentials", "Mailbox does not exist" and +"Relay access denied" each name a different fix. + +## See also + +- [Configuration](configuration.md): every environment variable, in one table. +- [Architecture](architecture.md): where a plugin call is decided, recorded and made. +- [Routines](routines.md): the other first-party connector that runs in-process, and the one to read + next if you want a Bot to check the mailbox on a schedule. diff --git a/server/package.json b/server/package.json index 4b6fa4c13..ca551e1b8 100644 --- a/server/package.json +++ b/server/package.json @@ -22,6 +22,9 @@ "cron-parser": "^5", "drizzle-orm": "^0.45.2", "hono": "^4.10.0", + "imapflow": "^1.7.8", + "mailparser": "^3.9.20", + "nodemailer": "^9.1.1", "postgres": "^3.4.9", "rxjs": "7.8.1", "yaml": "^2.9.0", @@ -29,6 +32,8 @@ }, "devDependencies": { "@copilotkit/aimock": "1.39.0", + "@types/mailparser": "^3.4.6", + "@types/nodemailer": "^7.0.4", "drizzle-kit": "^0.31.10", "eventsource": "3.0.7" } diff --git a/server/src/config.ts b/server/src/config.ts index 1c235f9c3..2e2491925 100644 --- a/server/src/config.ts +++ b/server/src/config.ts @@ -61,6 +61,53 @@ export type ComputerConfig = | SharedComputerConfig | SandboxComputerConfig; +/** + * The deployment's own mailbox: where it is and which accounts it signs in as. + * + * NO PASSWORD HERE, and that is the point of the shape. Everything in this file comes from the + * environment, which means a `.env` file on a laptop, a compose file in a repository and whatever a + * cluster hands a container as plain text. The mailbox passwords are the only secrets in this + * feature, so they live where the other secrets this deployment holds live: the encrypted + * credential vault, one credential per account, keyed by the address. See + * `plugins/builtin-mailbox.ts` for how one is resolved. + * + * Both ports default to the implicit-TLS ones (993 and 465) rather than to the STARTTLS ones, so a + * deployment that sets only the two hosts gets an encrypted connection rather than one that + * negotiates for it in the clear. + */ +export type MailboxConfig = { + imapHost: string; + imapPort: number; + smtpHost: string; + smtpPort: number; + /** + * The accounts on these hosts, lower-cased and in the order they were configured. + * + * Never empty: a mailbox with no account is refused at start-up rather than carried around as a + * shape every caller has to check. The FIRST is the default, which is the one a tool call that + * named no account works in, so the order in `MAILBOX_USERS` is a decision rather than a detail. + * + * Several accounts, one set of hosts: this is the shared-hosting shape, where `support@`, + * `sales@` and `billing@` are all mailboxes on the same IMAP and SMTP servers with a password + * each. Each password is its own vault credential, keyed by the address. + */ + users: readonly string[]; + /** + * The domains a Bot may send to. Empty means anywhere, which is the default. + * + * WHY A DEPLOYMENT MIGHT WANT THIS. The policy engine sees a tool call's NAME and its effect, not + * its arguments, so no rule can say "may email the company and nobody else": the only thing a + * rule can do about `send_message` is require approval for all of it or none of it. That leaves a + * gap this closes: the read tools bring text somebody else wrote into a model's context, and a + * Bot holding read plus unconstrained send is one persuasive message away from mailing the inbox + * to whoever asked. An allowlist bounds where anything can go, without a person in the loop. + * + * It is a floor and not a substitute for the approval rule. Inside the allowed domains the Bot + * can still send whatever it was talked into, so a deployment that cares should have both. + */ + allowedRecipientDomains: ReadonlySet; +}; + /** * Who a deployment lets in, and through which front door. * @@ -258,6 +305,16 @@ export type DeploymentConfig = { * mounted and failing: a capability that is not configured should be missing, not broken. */ computer?: ComputerConfig; + /** + * The deployment's mailbox. Absent means the Mailbox tools refuse rather than fail. + * + * Unlike {@link DeploymentConfig.computer}, absence does not unmount anything: the catalogue entry + * stays admissible and its tools stay grantable, because a Bot's grants are an administrator's + * decision and should not evaporate because a variable was unset during a deploy. What absence + * changes is what a call answers: a sentence naming the four things to set, rather than a + * connection attempt to nowhere. + */ + mailbox?: MailboxConfig; /** How far one Bot handing work to another may go. */ handoff: HandoffCaps; /** @@ -819,6 +876,152 @@ function generativeUiEnabled(environment: Environment): boolean { return on === "true" || on === "1"; } +/** + * A port from the environment, or the protocol's default. + * + * Refused rather than coerced, the same as every other number in this file. A deployment that typed + * `993 ` with a stray character and silently got 993 anyway is fine; one that typed `9993` and got + * the default would be talking to the right host on the wrong port with nothing saying so. + */ +function mailboxPort( + environment: Environment, + name: string, + fallback: number, +): number { + const raw = optional(environment, name); + if (!raw) return fallback; + + const port = Number(raw); + if (!Number.isInteger(port) || port < 1 || port > 65535) { + throw new Error(`${name} must be a port number between 1 and 65535`); + } + return port; +} + +/** + * The deployment's mailbox, if it has one. + * + * Absent when none of the three are set, which is the ordinary state of a deployment that does not + * want this. Set one or two of them and it refuses to start naming what is missing, rather than + * booting with half a mailbox: a deployment with a host and no user is one where every mail tool + * fails at the first login, at run time, in front of somebody, and the only evidence is an auth + * failure from a server that will not say which half was wrong. + * + * The passwords are deliberately not read here. See {@link MailboxConfig}. + */ +function mailboxConfig(environment: Environment): MailboxConfig | undefined { + const imapHost = optional(environment, "MAILBOX_IMAP_HOST"); + const smtpHost = optional(environment, "MAILBOX_SMTP_HOST"); + const users = mailboxUsers(environment); + if (!imapHost && !smtpHost && users.length === 0) return undefined; + + const missing = [ + imapHost ? null : "MAILBOX_IMAP_HOST", + smtpHost ? null : "MAILBOX_SMTP_HOST", + users.length > 0 ? null : "MAILBOX_USERS", + ].filter((name): name is string => name !== null); + if (missing.length > 0) { + throw new Error( + `${missing.join(", ")} must be set as well: a mailbox needs an IMAP host, an SMTP host and at least one user. Unset all three to switch the mailbox off.`, + ); + } + + return { + imapHost: imapHost as string, + // Implicit TLS on both, rather than the STARTTLS ports. See MailboxConfig. + imapPort: mailboxPort(environment, "MAILBOX_IMAP_PORT", 993), + smtpHost: smtpHost as string, + smtpPort: mailboxPort(environment, "MAILBOX_SMTP_PORT", 465), + users, + allowedRecipientDomains: allowedRecipientDomains(environment), + }; +} + +/** + * The accounts on this deployment's mail hosts, in the order they were written. + * + * `MAILBOX_USERS` is the list and its first entry is the default account. `MAILBOX_USER`, the + * singular this feature shipped with, is still read as a list of one, so a deployment that already + * has one does not have to be edited to keep working. Both set is refused rather than merged or + * silently preferred: they are two answers to the same question, and a deployment holding both has + * an intention nobody here can read. + * + * Lower-cased and deduplicated, because an address is case-insensitive and the address is also the + * key of the vault credential holding that account's password. `Support@` and `support@` written as + * two entries are one mailbox, and treating them as two would be a second account nothing can ever + * unlock. + * + * An entry that is not an address refuses to start, naming it. It would otherwise become an account + * a model can select, a credential key an administrator cannot guess, and a login failure at run + * time in front of somebody. + */ +function mailboxUsers(environment: Environment): string[] { + const list = optional(environment, "MAILBOX_USERS"); + const legacy = optional(environment, "MAILBOX_USER"); + if (list && legacy) { + throw new Error( + "MAILBOX_USERS and MAILBOX_USER are both set and they are the same setting. Keep MAILBOX_USERS, which holds the whole list, and unset MAILBOX_USER.", + ); + } + + const name = list ? "MAILBOX_USERS" : "MAILBOX_USER"; + const entries = list + ? commaSeparated(environment, "MAILBOX_USERS") + : legacy + ? [legacy] + : []; + + const users: string[] = []; + for (const entry of entries) { + if (!/^[^\s@,]+@[^\s@,]+$/.test(entry)) { + throw new Error( + `${name} entry "${entry}" must be an email address such as bot@example.com`, + ); + } + const address = entry.toLowerCase(); + if (!users.includes(address)) users.push(address); + } + return users; +} + +/** + * Where mail from this deployment may go, read from the environment. + * + * Unset and empty are the same answer, and it is "anywhere": a deployment that has not said + * otherwise keeps the behaviour it had, and one that empties the variable has switched the + * restriction off rather than switched every send off. The opposite reading would turn a blanked + * line in a `.env` into a mailbox that silently refuses everybody. + * + * Lower-cased, because a domain is case-insensitive and an allowlist that misses `Example.test` is + * an allowlist somebody will debug at the wrong end. A leading `@` is accepted and dropped, since + * that is how a person writes a domain when they are thinking of addresses. + * + * Refused rather than ignored when an entry is not a domain. `MAILBOX_ALLOWED_RECIPIENT_DOMAINS` is + * a safety list, and one written as `sales@example.test` that silently matched nothing would be a + * deployment that believes it is restricted and is not. + */ +function allowedRecipientDomains( + environment: Environment, +): ReadonlySet { + const domains = commaSeparated( + environment, + "MAILBOX_ALLOWED_RECIPIENT_DOMAINS", + ).map((entry) => entry.replace(/^@/, "").toLowerCase()); + + for (const domain of domains) { + if ( + !/^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)*$/.test( + domain, + ) + ) { + throw new Error( + `MAILBOX_ALLOWED_RECIPIENT_DOMAINS entry "${domain}" must be a domain such as example.com, not an address or a URL`, + ); + } + } + return new Set(domains); +} + /** * How long the audit trail is kept. * @@ -861,6 +1064,7 @@ export function loadConfig( const auth = authConfig(environment, google); const managedAgent = managedAgentConfig(environment); const workerSharedSecret = optional(environment, "WORKER_SHARED_SECRET"); + const mailbox = mailboxConfig(environment); return { databaseUrl: required(environment, "DATABASE_URL"), @@ -894,6 +1098,7 @@ export function loadConfig( ? { appDistDir: optional(environment, "APP_DIST_DIR") as string } : {}), computer: computerConfig(environment), + ...(mailbox ? { mailbox } : {}), handoff: handoffCaps(environment), ...(optional(environment, "AGENT_TOOL_TOKEN") ? { agentToolToken: optional(environment, "AGENT_TOOL_TOKEN") as string } diff --git a/server/src/index.ts b/server/src/index.ts index 6828cb9a9..18c74c38c 100644 --- a/server/src/index.ts +++ b/server/src/index.ts @@ -57,12 +57,14 @@ import { import { createCredentialAdminService, createCredentialStore, + decryptCredentialForUse, resolveModelApiKey, } from "./credentials"; import { createDatabase } from "./db/client"; import { intelligenceChannelMappings } from "./db/schema"; import { createOnboardingStore } from "./people/onboarding"; import { createPeopleStore } from "./people/store"; +import { mailboxCredentialFor, useMailbox } from "./plugins/builtin-mailbox"; import { useRoutineTools } from "./plugins/builtin-routines"; import { redirectUriFor } from "./plugins/oauth"; import { createPluginStore } from "./plugins/store"; @@ -338,6 +340,50 @@ const pluginStore = createPluginStore({ const routineStore = createRoutineStore(database); useRoutineTools(routineStore); +/** + * The mailbox, and where its password comes from. + * + * Installed the same way and for the same reason as the routine store above: the transport is a + * module, so this is the one seam it has. + * + * WHAT IS INSTALLED IS A WAY TO READ A PASSWORD, NOT A PASSWORD. Nothing here decrypts anything at + * boot. The closure runs when a tool call needs the secret, so an administrator who rotates an + * account's credential is obeyed by the next call rather than by the next restart, and one who + * revokes it stops the tools within a call, because `decryptCredentialForUse` refuses a revoked + * row, which `builtin-mailbox` reports as a failure rather than treating as an absent password. + * + * ONE CREDENTIAL PER ACCOUNT, keyed by the address, so a deployment with several mailboxes on a + * shared host holds several rows and each is rotated and revoked on its own. + * + * `mcp` is the kind, which is the vault's name for "the one token this deployment holds for this + * server", and it is exactly what a mailbox password is here: one secret, held by the deployment, + * used for everybody, never anybody's own grant. The provider and key id are the connector's key, so + * an administrator stores it at `/admin/credentials` the same way they store a custom server's + * token. + * + * Nothing is installed when no mailbox is configured. The catalogue entry stays admissible and its + * tools stay grantable either way (see `DeploymentConfig.mailbox`), and a call then answers with + * the sentence naming what to set. + */ +useMailbox( + config.mailbox + ? { + config: config.mailbox, + password: async (account) => { + const held = await credentialStore.findLiveByKey( + mailboxCredentialFor(account), + ); + if (!held) return null; + return decryptCredentialForUse( + config.keyEncryptionKey, + credentialStore, + held.id, + ); + }, + } + : null, +); + /** * Where a Bot handing work to another gets decided. * diff --git a/server/src/mailbox/client.ts b/server/src/mailbox/client.ts new file mode 100644 index 000000000..64ba1f60e --- /dev/null +++ b/server/src/mailbox/client.ts @@ -0,0 +1,866 @@ +import { ImapFlow } from "imapflow"; +import { simpleParser } from "mailparser"; +import { createTransport, type Transporter } from "nodemailer"; +import MailComposer from "nodemailer/lib/mail-composer"; +import type { MailboxConfig } from "../config"; + +/** + * The only place in this deployment that speaks IMAP and SMTP. + * + * One door, for the same reason `plugins/mcp.ts` is one door for MCP: every call out carries the + * mailbox password and brings back text a model will read, so both directions want a single place to + * be careful in. A second client somewhere else would be a second place to forget the timeout, the + * size cap, or the rule that the password never appears in anything anybody reads. + * + * NO SESSION IS KEPT. A connection is opened, used and closed for every listing, every read, every + * search and every send. A pooled IMAP session would be a long-lived authenticated socket that + * several Bots' calls arrive on, which is the shape of bug `mcp.ts` declines to have. IMAP is worse + * for it than HTTP, because a session carries SELECTED-mailbox state, so one call's `mailbox` + * argument would silently decide what another call read. + * + * WHAT THIS MODULE IS NOT. It makes no decision about whether a call is allowed, whose it is, or + * what a model should be told. It speaks the two protocols and hands back plain values. + * `plugins/builtin-mailbox.ts` holds the tools, the argument checking and the sentences; the split + * is the same one Routines has between `routines/store.ts` and `plugins/builtin-routines.ts`. + */ + +/** + * How long a server gets before we give up on it. + * + * One number for connecting, for the greeting and for socket inactivity, because a mailbox that is + * slow in any of those three ways is a turn that is hanging, and the person is waiting either way. + * Well under the 60s an MCP call is given: a mail server that has said nothing for half a minute is + * not about to. + */ +const TIMEOUT_MS = 30_000; + +/** + * The wall clock one network operation gets, whatever it is doing. + * + * The three timeouts above are all INACTIVITY timeouts, which is a different promise: a server that + * sends one byte every twenty seconds resets all three forever, and the call it is holding open is a + * turn nobody can end. This one does not care whether bytes are arriving. When it expires the socket + * is closed rather than left to the garbage collector, because an abandoned IMAP connection is an + * authenticated socket that the server keeps until its own idle timer notices. + * + * Per operation rather than per tool call, and the difference is visible in exactly one place: + * `send_message` with `in_reply_to` fetches over IMAP and then sends over SMTP, so its worst case is + * two deadlines rather than one. Both end, which is the property that matters. + */ +const CALL_DEADLINE_MS = 60_000; + +/** + * The most of one message this deployment will pull off the wire. + * + * imapflow will happily accept a literal up to a gigabyte and `simpleParser` has no cap of its own, + * so without this a single mail with a large attachment is a gigabyte in this process's heap to + * produce at most {@link MAX_BODY_CHARS} of text. 512 KB is far more prose than the body cap can + * ever show, so the only thing it truncates is content that was never going to be read out. + * + * A truncated source is still parsed, because the headers and the first text part are at the front + * of a message and are what the tools want. That it was cut is carried out with the message and + * said in the answer rather than left to look like a message that ended there. + */ +export const MAX_SOURCE_BYTES = 512 * 1024; + +/** One message as a listing shows it. No body: a listing of ten bodies is not a listing. */ +export type MessageHeader = { + uid: number; + /** The From: header rendered as `Name
`, or just the address. */ + from: string; + to: string; + subject: string; + /** The message date as an ISO string, or null when the server sent none. */ + date: string | null; + seen: boolean; +}; + +/** One message, opened. */ +export type FullMessage = MessageHeader & { + /** + * The Message-ID header, which is the only thing that makes a reply thread. + * + * Null for a message that arrived without one. A reply to such a message is still a reply; it just + * cannot be threaded, and {@link OutgoingMessage.reply} is left off rather than filled with a + * guess that would thread it into the wrong conversation. + */ + messageId: string | null; + /** The References header, oldest first, as the reply has to repeat it. */ + references: readonly string[]; + /** The text of the message, already bounded by the caller's cap. */ + body: string; + /** How long the body was before the cap, so the caller can say it was cut. */ + bodyLength: number; + /** + * True when only the first {@link MAX_SOURCE_BYTES} of the message were read off the wire. + * + * Separate from a body that was merely long: this one says the deployment never saw the rest, so + * an answer that says "that is the whole message" would be wrong rather than abbreviated. + */ + sourceTruncated: boolean; + /** What the server says the whole message weighs, in bytes, when it says. */ + sizeBytes: number | null; +}; + +/** A message on its way out. */ +export type OutgoingMessage = { + to: string; + subject: string; + body: string; + /** + * The message this one replies to, when it is a reply. + * + * Both headers together, never one: `In-Reply-To` alone threads in some clients and not others, + * and `References` alone loses which message was actually answered. They are computed from a + * message that was fetched, so a reply is threaded against something that exists rather than + * against an id a model produced. + */ + reply?: { messageId: string; references: readonly string[] }; +}; + +/** + * One open IMAP connection, as the four tools need it. + * + * Deliberately narrow. Nothing a model calls has any business deleting a message, moving one, or + * setting a flag, so none of that is here. Same reasoning that keeps `RoutineTools` down to four of + * `RoutineStore`'s methods. + */ +export type MailboxSession = { + /** The newest `limit` messages in `mailbox`, newest first. */ + recent(mailbox: string, limit: number): Promise; + /** One message by uid, or null when that mailbox holds no such uid. */ + message(mailbox: string, uid: number): Promise; + /** Messages whose subject, sender or text matches, newest first, at most `limit`. */ + search(mailbox: string, query: string, limit: number): Promise; +}; + +/** + * Some messages, and how many there were to choose from. + * + * `total` is the whole point of the shape. A page of ten headers with nothing else said reads to a + * model as the whole mailbox, and it will answer "you have ten messages" about a mailbox holding + * four thousand. The count is free at both call sites (the mailbox's own `exists`, and the length of + * what SEARCH returned) and it is the difference between a listing and a claim. + */ +export type MessagePage = { + headers: MessageHeader[]; + /** How many messages the mailbox or the search had, before `limit` was applied. */ + total: number; +}; + +/** + * The two protocols, as something the tools can be tested against. + * + * A seam rather than a direct import, and it is the same seam `PluginStoreOptions.callVendor` is: + * the properties worth asserting about this module are which arguments a call went out with and + * what a model is told, and asserting either otherwise would need a reachable mail server, which + * means the properties most worth testing would be the ones never tested. + */ +export type MailboxClients = { + /** Open a connection, run the work, close it. Closed whatever happened. */ + withSession(use: (session: MailboxSession) => Promise): Promise; + send(message: OutgoingMessage): Promise; +}; + +/** + * What happened to one outgoing message: it went, and whether a copy was filed. + * + * TWO OUTCOMES RATHER THAN ONE, and keeping them apart is the whole point of the shape. SMTP + * delivery and the IMAP copy are separate operations against separate servers, and the second can + * fail after the first has succeeded. Reported as one failure, that is a Bot telling somebody the + * mail did not go and sending it again, which is the expensive mistake: mail cannot be recalled and + * the recipient gets it twice. + */ +export type SendReceipt = { + messageId: string | null; + /** The folder the copy was appended to, or null when no copy was filed. */ + filedTo: string | null; + /** Why no copy was filed, when none was. Null when one was. */ + fileError: string | null; +}; + +/** + * The message as bytes, built once. + * + * WHY RAW RATHER THAN LETTING NODEMAILER COMPOSE AT SEND TIME. What SMTP delivers and what IMAP + * stores have to be the same message, and composing twice is two messages: different `Message-ID`, + * different `Date`, different boundaries. A person looking at Sent in webmail would be reading a + * near-copy of what the recipient got, and a Bot trying to verify its own send by `Message-ID` + * would find nothing. + */ +export type ComposedMessage = { + raw: Buffer; + /** The SMTP envelope, so the delivery uses the same addresses the headers name. */ + envelope: ReturnType< + ReturnType["compile"]>["getEnvelope"] + >; + messageId: string; +}; + +/** + * Build one outgoing message into the bytes that will be both delivered and stored. + * + * Exported because it is worth asserting on its own: it is the only place the `From` is decided, + * and the only place the two threading headers are written. + */ +export async function composeMessage( + account: string, + message: OutgoingMessage, +): Promise { + const composed = new MailComposer({ + // The selected account, never an address from the arguments. A `from` a model could name is a + // Bot sending mail as somebody else. + from: account, + to: message.to, + subject: message.subject, + text: message.body, + ...(message.reply + ? { + inReplyTo: message.reply.messageId, + references: [...message.reply.references], + } + : {}), + }).compile(); + + return { + raw: await composed.build(), + envelope: composed.getEnvelope(), + // `compile` has already ensured one, so this is the id that is in the bytes above rather than a + // second one generated here. + messageId: composed.messageId(), + }; +} + +/** The special-use flag every IMAP server that has a Sent folder marks it with (RFC 6154). */ +const SENT_SPECIAL_USE = "\\Sent"; + +/** What a Sent folder is called on a server that marks nothing. Lower-cased. */ +const SENT_NAMES = new Set(["sent", "sent items", "sent messages"]); + +/** + * Which folder a copy of a sent message belongs in, out of what the server listed. + * + * SPECIAL-USE FIRST, because it is the answer that survives a localised server: a French account's + * Sent folder is `Éléments envoyés` and no list of English names will ever find it. The names are + * the fallback for the servers that mark nothing, and they are the three spellings in the wild. + * + * Null rather than a guess when neither finds one. Appending to the wrong folder is worse than not + * appending: it puts outgoing mail somewhere a person will read it as incoming. + */ +export function sentFolderFrom( + boxes: readonly { path: string; name?: string; specialUse?: string }[], +): string | null { + const marked = boxes.find((box) => box.specialUse === SENT_SPECIAL_USE); + if (marked) return marked.path; + + const named = boxes.find((box) => + SENT_NAMES.has((box.name ?? box.path).toLowerCase()), + ); + return named ? named.path : null; +} + +/** The most of one body that is ever read out of a message. See {@link readBody}. */ +export const MAX_BODY_CHARS = 8_000; + +export class MailboxError extends Error { + constructor(message: string) { + super(message); + this.name = "MailboxError"; + } +} + +/** `Name
`, or the address alone, or an empty string. Never `undefined` in a listing. */ +function addressLine( + addresses: readonly { name?: string; address?: string }[] | undefined, +): string { + if (!addresses || addresses.length === 0) return ""; + return addresses + .map((one) => { + const address = one.address ?? ""; + const name = one.name?.trim(); + return name ? `${name} <${address}>` : address; + }) + .join(", "); +} + +/** + * A header line out of what the server sent. + * + * Every field is optional in IMAP's envelope and several are optional in practice: a message with no + * Subject, no Date, or a From the server could not parse is an ordinary message, not an error. Empty + * strings rather than absent fields, so the rendering above this never has to branch. + */ +function headerOf(message: { + uid: number; + flags?: Set; + envelope?: { + date?: Date; + subject?: string; + from?: { name?: string; address?: string }[]; + to?: { name?: string; address?: string }[]; + }; +}): MessageHeader { + const envelope = message.envelope ?? {}; + return { + uid: message.uid, + from: addressLine(envelope.from), + to: addressLine(envelope.to), + subject: envelope.subject ?? "", + date: envelope.date ? new Date(envelope.date).toISOString() : null, + seen: message.flags?.has("\\Seen") ?? false, + }; +} + +/** + * HTML as something worth putting in front of a model, for a message that carries no text part. + * + * Deliberately crude, and only ever a fallback. `mailparser` already hands back `text` for anything + * multipart/alternative, which is nearly everything; what is left is the marketing mail that ships + * HTML alone, where the choice is between this and telling the model the message was empty. Scripts + * and styles go first, because their contents are not prose and would otherwise arrive as prose. + */ +export function strippedHtml(html: string): string { + return ( + html + .replace(/<(script|style)\b[^>]*>[\s\S]*?<\/\1>/gi, " ") + .replace(//gi, "\n") + .replace(/<\/(p|div|tr|li|h[1-6])>/gi, "\n") + .replace(/<[^>]+>/g, " ") + .replace(/ /gi, " ") + .replace(/&/gi, "&") + .replace(/</gi, "<") + .replace(/>/gi, ">") + .replace(/"/gi, '"') + .replace(/'/gi, "'") + .replace(/[ \t]+/g, " ") + // The space every dropped tag left behind, where it now sits at the start or end of a line. A + // stripper that keeps them indents the whole message by one space and nobody can see why. + .replace(/[ \t]*\n[ \t]*/g, "\n") + .replace(/\n{3,}/g, "\n\n") + .trim() + ); +} + +/** + * The readable text of a parsed message, and how long it was before the cap. + * + * The text part wins whenever there is one. Falling back to the HTML is for the message that has no + * text part at all, rather than for the one whose text part is short: a plain-text alternative that + * says "this message needs an HTML viewer" is still what the sender wrote, and preferring the HTML + * on length would swap the sender's own words out for markup on any short message. + */ +export function readBody(parsed: { text?: string; html?: string | false }): { + body: string; + bodyLength: number; +} { + const text = parsed.text?.trim(); + const source = + text && text.length > 0 + ? text + : typeof parsed.html === "string" + ? strippedHtml(parsed.html) + : ""; + return { + body: + source.length > MAX_BODY_CHARS ? source.slice(0, MAX_BODY_CHARS) : source, + bodyLength: source.length, + }; +} + +/** The References header as a list, oldest first. Whitespace-separated, per RFC 5322. */ +function referencesOf(value: string | string[] | undefined): string[] { + if (!value) return []; + const parts = Array.isArray(value) ? value : value.split(/\s+/); + return parts.map((one) => one.trim()).filter((one) => one.length > 0); +} + +/** + * The sentence a mail server actually wrote, out of the error that carries it. + * + * WHY THIS IS NOT `error.message`. imapflow answers every IMAP `NO` and `BAD` with an Error whose + * message is the fixed string "Command failed" and puts the server's own words on `responseText` + * (`imapflow/lib/imap-flow.js`). So the naive reading turns "Invalid credentials", "Mailbox does not + * exist" and "Over quota" into one useless sentence that names none of them, which is the opposite + * of the reason this deployment keeps the vendor's wording at all: each of those has a different fix + * and only the server can tell them apart. + * + * nodemailer is the other half of the same story, from the other protocol: it puts the SMTP reply on + * `response`. Both are read, and both are read only when they are strings, because imapflow also + * uses `response` for the parsed response OBJECT, and `String(anObject)` is `[object Object]`. + */ +export function mailServerSentence(error: unknown): string { + if (error !== null && typeof error === "object") { + const carried = error as { responseText?: unknown; response?: unknown }; + if ( + typeof carried.responseText === "string" && + carried.responseText.trim() !== "" + ) { + return carried.responseText.trim(); + } + if ( + typeof carried.response === "string" && + carried.response.trim() !== "" + ) { + return carried.response.trim(); + } + } + return error instanceof Error ? error.message : String(error); +} + +/** The most folder names one refusal will list. See {@link noSuchFolderSentence}. */ +export const MAX_FOLDERS_LISTED = 40; + +/** + * Whether the server refused because the folder is not there, rather than for any other reason. + * + * Only ever asked about a failed SELECT, which narrows it a lot: at that point the connection is up + * and authenticated, so the interesting failures are "no such folder" and permission. Every server + * words the first differently ("Mailbox doesn't exist: support", "[NONEXISTENT] Unknown Mailbox", + * "NO Mailbox does not exist"), so the wordings are matched rather than a code that not every + * server sends. + */ +function looksLikeMissingFolder(error: unknown): boolean { + const sentence = mailServerSentence(error).toLowerCase(); + return ( + sentence.includes("doesn't exist") || + sentence.includes("does not exist") || + sentence.includes("nonexistent") || + sentence.includes("unknown mailbox") || + sentence.includes("no such mailbox") || + sentence.includes("no such folder") + ); +} + +/** + * What a model is told when it asked for a folder that is not there. + * + * THE VENDOR'S OWN SENTENCE IS A TRAP HERE, which is why this is the one place the rule about + * keeping it is broken. "Mailbox doesn't exist: support" reads as a folder that happens to be + * missing, so a model retries with another folder name, and the live failure this exists for was + * exactly that: refused an address in `folder`, it tried `support`, then `webmaster`, which are the + * local parts of two configured accounts. The folders that DO exist, and one sentence saying the + * account is not chosen this way, turn a loop into a correction. + * + * Trimmed to fit the 400 characters a failure is capped at by `plugins/builtin-mailbox.ts`, from + * the end of the list rather than the end of the sentence: the closing instruction is the half that + * changes what the model does next, so it is the half that must survive. + */ +export function noSuchFolderSentence( + folder: string, + account: string, + folders: readonly string[], +): string { + const head = `No folder named ${folder} in ${account}.`; + const tail = + "The account is chosen by `account`, not by folder; leave folder unset for INBOX."; + + const candidates = folders.slice(0, MAX_FOLDERS_LISTED); + const room = 380 - head.length - tail.length; + const shown: string[] = []; + let used = " Folders here: .".length; + for (const name of candidates) { + if (used + name.length + 2 > room) break; + shown.push(name); + used += name.length + 2; + } + + const middle = + shown.length === 0 + ? "" + : ` Folders here: ${shown.join(", ")}${shown.length < folders.length ? ", and more" : ""}.`; + return `${head}${middle} ${tail}`; +} + +/** + * Run one network operation against a wall clock, and shut the socket if the clock wins. + * + * The cleanup is the point rather than the rejection. A `Promise.race` that only rejects leaves the + * losing work running, holding an authenticated connection this deployment has stopped waiting for; + * `onExpiry` is what makes the deadline mean the operation is over rather than merely unwatched. The + * timer is cleared in a `finally` so a fast call does not hold the process open for a minute. + */ +export async function withDeadline( + work: () => Promise, + onExpiry: () => void, + what: string, + // A parameter with a default rather than a constant read inside, so the behaviour can be asserted + // in a test that finishes in milliseconds instead of one that takes a minute to prove a minute. + deadlineMs: number = CALL_DEADLINE_MS, +): Promise { + let timer: ReturnType | undefined; + const expiry = new Promise((_resolve, reject) => { + timer = setTimeout(() => { + // Best effort: a socket that is already gone throws here, and that is not a failure of the + // deadline, which has already decided what this call answers. + try { + onExpiry(); + } catch {} + reject( + new MailboxError( + `The mail server did not finish ${what} within ${Math.round(deadlineMs / 1000)} seconds.`, + ), + ); + }, deadlineMs); + }); + + try { + // Both promises are raced, so the loser's rejection is handled here rather than surfacing later + // as an unhandled one. + return await Promise.race([work(), expiry]); + } finally { + if (timer) clearTimeout(timer); + } +} + +/** + * The real thing: imapflow for reading, nodemailer for sending. + * + * Built per call rather than once, because it holds the password. Nothing here caches a connection, + * a transport or the secret itself beyond the call that asked for it. + * + * `account` is one of `config.users`, already chosen and checked by the caller, and it is both what + * the two protocols authenticate as and what the mail is from. It is a parameter rather than a + * field of the configuration because a deployment's hosts are one thing and the account a + * particular call is working in is another: the same hosts serve every account, and which one this + * client speaks for is decided per call. + */ +export type MailboxWire = { + /** How an IMAP client is built. Defaults to imapflow. */ + imap?: (options: ConstructorParameters[0]) => ImapLike; + /** How an SMTP transport is built. Defaults to nodemailer. */ + smtp?: (options: SmtpOptions) => SmtpLike; +}; + +/** As much of imapflow as this module speaks. */ +export type ImapLike = Pick< + ImapFlow, + | "connect" + | "logout" + | "close" + | "getMailboxLock" + | "mailbox" + | "fetch" + | "fetchOne" + | "search" + | "list" + | "append" +>; + +/** As much of nodemailer as this module speaks. */ +export type SmtpLike = Pick; + +/** What this module asks an SMTP transport to be built with. */ +export type SmtpOptions = { + host: string; + port: number; + secure: boolean; + auth: { user: string; pass: string }; + connectionTimeout: number; + greetingTimeout: number; + socketTimeout: number; +}; + +export function createMailboxClients( + config: MailboxConfig, + account: string, + password: string, + /* + * Injected only by tests, and only for the one property that cannot be asserted otherwise: that + * the bytes handed to SMTP and the bytes appended to Sent are the same bytes. Everything else + * about this module is asserted through `MailboxAccess.clients`, one level up. + */ + wire: MailboxWire = {}, +): MailboxClients { + /** + * Build, use and close a client. + * + * The `finally` logs out whatever happened, because a thrown error is the case where a leaked + * connection is most likely and least noticed. `logout` is given the same swallow `mcp.ts` gives + * `close`: a server that will not say goodbye has not failed the work that just succeeded. + */ + async function withClient( + use: (client: ImapLike) => Promise, + what = "reading the mailbox", + ) { + const build = wire.imap ?? ((options) => new ImapFlow(options)); + const client = build({ + host: config.imapHost, + port: config.imapPort, + // Implicit TLS, always. STARTTLS is negotiated in the clear, so a server that answers without + // it gets this deployment's mailbox password over a plain socket. + secure: true, + auth: { user: account, pass: password }, + connectionTimeout: TIMEOUT_MS, + greetingTimeout: TIMEOUT_MS, + socketTimeout: TIMEOUT_MS, + /* + * No logging at all, and this is a decision rather than noise control. imapflow's logger + * writes the commands it sends, and one of them is LOGIN. A deployment that turned this on + * would put the mailbox password into whatever collects its stdout. + */ + logger: false, + // The connection lives for one call, so idling would only ever be a command the next call has + // to interrupt. + disableAutoIdle: true, + }); + + try { + return await withDeadline( + async () => { + await client.connect(); + return await use(client); + }, + // Closed rather than logged out: a deadline has expired because the server is not answering, + // and LOGOUT is another command to wait for. + () => client.close(), + what, + ); + } catch (error) { + // Rewrapped so a caller never has to care whether the failure came from the socket, the + // login or the command, and so what reaches an audit row is one sentence rather than a stack. + throw new MailboxError(mailServerSentence(error)); + } finally { + await client.logout().catch(() => { + client.close(); + }); + } + } + + /** + * Hold the mailbox lock for exactly the work that reads it. + * + * imapflow's own rule, and the reason it offers a lock at all: SELECT is connection state, so two + * pieces of work sharing a connection can otherwise read each other's mailbox. Released in a + * `finally` for the same reason the client is closed in one. + */ + async function withMailbox( + client: ImapLike, + mailbox: string, + use: () => Promise, + ): Promise { + let lock: Awaited>; + try { + lock = await client.getMailboxLock(mailbox); + } catch (error) { + /* + * A folder that is not there is answered with the folders that are, on the same connection + * and inside the same deadline. See noSuchFolderSentence for why the server's own words are + * not enough here. + */ + if (!looksLikeMissingFolder(error)) throw error; + let folders: string[] = []; + try { + folders = (await client.list()) + .map((box) => box.path) + .filter( + (path): path is string => typeof path === "string" && path !== "", + ); + } catch { + // A LIST that fails leaves the correction without its examples, which is still better than + // the vendor's sentence. Never a reason to lose the refusal itself. + } + throw new MailboxError(noSuchFolderSentence(mailbox, account, folders)); + } + + try { + return await use(); + } finally { + lock.release(); + } + } + + return { + async withSession(use: (session: MailboxSession) => Promise) { + return withClient(async (client) => { + const session: MailboxSession = { + async recent(mailbox, limit) { + return withMailbox(client, mailbox, async () => { + const open = client.mailbox; + const exists = open ? open.exists : 0; + if (exists === 0) return { headers: [], total: 0 }; + + /* + * The tail of the mailbox by SEQUENCE, which is what "newest" means here. + * + * Sequence numbers are ordered by arrival and are contiguous, so the last `limit` of + * them is one range and one round trip. Uids are neither ordered by arrival in any + * guaranteed way nor contiguous, so asking for "the highest N uids" would be a fetch + * of the whole mailbox first. + */ + const first = Math.max(1, exists - limit + 1); + const headers: MessageHeader[] = []; + for await (const message of client.fetch(`${first}:${exists}`, { + uid: true, + envelope: true, + flags: true, + })) { + headers.push(headerOf(message)); + } + // Newest first, which is the order a person reads a mailbox in. `exists` is the + // whole mailbox, so the caller can say ten of four thousand rather than ten. + return { headers: headers.reverse(), total: exists }; + }); + }, + + async message(mailbox, uid) { + return withMailbox(client, mailbox, async () => { + const message = await client.fetchOne( + String(uid), + { + uid: true, + envelope: true, + flags: true, + // Bounded on the WIRE, not after the fact. See MAX_SOURCE_BYTES. + source: { maxLength: MAX_SOURCE_BYTES }, + // What the whole message weighs, which is the only way to tell a message that + // ended at the cap from one that happens to be exactly that long. + size: true, + }, + { uid: true }, + ); + // `false` is imapflow's "no such message", and it is a different fact from a + // message that arrived without a source. Compared rather than falsy-checked so the + // type narrows: `message?.source` leaves `false` in the union below. + if (message === false || !message.source) return null; + + const parsed = await simpleParser(message.source); + const { body, bodyLength } = readBody(parsed); + const sizeBytes = + typeof message.size === "number" ? message.size : null; + return { + ...headerOf(message), + messageId: parsed.messageId ?? null, + references: referencesOf(parsed.references), + body, + bodyLength, + /* + * Either the server said the message is bigger than what we asked for, or it did not + * say and we got exactly the cap. The second is a message that MIGHT be exactly + * 512 KB, and saying it was cut when it was not is the better of the two errors: it + * understates what we hold rather than overstating it. + */ + sourceTruncated: + sizeBytes !== null + ? sizeBytes > MAX_SOURCE_BYTES + : message.source.length >= MAX_SOURCE_BYTES, + sizeBytes, + }; + }); + }, + + async search(mailbox, query, limit) { + return withMailbox(client, mailbox, async () => { + /* + * One IMAP SEARCH over the three fields a person means by "find the mail about X". + * `text` already covers headers and body, and subject and from are named beside it + * anyway: a server that indexes headers separately answers those far faster, and the + * OR of the three is what an unindexed server would have scanned regardless. + */ + const uids = await client.search( + { + or: [{ subject: query }, { from: query }, { text: query }], + }, + { uid: true }, + ); + if (!uids || uids.length === 0) return { headers: [], total: 0 }; + + // Highest uid last, so the tail is the newest matches. Reversed after fetching so the + // answer reads newest first, like the listing. + const wanted = uids.slice(-limit); + const headers: MessageHeader[] = []; + for await (const message of client.fetch( + wanted, + { uid: true, envelope: true, flags: true }, + { uid: true }, + )) { + headers.push(headerOf(message)); + } + // How many matched, not how many are being shown. A search that found 300 and is + // answering with 20 has to be able to say so. + return { headers: headers.reverse(), total: uids.length }; + }); + }, + }; + + return use(session); + }); + }, + + async send(message) { + /* + * COMPOSED ONCE, USED TWICE. `raw` goes to SMTP and the same `raw` is appended to Sent, so + * what the recipient holds and what the account's Sent folder holds are byte for byte the + * same message, down to the Message-ID a Bot would verify its own send by. + */ + const composed = await composeMessage(account, message); + + const build = + wire.smtp ?? ((options) => createTransport(options) as SmtpLike); + const transport = build({ + host: config.smtpHost, + port: config.smtpPort, + // Implicit TLS on 465, for the same reason IMAP uses it: SMTP AUTH sends the password. + secure: true, + auth: { user: account, pass: password }, + connectionTimeout: TIMEOUT_MS, + greetingTimeout: TIMEOUT_MS, + socketTimeout: TIMEOUT_MS, + }); + + try { + await withDeadline( + () => + transport.sendMail({ + envelope: composed.envelope, + raw: composed.raw, + }), + () => transport.close(), + "sending the message", + ); + } catch (error) { + throw new MailboxError(mailServerSentence(error)); + } finally { + transport.close(); + } + + const filed = await fileInSent(composed.raw); + return { + messageId: composed.messageId, + filedTo: filed.folder, + fileError: filed.error, + }; + }, + }; + + /** + * Put a copy of a message that has already gone out into the account's Sent folder. + * + * AFTER THE SEND AND NEVER IN FRONT OF IT. The mail is already delivered by the time this runs, + * so nothing it does can stop or duplicate a delivery, and nothing it fails at is a failed send. + * That is why every failure here is caught and RETURNED rather than thrown: a throw would reach + * the tools as "the send failed", and a Bot told that resends a message that cannot be recalled. + * + * Why it is needed at all: SMTP delivers, it does not file. Without this the account's Sent + * folder stays empty, webmail shows nothing sent, and a person checking concludes the mail was + * never sent. That happened, to three messages that had all been delivered. + * + * On its own connection, inside the same per-operation deadline as everything else here, and + * marked `\\Seen` because the account did not receive this message, it wrote it. + */ + async function fileInSent( + raw: Buffer, + ): Promise<{ folder: string | null; error: string | null }> { + try { + return await withClient(async (client) => { + const folder = sentFolderFrom(await client.list()); + if (!folder) { + return { folder: null, error: "this account has no Sent folder" }; + } + await client.append(folder, raw, ["\\Seen"], new Date()); + return { folder, error: null }; + }, "filing the copy in Sent"); + } catch (error) { + return { folder: null, error: mailServerSentence(error) }; + } + } +} diff --git a/server/src/plugins/builtin-mailbox.ts b/server/src/plugins/builtin-mailbox.ts new file mode 100644 index 000000000..c2353ddac --- /dev/null +++ b/server/src/plugins/builtin-mailbox.ts @@ -0,0 +1,1048 @@ +import type { MailboxConfig } from "../config"; +import { + createMailboxClients, + type FullMessage, + MAX_BODY_CHARS, + MAX_SOURCE_BYTES, + type MailboxClients, + MailboxError, + type MessageHeader, + type MessagePage, + type OutgoingMessage, +} from "../mailbox/client"; +import { MAX_RESULT_CHARS, type McpCallResult, type McpTool } from "./mcp"; + +/** + * The builtin transport for the Mailbox: a Bot reading and answering the deployment's mail, without + * leaving the building. + * + * WHAT MAKES THIS DIFFERENT FROM ROUTINES, the other builtin. Routines has no credential at all, + * because it acts on this deployment's own tables and the ACTOR is the authorization. This one does + * have a credential, a mailbox password, and it is the deployment's rather than anybody's: there is one + * mailbox, every Bot granted these tools reads the same one, and no person consents to it. So the + * authorization here is the GRANT: an administrator decides which Bots may read the mail and which + * may answer it, per tool, on the Plugins page, exactly as they would for a vendor's connector. + * + * That is why nothing below reads `connection.actorId` as permission. A run that reaches here has + * already passed the grant check and the policy decision in `plugins/store.ts`, and there is no + * per-person narrowing left to do: the mailbox does not have somebody's half. + * + * THE PASSWORD IS NEVER IN THE ENVIRONMENT and never in an answer. The hosts and the accounts come + * from `config.ts`; each account's password comes from the encrypted credential vault, as that + * account's own credential, resolved through {@link MailboxAccess.password} at the moment a call + * needs it and thrown away after. {@link redacted} is the last line of that: a failure sentence + * from a mail server that echoed the login back is scrubbed before anybody reads it. + * + * SEVERAL ACCOUNTS, ONE DEPLOYMENT. `support@`, `sales@` and `billing@` on the same shared host are + * one configuration with one host pair and a password each. Which one a call works in is the + * `account` argument, defaulting to the first configured. It changes nothing about the access + * model: the accounts are all the deployment's, the grant is still per tool rather than per + * account, and a Bot granted these tools reaches every one of them. + * + * It implements the same interface as every other transport, as module-level exports, because that + * is the shape {@link ./transport} resolves: a `TransportKind` maps to a MODULE. Which is also why + * the configuration and the vault arrive through {@link useMailbox} rather than a constructor: the + * registry is built at import time, long before `index.ts` has either. + */ + +/** + * What the tools act on: where the mailbox is, how to unlock it, and how the protocols are spoken. + * + * `password` is a function rather than a string because a secret read once at boot is a secret held + * in memory for the life of the process and stale the moment an administrator rotates it. Read per + * call, a rotation takes effect on the next call and a revocation refuses it. + */ +export type MailboxAccess = { + config: MailboxConfig; + /** + * That account's password from the vault, or null when this deployment holds none for it. + * + * Per account rather than per deployment, because a shared host gives each mailbox its own + * login. A deployment can hold the password for one account and not another, and that is a + * working deployment for the account it has: the answer names the account that is missing one. + */ + password: (account: string) => Promise; + /** + * How IMAP and SMTP are actually spoken. Defaults to imapflow and nodemailer. + * + * Injected so a test can assert what a call was about to go out with, which is the same reason + * `PluginStoreOptions.callVendor` exists: the reply headers, the caps and the argument checking + * are the properties worth being sure about, and asserting them otherwise would need a reachable + * mail server. + */ + clients?: ( + config: MailboxConfig, + account: string, + password: string, + ) => MailboxClients; +}; + +/** + * Which credential in the vault is one account's password. + * + * Here rather than at the one call site in `index.ts`, because it is a contract with two other + * parties: the administrator who types these three values at `/admin/credentials`, and + * `docs/mailbox.md`, which tells them to. Three strings agreeing across three places by convention + * is how a deployment ends up holding the right secret under a key nothing reads. + * + * THE KEY ID IS THE ADDRESS, which is what makes several accounts possible at all: one credential + * per configured address, so a rotation, a revocation and a missing password are each one + * account's rather than the whole mailbox's. Lower-cased on the way in, matching what `config.ts` + * stores, so an administrator who typed `Support@` and a caller who asked for `support@` reach the + * same row. + * + * `mcp` is the kind because that is the vault's name for "the one token this deployment holds for + * this server", which is the same kind a custom MCP server's own bearer token is stored under and + * the same thing a mailbox password is: one secret, the deployment's, used for every Bot granted + * the tools. + */ +export function mailboxCredentialFor(account: string): { + kind: "mcp"; + provider: "mailbox"; + keyId: string; +} { + return { kind: "mcp", provider: "mailbox", keyId: account.toLowerCase() }; +} + +let installed: MailboxAccess | null = null; + +/** + * Hand this module the mailbox, once, from the place that has the configuration and the vault. + * + * A module-level binding rather than a constructor argument, for the reason {@link + * ./builtin-routines.useRoutineTools} gives: `transportFor` resolves a kind to a MODULE and there is + * no seam to pass anything through. `null` is a supported argument, and not only for symmetry: the + * suite is one process, so a test that installs a stub has to be able to take it back out again, + * and a deployment with no mailbox configured installs nothing. + */ +export function useMailbox(access: MailboxAccess | null): void { + installed = access; +} + +/** The IMAP folder a tool reads when the call did not name one. */ +const DEFAULT_FOLDER = "INBOX"; + +/** Bounds on how much mail one call may return. See {@link boundedLimit}. */ +const LIST_LIMIT = { fallback: 10, max: 50 } as const; +const SEARCH_LIMIT = { fallback: 20, max: 50 } as const; + +/** + * The largest number that can be a uid, which is what IMAP's own 32-bit field allows (RFC 3501). + * + * Checked here so a model that produced `1e21` is told there is no such message, in the same words + * as any other uid that is not there. Without it the number reaches imapflow, which refuses to + * compile a sequence set out of it, and the model is handed "Invalid sequence set value" about an + * argument it thinks of as a message number. + */ +const MAX_UID = 4_294_967_295; + +/** + * What a call answers when this deployment has no mailbox configured. + * + * Names all four things, including the one that is not an environment variable, because the half a + * reader is most likely to be missing is the half that is not in `.env`. The catalogue entry stays + * admissible and grantable without any of it (see `DeploymentConfig.mailbox`), so this sentence, + * rather than a missing connector, is how a deployment finds out. + */ +const NOT_CONFIGURED = + "Mailbox is not configured. Set MAILBOX_IMAP_HOST, MAILBOX_SMTP_HOST, MAILBOX_USERS and store each account's password as its mailbox credential."; + +/** + * What a call answers when the hosts are configured and the vault holds no password for the account. + * + * Its own sentence rather than the one above, because it is a different job with a different fix: an + * administrator has already set the three variables and has one step left, at a different screen. + * Telling them to set variables they can see are already set is how a correct instruction gets read + * as a broken deployment. + * + * IT NAMES THE ACCOUNT, and with several of them that is the whole content of the message. A + * deployment that stored `support@` and forgot `billing@` is working for one account and broken for + * the other, and a sentence saying only "the mailbox" would send an administrator to look at the + * credential that is already there. + */ +function noPassword(account: string): string { + return `This deployment holds no password for the mailbox ${account}. An administrator has to store it as that account's mailbox credential (kind \`mcp\`, provider \`mailbox\`, key id \`${account}\`) before mail can be read or sent.`; +} + +/** + * The four tools, as the same shape a server would have answered `tools/list` with. + * + * THE DESCRIPTIONS CARRY THE THINGS A MODEL CANNOT SEE. Two of them matter enough to be spelled out + * rather than implied by a field name. The first is that a uid belongs to one mailbox: a uid read + * out of a listing of `Archive` names a different message in `INBOX`, and a model that carries one + * across will confidently open the wrong mail. The second is that this is ONE shared mailbox rather + * than the mailbox of whoever is asking. A Bot that believes it is reading the person's own mail + * will summarize somebody else's inbox to them without either party ever saying so. + */ +/** + * The `folder` argument, in the one wording all four tools use. + * + * IT SAYS WHAT IT IS NOT, and that sentence is the whole point of this constant. A smaller model + * given two arguments that both read as "which mailbox" puts the email address in the wrong one: + * observed live, a Bot passed `support@example.com` as the folder and was answered "Character not + * allowed in mailbox name" by the IMAP server, then "Mailbox doesn't exist: support", and never + * tried `account` at all. Naming the folder `folder`, saying it is not an address, and pointing at + * the argument that is one costs three lines here and saves a run. + */ +const FOLDER_ARGUMENT = Object.freeze({ + type: "string", + description: `IMAP folder to read, such as ${DEFAULT_FOLDER}, Sent or Archive. Default ${DEFAULT_FOLDER}. Leave it unset unless the person names a folder. This is not an email address, and it is not the part before the @ of one; to choose the account, use \`account\`.`, +} as const); + +const TOOLS: readonly McpTool[] = Object.freeze([ + { + name: "list_messages", + description: [ + "List the newest messages in this deployment's mailbox, newest first: each one's uid, when it", + "arrived, who sent it, its subject and whether it has been read. No bodies: open one with", + "`read_message`.", + "", + "This is a single shared mailbox belonging to the deployment, not the mailbox of the person you are", + "talking to. Say whose it is if it is not obvious from the conversation, and never present its", + "contents as their own mail.", + "", + "`uid` values are per folder. A uid from a listing of one folder names a different message in", + "another, so pass the same `folder` to `read_message` that you listed.", + ].join("\n"), + inputSchema: { + type: "object", + properties: { + limit: { + type: "integer", + description: `How many messages to list. Default ${LIST_LIMIT.fallback}, at most ${LIST_LIMIT.max}.`, + }, + folder: FOLDER_ARGUMENT, + }, + }, + }, + { + name: "read_message", + description: [ + "Open one message and read it: its headers and its text, by the uid from `list_messages` or", + "`search_messages`.", + "", + "A long message is cut off and says so. Nothing is marked read, moved or deleted by opening it.", + "", + "Pass the same `folder` the uid came from. Uids are per folder, so a uid listed in one names a", + "different message in another.", + ].join("\n"), + inputSchema: { + type: "object", + properties: { + uid: { + type: "integer", + description: "The message's uid, from a listing or a search.", + }, + folder: FOLDER_ARGUMENT, + }, + required: ["uid"], + }, + }, + { + name: "search_messages", + description: [ + "Find messages whose subject, sender or text matches `query`, newest first. Answers with the same", + "header lines `list_messages` does; open one with `read_message`.", + "", + "The match is the mail server's own, which is a plain substring search rather than a search engine:", + "one or two distinctive words find more than a sentence does, and there is no ranking, no stemming", + "and no boolean syntax. A search that finds nothing says so.", + ].join("\n"), + inputSchema: { + type: "object", + properties: { + query: { + type: "string", + description: + "What to look for, matched against subject, sender and message text.", + }, + limit: { + type: "integer", + description: `How many matches to return. Default ${SEARCH_LIMIT.fallback}, at most ${SEARCH_LIMIT.max}.`, + }, + folder: FOLDER_ARGUMENT, + }, + required: ["query"], + }, + }, + { + name: "send_message", + description: [ + "Send mail from this deployment's mailbox. It goes out immediately and there is nothing to recall,", + "so read what you are about to send back to the person first whenever the wording is theirs to", + "approve.", + "", + "To answer a message rather than start a new conversation, give `in_reply_to` as the uid of the", + "message you are answering. The reply is then threaded properly in the recipient's mail client, and", + '"Re: " is put in front of the subject if it is not already there. Without it the message opens a new', + "thread, however the subject is worded.", + "", + "The sender is always the account this call works in, which is one of the deployment's own", + "mailboxes. There is no field for it, and the mail says who it is from, so do not sign it as", + "somebody else.", + "", + "Some deployments only allow mail to certain domains. If this is one of them, a recipient outside", + "them is refused, nothing is sent, and the refusal names the domain: report that plainly rather", + "than trying another address.", + ].join("\n"), + inputSchema: { + type: "object", + properties: { + to: { + type: "string", + description: + "The recipient's address. Several are separated by commas.", + }, + subject: { type: "string", description: "The subject line." }, + body: { + type: "string", + description: "The message itself, as plain text.", + }, + in_reply_to: { + type: "integer", + description: + "The uid of the message this answers, so the reply threads. Omit for a new conversation.", + }, + folder: FOLDER_ARGUMENT, + }, + required: ["to", "subject", "body"], + }, + }, +]); + +/** + * Who this call is for, and which Bot is making it. + * + * The whole shared connection shape, all of it unused: there is no host to dial from a URL, no token + * to send, and the actor is not what authorizes this one. Declared anyway because it is the + * transport interface, and named here so the reason is written down where somebody would look for + * a missing check. + */ +type Connection = { + url: string; + token?: string; + actorId?: string; + botId?: string; +}; + +/** + * The list is static and needs neither a credential nor a configured mailbox. + * + * The four definitions are schemas in this file: nothing to discover, nobody to ask. Listing them + * without a mailbox configured is deliberate rather than an oversight. An administrator sets a + * connector up and grants its tools before, or instead of, the deployment ever having the secret, + * and a tool list that emptied itself when a variable was unset would revoke grants by accident. + */ +export async function listTools(): Promise { + const users = installed?.config.users ?? null; + return TOOLS.map((tool) => withAccount(tool, users)); +} + +/** + * One tool definition, plus the `account` argument and the sentence explaining it. + * + * ADDED HERE RATHER THAN WRITTEN INTO {@link TOOLS} because the choices are a deployment's, not this + * file's. A model that is told the addresses can pick one; a model told only that an `account` + * argument exists has to guess at a string, and a guessed address is a refusal at best. The names + * are already in front of it in every listing this connector answers with, so naming them in the + * description reveals nothing a granted Bot could not already read. + * + * The generic wording is for the deployment with no mailbox configured, where the list is still + * answered (see {@link listTools}) and there are no addresses to name yet. + */ +function withAccount(tool: McpTool, users: readonly string[] | null): McpTool { + const configured = users && users.length > 0 ? users : null; + const oneOf = configured + ? `One of: ${configured.join(", ")}.` + : "One of the deployment's configured addresses."; + const fallback = configured + ? `Default ${configured[0]}.` + : "Default is the first of them."; + + /* + * ACCOUNT FIRST, deliberately. A model reads a schema in order, and the argument it meets first is + * the one it reaches for when it wants to say "the support mailbox". Meeting `folder` first is how + * an address, and then the local part of an address, ends up in it. + */ + const properties = { + account: { + type: "string", + description: `Which mailbox account to use, as its email address. ${oneOf} ${fallback}`, + }, + ...((tool.inputSchema.properties as Record | undefined) ?? + {}), + }; + + return { + ...tool, + description: [ + tool.description, + "", + `\`account\` is the email address of the mailbox to work in. ${oneOf} ${fallback} It is a different argument from \`folder\`, which names an IMAP folder such as ${DEFAULT_FOLDER}: an address belongs in \`account\` and never in \`folder\`. A uid, like a folder, belongs to one account, so do not carry one across.`, + ].join("\n"), + inputSchema: { ...tool.inputSchema, properties }, + }; +} + +export const listNeedsCredential = false; + +const failure = (message: string): McpCallResult => ({ + text: message, + isError: true, + truncated: false, +}); + +/** Success as a result, with the same visible cap the vendor transports use. */ +function asResult(text: string): McpCallResult { + if (text.length <= MAX_RESULT_CHARS) { + return { text, isError: false, truncated: false }; + } + return { + text: `${text.slice(0, MAX_RESULT_CHARS)}\n\n[truncated: the tool returned ${text.length} characters]`, + isError: false, + truncated: true, + }; +} + +/** A string argument that was actually given, or nothing. Blank is not a value. */ +function stringArg( + args: Record, + key: string, +): string | undefined { + const value = args[key]; + return typeof value === "string" && value.trim() !== "" + ? value.trim() + : undefined; +} + +/** + * A whole number argument, or nothing, or a refusal. + * + * A string of digits counts. Models produce `"42"` for an integer field often enough that refusing + * it would be refusing a correct intention over a JSON type, and there is nothing ambiguous about + * it. Anything else that is not a positive whole number is refused rather than rounded or coerced: + * a uid is an identity, and `12.7` silently becoming 12 is a different message. + */ +function integerArg( + args: Record, + key: string, +): { value?: number; error?: string } { + const raw = args[key]; + if (raw === undefined || raw === null || raw === "") return {}; + + const value = + typeof raw === "number" + ? raw + : typeof raw === "string" + ? Number(raw.trim()) + : Number.NaN; + if (!Number.isInteger(value) || value < 1) { + return { error: `\`${key}\` has to be a whole number, at least 1.` }; + } + return { value }; +} + +/** + * How many messages one call returns, and whether the ask was cut down. + * + * Capped rather than refused, which is the opposite of what this file does with a malformed number, + * and the difference is what the number means. A malformed uid is a mistake only the model can fix; + * "give me 200 messages" is a perfectly clear intention that this tool simply does not serve, and + * refusing it would cost a whole extra round trip to be told a smaller number. The cap is SAID, in + * the answer, so a model that asked for 200 and got 50 knows there may be more rather than + * concluding the mailbox holds fifty messages. + */ +export function boundedLimit( + asked: number | undefined, + bounds: { fallback: number; max: number }, +): { limit: number; capped: boolean } { + if (asked === undefined) return { limit: bounds.fallback, capped: false }; + if (asked > bounds.max) return { limit: bounds.max, capped: true }; + return { limit: asked, capped: false }; +} + +/** + * How a mailbox is named in an answer: the folder, and which account's folder it is. + * + * One function so every sentence says it the same way. The account is in all of them rather than + * only in the ambiguous ones, because a model reading "showing 10 of 236 messages in INBOX" across + * two accounts in one turn has no way to tell which INBOX either page came from, and will merge + * them. + * + * The word "folder" is in it for the model rather than for the reader: every answer this connector + * gives then models the vocabulary the arguments use, so the thing before "of" reads as a folder + * and the thing after it reads as an account, in the same sentence. + */ +function where(folder: string, account: string): string { + return `folder ${folder} of ${account}`; +} + +/** + * The sentence for a uid that names nothing, wherever it was noticed. + * + * One function because there are two ways to arrive at it and they are the same fact to whoever is + * reading: the mailbox answered with no such message, or the number was never one a mailbox could + * hold. A model told two different things about one situation will try two different fixes. + */ +function noSuchMessage(uid: number, place: string): string { + return `There is no message with uid ${uid} in ${place}. Uids are per folder, so check the listing this one came from.`; +} + +/** + * The lines that go under a listing when there is more than it showed. + * + * Two separate facts, and a call can have both. "There are 4000 messages and you are seeing 10" is + * about the mailbox; "50 is the most this tool will list" is about the tool, and only appears when + * somebody asked for more than that. Said, because a page of ten headers with nothing else on it + * reads to a model as the whole mailbox, and it will answer "you have ten messages" about a mailbox + * holding four thousand. + */ +function pageNotes( + page: MessagePage, + capped: boolean, + max: number, + what: string, +): string[] { + const notes: string[] = []; + if (page.total > page.headers.length) { + notes.push( + `[showing ${page.headers.length} of ${page.total} ${what}, newest first.]`, + ); + } + if (capped) { + notes.push( + `[${max} is the most this tool lists at once, so there may be more than these.]`, + ); + } + return notes.length > 0 ? ["", ...notes] : []; +} + +/** One message as a line in a listing. Empty fields are named, never left blank. */ +function headerLine(header: MessageHeader): string { + return [ + `uid ${header.uid}`, + header.date ?? "no date", + `from ${header.from || "an unnamed sender"}`, + `"${header.subject || "(no subject)"}"`, + header.seen ? "read" : "unread", + ].join(" · "); +} + +/** + * One message, opened. + * + * The cut is stated in the body's own terms (how long it was, where it stopped) rather than as a + * generic truncation note, because the model's next move depends on it: a message cut at 8000 of + * 9000 characters has almost certainly said what it came to say, and one cut at 8000 of 400000 has + * not. Same reasoning as `mcp.ts`'s cap note, applied one level down, since the whole result is + * capped again above this. + */ +function messageInWords(message: FullMessage, place: string): string { + const lines = [ + `uid ${message.uid} in ${place}`, + `From: ${message.from || "an unnamed sender"}`, + `To: ${message.to || "nobody named"}`, + `Date: ${message.date ?? "not stated"}`, + `Subject: ${message.subject || "(no subject)"}`, + "", + message.body || "This message has no readable text.", + ]; + if (message.bodyLength > MAX_BODY_CHARS) { + lines.push( + "", + `[truncated: the message body is ${message.bodyLength} characters and the first ${MAX_BODY_CHARS} are shown]`, + ); + } + /* + * A different fact from a long body, and it has to be said separately. + * + * A cut body means this deployment holds the whole message and is showing part of it. A cut + * SOURCE means it never read the rest off the wire, so what is missing is missing everywhere: + * later parts, attachments, and anything a model might otherwise offer to go back for. + */ + if (message.sourceTruncated) { + const weight = + message.sizeBytes === null + ? "" + : ` of the message's ${message.sizeBytes} bytes`; + lines.push( + `[only the first ${MAX_SOURCE_BYTES} bytes${weight} were read, so anything later in it, including attachments, was not seen]`, + ); + } + return lines.join("\n"); +} + +/** + * The subject and the two headers that make a reply a reply. + * + * WHY THE ORIGINAL IS FETCHED RATHER THAN NAMED. Threading is done on `Message-ID`, and a model + * cannot know one: it is a header a person never sees. Given the uid of a message that exists, the + * ids come from the message itself, so a reply is threaded against something real or is not + * threaded at all. + * + * `References` is the original's own chain with the original appended, which is RFC 5322 §3.6.4's + * rule and the thing mail clients actually walk. A message with no `Message-ID` gets neither header: + * an `In-Reply-To` pointing at nothing is not a reply, and inventing an id would thread the answer + * into a conversation that does not exist. + * + * The subject keeps whatever the caller wrote and only gains a prefix. A reply marker already + * there is left alone rather than stacked, and the check is not just for English: `AW:` is German, + * `SV:` Swedish, `Antw:` Dutch and `Ref:` Italian, and a client that only knew `Re:` is how a thread + * ends up titled `Re: AW: Re: AW: the numbers`. Case-insensitive, and tolerant of the space some + * clients put before the colon. + */ +const REPLY_PREFIX = /^(re|aw|sv|antw|ref)\s*:/i; +export function replyFrom( + original: Pick, + subject: string, +): { subject: string; reply?: OutgoingMessage["reply"] } { + const prefixed = REPLY_PREFIX.test(subject.trim()) + ? subject + : `Re: ${subject}`.trim(); + if (!original.messageId) return { subject: prefixed }; + return { + subject: prefixed, + reply: { + messageId: original.messageId, + references: [...original.references, original.messageId], + }, + }; +} + +/** + * A sentence with the password taken out of it, in every spelling it can appear in. + * + * Belt and braces over a rule already kept: imapflow is built with `logger: false` and nothing in + * this deployment prints the secret. What this covers is the sentence a SERVER wrote. Both protocols + * quote the offending command back on a failed login, and nodemailer appends the raw SMTP reply to + * its error message, so the credential can arrive here inside somebody else's words. That text goes + * into an audit row and in front of a model, and neither is a place for the mailbox password. + * + * THE BASE64 FORMS ARE NOT PARANOIA, they are the common case. Neither client prefers plaintext + * `LOGIN`: imapflow authenticates with `AUTH=PLAIN` when the server offers it and falls back to + * `AUTH=LOGIN`, and both put the credential on the wire base64-encoded. So a quoted command carries + * `base64("\0user\0password")` or `base64(password)` rather than the password as typed, and a + * redaction that only knew the plaintext would pass it through unchanged while looking like it + * worked. + * + * A short or empty password is skipped rather than replaced everywhere, since replacing a + * one-character string would redact half the alphabet out of an unrelated message. + */ +export function redacted( + message: string, + password: string | null, + user?: string, +): string { + if (!password || password.length < 4) return message; + + const base64 = (value: string) => + Buffer.from(value, "utf8").toString("base64"); + const forms = [ + password, + // AUTH=LOGIN sends the password on its own line. + base64(password), + // AUTH=PLAIN sends authzid, authcid and password as one NUL-separated blob. + ...(user ? [base64(`\u0000${user}\u0000${password}`)] : []), + ]; + + let scrubbed = message; + for (const form of forms) { + if (form.length < 4) continue; + scrubbed = scrubbed.split(form).join("[redacted]"); + } + return scrubbed; +} + +/** + * The recipient domains this deployment will not send to, out of a `to` field. + * + * Returned rather than thrown, and returned as the DOMAINS rather than as a yes or no, because the + * refusal has to name what was wrong: a model told only that the recipient was refused will try + * another address, and one told the domain will say plainly that this deployment does not mail + * outside it. + * + * `to` is what the model wrote, so this parses defensively: comma-separated, each part either a bare + * address or `Name
`. Anything with no `@`, or nothing after it, is reported as an offender + * too, because an unparseable recipient is not a recipient this list has cleared, and letting it + * through to be somebody else's validation error would be a hole in a safety check. + * + * An empty allowlist means unrestricted and is answered before any parsing. + */ +export function refusedRecipients( + to: string, + allowed: ReadonlySet, +): string[] { + if (allowed.size === 0) return []; + + const refused: string[] = []; + for (const part of to.split(",")) { + const trimmed = part.trim(); + if (trimmed === "") continue; + const angled = /<([^>]*)>/.exec(trimmed); + const address = (angled ? angled[1] : trimmed).trim(); + const at = address.lastIndexOf("@"); + const domain = at === -1 ? "" : address.slice(at + 1).toLowerCase(); + if (domain === "" || !allowed.has(domain)) { + refused.push(domain === "" ? address : domain); + } + } + return [...new Set(refused)]; +} + +/** + * Which account this call works in. + * + * Unset is the first configured account, which is what makes `account` an argument a model may + * ignore: the common deployment has one mailbox and never sees this. + * + * AN ACCOUNT THAT IS NOT CONFIGURED IS REFUSED HERE, before a password is read and long before + * anything is dialled, and the refusal lists the ones that exist. Two reasons. A model that + * invented an address should be corrected rather than handed a login failure from a mail server, + * which is a sentence about credentials for a mailbox that does not exist; and `send_message` must + * not reach the network on a mistaken argument, since the interesting mistake is a Bot that was + * talked into naming an account by the mail it just read. + * + * Matched case-insensitively, because the configured list is lower-cased and an address is. + */ +export function selectAccount( + args: Record, + users: readonly string[], +): { account?: string; error?: string } { + const asked = stringArg(args, "account"); + if (asked === undefined) return { account: users[0] }; + + const wanted = asked.toLowerCase(); + if (!users.includes(wanted)) { + return { + error: `${asked} is not one of this deployment's mailbox accounts. It has ${users.join(", ")}. Nothing was read and nothing was sent.`, + }; + } + return { account: wanted }; +} + +/** + * Which IMAP folder this call reads, and the one mistake worth catching by hand. + * + * AN ADDRESS IN `folder` IS REFUSED HERE, before the vault and before the network. It is the + * mistake a smaller model actually makes: two arguments that both read as "which mailbox", and the + * address goes in the wrong one. Left to the mail server it comes back as "Character not allowed in + * mailbox name: '.'" and then "Mailbox doesn't exist: support", which are sentences about IMAP + * folder naming that no model turns into "use the other argument". Said here, the fix is in the + * refusal. + * + * THE LOCAL PART IS THE SECOND HALF OF THE SAME MISTAKE, and it was made live: refused an address + * in `folder`, the model retried with `support`, then `webmaster`, which are the parts before the @ + * of two configured accounts. That is not an address any more, so the check above lets it through, + * and the mail server answers "Mailbox doesn't exist: support", which reads as a folder that is + * merely missing rather than as an argument that is wrong. A folder genuinely named after an + * account's local part is possible and is given up here on purpose: the mistake is common and the + * collision is not. + * + * `mailbox` is still read as a name for the same argument. This connector shipped with that key, + * and a Bot holding a tool list from before the rename would otherwise pass a folder that is + * silently ignored and read INBOX while believing it read Archive. Both checks cover it either + * way, so the old name cannot reintroduce the trap. + */ +export function selectFolder( + args: Record, + users: readonly string[], +): { folder?: string; error?: string } { + const asked = stringArg(args, "folder") ?? stringArg(args, "mailbox"); + if (asked === undefined) return { folder: DEFAULT_FOLDER }; + if (asked.includes("@")) { + return { + error: `${asked} looks like an email address, and \`folder\` names an IMAP folder such as ${DEFAULT_FOLDER}. Pass the address as \`account\` instead. Nothing was read and nothing was sent.`, + }; + } + + const wanted = asked.toLowerCase(); + const named = users.find((user) => user.split("@")[0] === wanted); + if (named) { + return { + error: `${asked} is the account ${named}, not a folder. Pass account=${named} and leave folder unset to read ${DEFAULT_FOLDER}. Nothing was read and nothing was sent.`, + }; + } + return { folder: asked }; +} + +/** + * Which account and which folder, together, because the interesting case is a mistake across both. + * + * THE ADOPTION. A model that puts a configured address in `folder` has said something unambiguous: + * there is exactly one mailbox it can mean, and it named it. Refusing that costs a whole turn to + * learn a vocabulary lesson, and live runs show the FIRST mailbox call of a turn making this + * mistake, so the lesson is paid for before any work happens. So the address is taken as the + * account, the folder falls back to {@link DEFAULT_FOLDER}, and the answer says what was done. The + * note teaches the same lesson the refusal did, on the way past rather than instead of the work. + * + * WHAT IS STILL REFUSED, because neither is unambiguous: + * + * - An address in `folder` that is NOT a configured account. There is nothing to adopt: the model + * is asking for a mailbox this deployment does not have, and guessing which one it meant would + * read somebody else's mail to answer a question about a mailbox that is not there. + * - `folder` holding one configured address while `account` names a different one. Two arguments + * naming two mailboxes is a model that has lost track of which it is reading, and picking either + * would be picking for it. The refusal names both. + * + * The local-part refusal in {@link selectFolder} is deliberately left alone. `support` is not an + * address, and a folder genuinely called `support` can exist, so adopting it would be guessing + * where the address case is certain. + */ +export function selectMailbox( + args: Record, + users: readonly string[], +): { account?: string; folder?: string; note?: string; error?: string } { + const askedFolder = stringArg(args, "folder") ?? stringArg(args, "mailbox"); + const adopted = askedFolder?.toLowerCase(); + + if (adopted !== undefined && users.includes(adopted)) { + const askedAccount = stringArg(args, "account")?.toLowerCase(); + if (askedAccount !== undefined && askedAccount !== adopted) { + return { + error: `folder is ${askedFolder}, which is the account ${adopted}, while account is ${askedAccount}. Those are two different mailboxes and this call names no folder at all. Pass the one you mean as \`account\` and leave \`folder\` unset. Nothing was read and nothing was sent.`, + }; + } + return { + account: adopted, + folder: DEFAULT_FOLDER, + note: `[folder took the address ${adopted}; it was used as account, reading ${DEFAULT_FOLDER}.]`, + }; + } + + const account = selectAccount(args, users); + if (account.error || !account.account) return { error: account.error }; + const folder = selectFolder(args, users); + if (folder.error || !folder.folder) return { error: folder.error }; + return { account: account.account, folder: folder.folder }; +} + +/** + * Call one tool. + * + * The grant and the policy are already settled by the time anything gets here: `plugins/store.ts` + * checks what this Bot was given, evaluates the policy against the tool's effect, and writes the + * audit row, exactly as it does for a vendor's server. There is no second path to the mailbox and + * nothing here re-decides any of that. + * + * Nothing thrown escapes. A mail server that refused, timed out or answered nonsense comes back as + * an `isError` result rather than as a throw, which is what the vendor transports do and what + * `plugins/tools.ts` expects: it prefixes the sentence with "The vendor reported an error: " and the + * sentence survives intact, which is the part that matters to whoever reads the transcript. + */ +export async function callTool( + _connection: Connection, + toolName: string, + args: Record, +): Promise { + const access = installed; + if (!access) return failure(NOT_CONFIGURED); + + const chosen = selectMailbox(args, access.config.users); + if (chosen.error || !chosen.account || !chosen.folder) { + return failure(chosen.error ?? NOT_CONFIGURED); + } + const { account, folder, note } = chosen; + + const result = await runTool(access, account, folder, toolName, args); + if (!note) return result; + /* + * The note rides on the answer rather than replacing it. A model that is told what it did wrong + * AND handed the mail it asked for learns the argument without spending a turn on the lesson, + * which is the whole point of adopting the address instead of refusing it. + */ + return { ...result, text: `${note}\n${result.text}` }; +} + +/** The work itself, once the account and the folder are settled. */ +async function runTool( + access: MailboxAccess, + account: string, + folder: string, + toolName: string, + args: Record, +): Promise { + let password: string | null = null; + try { + password = await access.password(account); + if (!password) return failure(noPassword(account)); + + const clients = (access.clients ?? createMailboxClients)( + access.config, + account, + password, + ); + // The folder and whose folder, which is how every sentence below names it. See `where`. + const place = where(folder, account); + + if (toolName === "list_messages") { + const asked = integerArg(args, "limit"); + if (asked.error) return failure(asked.error); + const { limit, capped } = boundedLimit(asked.value, LIST_LIMIT); + + const page = await clients.withSession((session) => + session.recent(folder, limit), + ); + if (page.headers.length === 0) { + // Said in words rather than returned as an empty string: an empty result reads to a model + // as "the tool had nothing to say" and gets filled in from memory. + return asResult(`There are no messages in ${place}.`); + } + return asResult( + [ + ...page.headers.map((header) => `- ${headerLine(header)}`), + ...pageNotes(page, capped, LIST_LIMIT.max, `messages in ${place}`), + ].join("\n"), + ); + } + + if (toolName === "read_message") { + const uid = integerArg(args, "uid"); + if (uid.error) return failure(uid.error); + if (uid.value === undefined) { + return failure( + "Say which message to read, by the uid from list_messages or search_messages.", + ); + } + // A number no mailbox could hold is a message that is not there, and is answered as one + // rather than dialled and turned into a sequence-set complaint. See MAX_UID. + if (uid.value > MAX_UID) { + return failure(noSuchMessage(uid.value, place)); + } + + const message = await clients.withSession((session) => + session.message(folder, uid.value as number), + ); + if (!message) { + return failure(noSuchMessage(uid.value, place)); + } + return asResult(messageInWords(message, place)); + } + + if (toolName === "search_messages") { + const query = stringArg(args, "query"); + if (!query) return failure("Say what to search the mailbox for."); + const asked = integerArg(args, "limit"); + if (asked.error) return failure(asked.error); + const { limit, capped } = boundedLimit(asked.value, SEARCH_LIMIT); + + const page = await clients.withSession((session) => + session.search(folder, query, limit), + ); + if (page.headers.length === 0) { + return asResult( + `Nothing in ${place} matches "${query}". There is nothing here to answer from.`, + ); + } + return asResult( + [ + ...page.headers.map((header) => `- ${headerLine(header)}`), + ...pageNotes(page, capped, SEARCH_LIMIT.max, `matches in ${place}`), + ].join("\n"), + ); + } + + if (toolName === "send_message") { + const to = stringArg(args, "to"); + if (!to) return failure("Say who the message is to."); + const subject = stringArg(args, "subject"); + if (!subject) return failure("A message needs a subject."); + const body = stringArg(args, "body"); + if (!body) return failure("A message needs something in it."); + + /* + * WHERE IT IS GOING IS DECIDED BEFORE ANYTHING IS DIALLED. + * + * First, so a refused recipient costs no connection and touches no mailbox: this is the check + * that stands between a Bot that was talked into something by the mail it just read and an + * address outside the deployment. Naming the domain rather than the address, because the + * domain is the thing the allowlist is written about and the thing an administrator would + * change. + */ + const refused = refusedRecipients( + to, + access.config.allowedRecipientDomains, + ); + if (refused.length > 0) { + return failure( + `This deployment only sends mail to ${[...access.config.allowedRecipientDomains].sort().join(", ")}, and ${refused.join(", ")} is not among them. Nothing was sent. An administrator sets MAILBOX_ALLOWED_RECIPIENT_DOMAINS.`, + ); + } + + const inReplyTo = integerArg(args, "in_reply_to"); + if (inReplyTo.error) return failure(inReplyTo.error); + // The same answer a uid that is not there gets, for the same reason as `read_message`, and + // with the same promise that nothing was sent. + if (inReplyTo.value !== undefined && inReplyTo.value > MAX_UID) { + return failure( + `There is no message with uid ${inReplyTo.value} in ${place}, so there is nothing to reply to. Nothing was sent.`, + ); + } + + let outgoing: OutgoingMessage = { to, subject, body }; + if (inReplyTo.value !== undefined) { + const original = await clients.withSession((session) => + session.message(folder, inReplyTo.value as number), + ); + /* + * Refused rather than sent as a new message. A model that asked for a reply and got an + * unthreaded mail to the same person has been told the wrong thing about what it did, and + * the recipient sees an answer that appears to be about nothing. + */ + if (!original) { + return failure( + `There is no message with uid ${inReplyTo.value} in ${place}, so there is nothing to reply to. Nothing was sent.`, + ); + } + const threaded = replyFrom(original, subject); + outgoing = { + to, + subject: threaded.subject, + body, + ...(threaded.reply ? { reply: threaded.reply } : {}), + }; + } + + const sent = await clients.send(outgoing); + return asResult( + [ + `Sent from ${account} to ${to}, subject "${outgoing.subject}".`, + outgoing.reply + ? "It threads as a reply to that message." + : "It starts a new thread.", + /* + * WHERE THE COPY WENT, or why there is none, and never as a failure. + * + * Naming the folder is what lets a Bot check its own work: it can list that folder and + * find the message it just sent. The other branch is the one that matters more. Filing is + * a separate operation against a separate server and can fail after the mail has gone, so + * it says plainly that the mail WAS sent. A model told only that something failed sends + * the message again, and mail cannot be recalled. + */ + sent.filedTo + ? `A copy is in folder ${sent.filedTo}.` + : `The mail was sent, but no copy could be filed in the Sent folder (${redacted(sent.fileError ?? "no reason given", password, account).slice(0, 200)}). It was delivered, so do not send it again.`, + sent.messageId ? `Message id ${sent.messageId}.` : null, + ] + .filter((one): one is string => one !== null) + .join(" "), + ); + } + + return failure( + `${toolName} is not a tool Mailbox implements. The stored tool list is out of date; refresh it on the Plugins page.`, + ); + } catch (error) { + /* + * The mail server's own sentence, scrubbed of the password and nothing else. + * + * It is the most useful thing available, since "Invalid credentials", "Mailbox does not exist" + * and "Relay access denied" each name a different fix, and rewording it here would turn a + * specific failure into a vague one. Capped, because a failure is not a promise about length. + */ + const message = + error instanceof MailboxError || error instanceof Error + ? error.message + : String(error); + return failure(redacted(message, password, account).slice(0, 400)); + } +} diff --git a/server/src/plugins/catalogue.ts b/server/src/plugins/catalogue.ts index b8445bfce..4ce2008db 100644 --- a/server/src/plugins/catalogue.ts +++ b/server/src/plugins/catalogue.ts @@ -38,10 +38,16 @@ export type CatalogueAuth = /** One token, held by the deployment, used for everybody. */ | { kind: "deployment-bearer" } /** - * First-party and in-process. There is no credential, because there is nothing to authenticate - * to: the call runs against this deployment's own tables, as the person whose turn it is. + * First-party and in-process. There is no vendor to authenticate to, because the call runs inside + * this deployment rather than against somebody else's server. + * + * `reachedAs` is not a formality. The two builtins differ on the one question the audit trail + * asks: Routines touches the asking person's own rows and is therefore reached AS them, while + * Mailbox opens one mailbox that belongs to the deployment, on a password the deployment holds, + * and is reached as the deployment however many people ask. Recording the second as the asker + * would put a person's id on a row describing access that was never theirs. */ - | { kind: "builtin" } + | { kind: "builtin"; reachedAs: "actor" | "deployment" } /** * The asker's own grant. The deployment registers an OAuth client; each person consents once and * the call runs on their token, so the vendor decides what comes back. @@ -280,7 +286,9 @@ export const CATALOGUE: readonly CatalogueEntry[] = Object.freeze([ host: "builtin://routines", path: "/", transport: "builtin-routines", - auth: Object.freeze({ kind: "builtin" }), + // Reached as the person asking: a routine is theirs, it runs with their grants, and the rows it + // touches are their own. + auth: Object.freeze({ kind: "builtin", reachedAs: "actor" }), writeTools: Object.freeze([ "create_routine", "update_routine", @@ -288,6 +296,38 @@ export const CATALOGUE: readonly CatalogueEntry[] = Object.freeze([ ]), docsUrl: "https://github.com/CopilotKit/OpenBot/blob/main/docs/routines.md", }, + { + key: "mailbox", + title: "Mailbox", + vendor: "OpenBot", + summary: + "Read and send email from the deployment's mailbox, as the Bot granted it.", + /* + * In-process, like Routines, and in the catalogue for a sharper version of the same reason. + * There is no vendor to review here, but there is a capability to grant: mail is the one thing + * a Bot can do that reaches people who never agreed to talk to it, and `send_message` is + * irrevocable the instant it runs. Which Bots may read the mail, and which of those may answer + * it, is a decision an administrator makes per tool on the Plugins page. + * + * `builtin` names the auth kind because nothing here is authenticated per person: there is one + * mailbox, it belongs to the deployment, and its password is the deployment's own. That + * password is NOT reached through `credential_id` on the server row: see + * `plugins/builtin-mailbox.ts`, which resolves it from the vault as the `mailbox` credential. + */ + host: "builtin://mailbox", + path: "/", + transport: "builtin-mailbox", + // Reached as the deployment: one mailbox, one password this deployment holds, the same mail for + // everybody who asks. Whoever asked is still on the audit row as the actor; what this settles is + // that the access was not theirs. + auth: Object.freeze({ kind: "builtin", reachedAs: "deployment" }), + /* + * One write, and it is the whole reason this entry is grantable per tool: reading mail is + * recoverable and sending it is not. + */ + writeTools: Object.freeze(["send_message"]), + docsUrl: "https://github.com/CopilotKit/OpenBot/blob/main/docs/mailbox.md", + }, ]); const BY_KEY = new Map(CATALOGUE.map((entry) => [entry.key, entry])); diff --git a/server/src/plugins/store.ts b/server/src/plugins/store.ts index 11867e4ce..07d36905a 100644 --- a/server/src/plugins/store.ts +++ b/server/src/plugins/store.ts @@ -315,13 +315,22 @@ const iso = (value: Date | string | null): string | null => * the row a per-person connector exists to be able to trust. * * `deployment` for a shared token; the asker's own id for a server reached as the person asking. - * `builtin` is the third case and the only one with no credential at all — the actor is not whose - * token was used, it is whose rows were touched. + * `builtin` is the third case and the only one with no credential at all, so it answers for itself: + * Routines touches the asker's own rows and says so, while Mailbox opens one mailbox belonging to + * the deployment and says that instead. The entry decides, rather than this expression guessing from + * the kind, because a builtin reached as the deployment but recorded as the asker would put + * somebody's id on a row describing access that was never theirs. */ -const reachedAsFor = (entry: CatalogueEntry | null, actorId: string): string => - entry?.auth.kind === "user-oauth" || entry?.auth.kind === "builtin" - ? actorId - : "deployment"; +const reachedAsFor = ( + entry: CatalogueEntry | null, + actorId: string, +): string => { + if (entry?.auth.kind === "user-oauth") return actorId; + if (entry?.auth.kind === "builtin") { + return entry.auth.reachedAs === "actor" ? actorId : "deployment"; + } + return "deployment"; +}; /** * Where this server actually is, when the stored row and the catalogue disagree. diff --git a/server/src/plugins/transport.ts b/server/src/plugins/transport.ts index 4fecdb325..8ae7ecf12 100644 --- a/server/src/plugins/transport.ts +++ b/server/src/plugins/transport.ts @@ -1,3 +1,4 @@ +import * as builtinMailbox from "./builtin-mailbox"; import * as builtinRoutines from "./builtin-routines"; import type { CatalogueEntry } from "./catalogue"; import * as driveRest from "./google-drive-rest"; @@ -44,9 +45,10 @@ export type VendorTransport = { * Who this call is for, and which Bot is making it. * * Ignored by every transport that dials a vendor: MCP and Drive answer to a credential, and who - * holds it is already decided by the time the connection is built. The builtin transport has no - * credential and no vendor — it acts on this deployment's own tables — so the actor is not - * context, it is the authorization, and it refuses without one. A routine is somebody's. + * holds it is already decided by the time the connection is built. A builtin transport has no + * vendor, so it answers for itself: Routines refuses a call that names no actor, because a + * routine is somebody's and the actor is the authorization, while Mailbox does not, because + * there is one mailbox belonging to the deployment and the Bot's grant is the authorization. */ actorId?: string; /** The Bot the run belongs to. A routine runs as its Bot, which is never a name a model supplies. */ @@ -60,9 +62,10 @@ export type VendorTransport = { * Who this call is for, and which Bot is making it. * * Ignored by every transport that dials a vendor: MCP and Drive answer to a credential, and who - * holds it is already decided by the time the connection is built. The builtin transport has no - * credential and no vendor — it acts on this deployment's own tables — so the actor is not - * context, it is the authorization, and it refuses without one. A routine is somebody's. + * holds it is already decided by the time the connection is built. A builtin transport has no + * vendor, so it answers for itself: Routines refuses a call that names no actor, because a + * routine is somebody's and the actor is the authorization, while Mailbox does not, because + * there is one mailbox belonging to the deployment and the Bot's grant is the authorization. */ actorId?: string; /** The Bot the run belongs to. A routine runs as its Bot, which is never a name a model supplies. */ @@ -79,12 +82,17 @@ export type VendorTransport = { * A closed union rather than a string, so adding one is a change to this file and to the registry * below together. An entry naming a transport that does not exist should not typecheck. */ -export type TransportKind = "mcp" | "google-drive-rest" | "builtin-routines"; +export type TransportKind = + | "mcp" + | "google-drive-rest" + | "builtin-routines" + | "builtin-mailbox"; const TRANSPORTS: Record = { mcp, "google-drive-rest": driveRest, "builtin-routines": builtinRoutines, + "builtin-mailbox": builtinMailbox, }; /** diff --git a/server/tests/builtin-mailbox.test.ts b/server/tests/builtin-mailbox.test.ts new file mode 100644 index 000000000..bb1653c49 --- /dev/null +++ b/server/tests/builtin-mailbox.test.ts @@ -0,0 +1,1819 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import type { MailboxConfig } from "../src/config"; +import { + composeMessage, + createMailboxClients, + type FullMessage, + type ImapLike, + MAX_BODY_CHARS, + MAX_FOLDERS_LISTED, + MAX_SOURCE_BYTES, + type MailboxClients, + MailboxError, + type MailboxSession, + type MessageHeader, + type MessagePage, + mailServerSentence, + noSuchFolderSentence, + type OutgoingMessage, + readBody, + type SendReceipt, + type SmtpLike, + sentFolderFrom, + strippedHtml, + withDeadline, +} from "../src/mailbox/client"; +import { + boundedLimit, + callTool, + listTools, + redacted, + refusedRecipients, + replyFrom, + selectAccount, + selectFolder, + selectMailbox, + useMailbox, +} from "../src/plugins/builtin-mailbox"; +import { MAX_RESULT_CHARS } from "../src/plugins/mcp"; + +/** + * The builtin Mailbox transport, asserted without a mail server. + * + * What is under test is the boundary, not IMAP: which arguments reach the protocol, what a model is + * told when they are wrong, how much text can come back, and whether a reply is threaded against a + * message that actually exists. A recording stub is installed through {@link useMailbox}, which is + * the seam this module has for exactly that reason: `transportFor` resolves a kind to a MODULE, so + * there is no constructor to pass a client to. + * + * The two properties this file exists for are the ones that are expensive to be wrong about. The + * first is the password: it is resolved per call, it must never appear in an answer, and a mail + * server that echoes it back into an error must not be able to launder it into a transcript or an + * audit row. The second is bounding: every result here is somebody else's text, arriving in a + * model's context window, and an unbounded body is a sender deciding how much of it to spend. + */ + +const CONNECTION = { + url: "builtin://mailbox/", + actorId: "user_asker", + botId: "bot_helper", +}; + +/** + * Two accounts on one pair of hosts, which is the shared-hosting shape this connector serves. + * + * The first is the default, so every case that names no account is also a case about which one that + * is. The second exists so "the account was honoured" is a claim that can fail: with one configured + * account every answer would name the right address by accident. + */ +const ACCOUNTS = ["bot@example.test", "sales@example.test"] as const; +const DEFAULT_ACCOUNT = ACCOUNTS[0]; + +const CONFIG: MailboxConfig = { + imapHost: "imap.example.test", + imapPort: 993, + smtpHost: "smtp.example.test", + smtpPort: 465, + users: [...ACCOUNTS], + // Unrestricted, which is the default and what every case here but the allowlist ones wants. + allowedRecipientDomains: new Set(), +}; + +const PASSWORD = "correct-horse-battery-staple"; + +const HEADER: MessageHeader = { + uid: 42, + from: "Dana Reid ", + to: "bot@example.test", + subject: "The Friday numbers", + date: "2026-08-30T09:15:00.000Z", + seen: false, +}; + +const MESSAGE: FullMessage = { + ...HEADER, + messageId: "", + references: [""], + body: "Can you send the Friday numbers?", + bodyLength: 32, + sourceTruncated: false, + sizeBytes: 1_200, +}; + +/** One page of one message, which is what most of these cases want back. */ +const onePage = (header: MessageHeader = HEADER): MessagePage => ({ + headers: [{ ...header }], + total: 1, +}); + +type Recorded = + | { method: "recent"; mailbox: string; limit: number } + | { method: "message"; mailbox: string; uid: number } + | { method: "search"; mailbox: string; query: string; limit: number } + | { method: "send"; message: OutgoingMessage }; + +type Stubs = { + recent?: (mailbox: string, limit: number) => Promise; + message?: (mailbox: string, uid: number) => Promise; + search?: ( + mailbox: string, + query: string, + limit: number, + ) => Promise; + send?: (message: OutgoingMessage) => Promise; + password?: (account: string) => Promise; + /** A deployment configured differently from {@link CONFIG}, for the allowlist cases. */ + config?: MailboxConfig; +}; + +/** + * Installs a mailbox whose protocols are recorded rather than spoken. + * + * The config and the password handed to the factory are recorded too, because "the call went out + * with the configured host and the vault's password" is a claim worth being able to check without a + * server that would have to answer it. + */ +function recordingMailbox(stubs: Stubs = {}): { + calls: Recorded[]; + built: { config: MailboxConfig; account: string; password: string }[]; + /** Which account each password lookup was for, in order. One vault row per account. */ + unlocked: string[]; +} { + const calls: Recorded[] = []; + const built: { + config: MailboxConfig; + account: string; + password: string; + }[] = []; + const unlocked: string[] = []; + + useMailbox({ + config: stubs.config ?? CONFIG, + password: async (account) => { + unlocked.push(account); + return stubs.password ? await stubs.password(account) : PASSWORD; + }, + clients: (config, account, password): MailboxClients => { + built.push({ config, account, password }); + const session: MailboxSession = { + async recent(mailbox, limit) { + calls.push({ method: "recent", mailbox, limit }); + return stubs.recent ? await stubs.recent(mailbox, limit) : onePage(); + }, + async message(mailbox, uid) { + calls.push({ method: "message", mailbox, uid }); + return stubs.message ? await stubs.message(mailbox, uid) : MESSAGE; + }, + async search(mailbox, query, limit) { + calls.push({ method: "search", mailbox, query, limit }); + return stubs.search + ? await stubs.search(mailbox, query, limit) + : onePage(); + }, + }; + return { + withSession: (use) => use(session), + async send(message) { + calls.push({ method: "send", message }); + return stubs.send + ? await stubs.send(message) + : { + messageId: "", + filedTo: "Sent", + fileError: null, + }; + }, + }; + }, + }); + + return { calls, built, unlocked }; +} + +// The binding is module-level and the suite is one process, so a mailbox left installed here would +// be the one some other file's test unexpectedly reaches. +afterEach(() => { + useMailbox(null); +}); + +describe("the tool list", () => { + test("is the four mailbox tools, named exactly", async () => { + const tools = await listTools(); + expect(tools.map((tool) => tool.name)).toEqual([ + "list_messages", + "read_message", + "search_messages", + "send_message", + ]); + for (const tool of tools) { + expect(tool.description.length).toBeGreaterThan(0); + expect(tool.inputSchema).toBeDefined(); + } + }); + + test("requires only what a call cannot be made without", async () => { + const required = Object.fromEntries( + (await listTools()).map((tool) => [ + tool.name, + (tool.inputSchema as { required?: string[] }).required ?? [], + ]), + ); + expect(required.list_messages).toEqual([]); + expect(required.read_message).toEqual(["uid"]); + expect(required.search_messages).toEqual(["query"]); + expect(required.send_message).toEqual(["to", "subject", "body"]); + }); + + test("says the two things a model cannot see: whose mailbox, and that uids are per mailbox", async () => { + const tools = await listTools(); + const list = tools.find((tool) => tool.name === "list_messages"); + // A Bot that believes it is reading the asker's own mail will summarize somebody else's inbox + // to them, and neither party will ever be told. + expect(list?.description ?? "").toContain("shared mailbox"); + expect(list?.description ?? "").toContain("per folder"); + + const send = tools.find((tool) => tool.name === "send_message"); + expect(send?.description ?? "").toContain("in_reply_to"); + expect(send?.description ?? "").toContain("nothing to recall"); + }); + + test("names the folder argument `folder`, and says it is not an address", async () => { + /* + * The live failure this rename came from: a smaller model passed the email address as + * `mailbox`, was told "Character not allowed in mailbox name: '.'" by the IMAP server, and + * never tried `account`. Two arguments that both read as "which mailbox" is the trap, so one of + * them is called `folder` and says in its own description what it is not. + */ + recordingMailbox(); + for (const tool of await listTools()) { + const properties = ( + tool.inputSchema as { + properties?: Record; + } + ).properties; + expect(properties?.folder).toBeDefined(); + expect(properties?.mailbox).toBeUndefined(); + const description = properties?.folder?.description ?? ""; + expect(description).toContain("IMAP folder"); + expect(description).toContain( + "Leave it unset unless the person names a folder", + ); + expect(description).toContain("This is not an email address"); + // The local part is the second half of the same mistake, so the argument says so itself. + expect(description).toContain("not the part before the @"); + expect(description).toContain("use `account`"); + + /* + * A model reads a schema in order, and the argument it meets first is the one it reaches for + * when it wants to say "the support mailbox". `folder` first is how an address, and then the + * local part of one, ends up in it. + */ + const order = Object.keys(properties ?? {}); + expect(order[0]).toBe("account"); + expect(order.indexOf("account")).toBeLessThan(order.indexOf("folder")); + } + }); + + test("offers `account` and names the addresses a deployment actually has", async () => { + /* + * The choices are the deployment's, not this file's, so they are added at list time. A model + * told only that an `account` argument exists has to guess at an address, and a guessed address + * is a refusal at best. + */ + recordingMailbox(); + for (const tool of await listTools()) { + const properties = ( + tool.inputSchema as { + properties?: Record; + } + ).properties; + expect(properties?.account).toBeDefined(); + const description = properties?.account?.description ?? ""; + expect(description).toContain("as its email address"); + expect(description).toContain( + "One of: bot@example.test, sales@example.test.", + ); + expect(description).toContain("Default bot@example.test."); + expect(tool.description).toContain( + "bot@example.test, sales@example.test", + ); + expect(tool.description).toContain("`account`"); + } + // Never required: the deployment with one mailbox should not have to think about this at all. + const required = (await listTools()).flatMap( + (tool) => (tool.inputSchema as { required?: string[] }).required ?? [], + ); + expect(required).not.toContain("account"); + }); + + test("says an account can be named even with no mailbox configured", async () => { + // The list is answered either way, so the wording has to work before there are addresses to + // name. See listTools. + useMailbox(null); + for (const tool of await listTools()) { + expect(tool.description).toContain( + "the deployment's configured addresses", + ); + } + }); + + test("needs no mailbox, no credential and no actor", async () => { + // The only call site is `refreshTools`, which passes `{url, token}`. A list that emptied itself + // when a variable was unset would revoke an administrator's grants by accident. + useMailbox(null); + expect(await listTools()).toHaveLength(4); + }); +}); + +describe("a deployment with no mailbox", () => { + test("refuses every tool, naming all four things to set", async () => { + useMailbox(null); + for (const tool of [ + "list_messages", + "read_message", + "search_messages", + "send_message", + ]) { + const result = await callTool(CONNECTION, tool, { uid: 1 }); + expect(result.isError).toBe(true); + expect(result.text).toBe( + "Mailbox is not configured. Set MAILBOX_IMAP_HOST, MAILBOX_SMTP_HOST, MAILBOX_USERS and store each account's password as its mailbox credential.", + ); + } + }); + + test("a configured mailbox with no password in the vault says so separately, naming the account", async () => { + // A different job with a different fix. Telling an administrator to set three variables they can + // see are already set is how a correct instruction reads as a broken deployment. + const { calls } = recordingMailbox({ password: async () => null }); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(result.isError).toBe(true); + expect(result.text).toContain( + `holds no password for the mailbox ${DEFAULT_ACCOUNT}`, + ); + // The key id is the address, so the sentence tells an administrator exactly which row to store. + expect(result.text).toContain(`key id \`${DEFAULT_ACCOUNT}\``); + // Nothing was dialled, so nothing could have been sent to a server unauthenticated. + expect(calls).toEqual([]); + }); + + test("a password missing for one account is that account's problem, not the mailbox's", async () => { + /* + * A deployment that stored `bot@` and forgot `sales@` works for the one it has. The refusal has + * to name the account that is missing a password, or an administrator goes to look at the + * credential that is already there and finds nothing wrong with it. + */ + const { calls } = recordingMailbox({ + password: async (account) => + account === DEFAULT_ACCOUNT ? PASSWORD : null, + }); + + const working = await callTool(CONNECTION, "list_messages", {}); + expect(working.isError).toBe(false); + + const missing = await callTool(CONNECTION, "list_messages", { + account: "sales@example.test", + }); + expect(missing.isError).toBe(true); + expect(missing.text).toContain( + "holds no password for the mailbox sales@example.test", + ); + // Only the account that had one was ever dialled. + expect(calls).toEqual([{ method: "recent", mailbox: "INBOX", limit: 10 }]); + }); +}); + +/** + * Several accounts on one pair of hosts, which is what a shared host gives a deployment. + * + * The three properties worth pinning: the default is the first configured, so a deployment that + * only ever had one mailbox behaves exactly as it did; an account that was named is the one dialled, + * unlocked and answered about; and an account that is not configured is refused before anything + * leaves this process. + */ +describe("several accounts", () => { + test("works in the first configured account when none was named", async () => { + const { built, unlocked } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(unlocked).toEqual([DEFAULT_ACCOUNT]); + expect(built.map((one) => one.account)).toEqual([DEFAULT_ACCOUNT]); + expect(result.text).not.toContain("sales@example.test"); + }); + + test("unlocks and dials the account that was named, and names it back", async () => { + const { built, unlocked } = recordingMailbox({ + recent: async () => ({ headers: [{ ...HEADER }], total: 236 }), + }); + const result = await callTool(CONNECTION, "list_messages", { + account: "sales@example.test", + }); + + // The password is that account's own vault row, so the lookup is per account. + expect(unlocked).toEqual(["sales@example.test"]); + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + // A model reading two accounts in one turn cannot tell two INBOXes apart otherwise. + expect(result.text).toContain( + "[showing 1 of 236 messages in folder INBOX of sales@example.test, newest first.]", + ); + }); + + test("takes the address in whatever case it was written", async () => { + // The configured list is lower-cased and so is the credential key, so a model that shouted the + // address reaches the same mailbox rather than an account that does not exist. + const { built } = recordingMailbox(); + const result = await callTool(CONNECTION, "read_message", { + uid: 42, + account: "Sales@Example.TEST", + }); + + expect(result.isError).toBe(false); + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + expect(result.text).toContain( + "uid 42 in folder INBOX of sales@example.test", + ); + }); + + test("refuses an account this deployment does not have, before anything is dialled", async () => { + /* + * Before the vault and before the network. A model that invented an address should be corrected + * with the list of real ones rather than handed a login failure about a mailbox that does not + * exist, and `send_message` must not reach a mail server on a mistaken argument. + */ + const { calls, built, unlocked } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + account: "billing@example.test", + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("billing@example.test is not one of"); + // The configured addresses, so the next call can be right. + expect(result.text).toContain("bot@example.test, sales@example.test"); + expect(result.text).toContain("nothing was sent"); + expect(calls).toEqual([]); + expect(built).toEqual([]); + expect(unlocked).toEqual([]); + }); + + test("sends from the account it was told to, and says which", async () => { + const { calls, built } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Attached.", + account: "sales@example.test", + }); + + expect(result.isError).toBe(false); + // From is the client's, never an argument: the account decides who the mail is from. + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + expect(result.text).toContain( + "Sent from sales@example.test to dana@example.test", + ); + expect(calls.some((call) => call.method === "send")).toBe(true); + }); + + test("scrubs the account's own password out of that account's failure", async () => { + /* + * Redaction is per selected account: the AUTH=PLAIN blob a server quotes back carries the + * account and its password together, so scrubbing with the default account's name would leave + * another account's credential in the sentence. + */ + const OTHER = "a-different-secret-entirely"; + recordingMailbox({ + password: async (account) => + account === "sales@example.test" ? OTHER : PASSWORD, + recent: async () => { + const blob = Buffer.from( + `\u0000sales@example.test\u0000${OTHER}`, + "utf8", + ).toString("base64"); + throw new MailboxError(`A1 BAD failed: AUTHENTICATE PLAIN ${blob}`); + }, + }); + const result = await callTool(CONNECTION, "list_messages", { + account: "sales@example.test", + }); + + expect(result.isError).toBe(true); + expect(result.text).not.toContain(OTHER); + expect(result.text).not.toContain( + Buffer.from(`\u0000sales@example.test\u0000${OTHER}`, "utf8").toString( + "base64", + ), + ); + expect(result.text).toContain("[redacted]"); + expect(result.text).toContain("BAD failed"); + }); + + test("refuses an address in `folder`, before anything is dialled, pointing at `account`", async () => { + /* + * The mistake as it actually happened: the address in the folder argument. Left to the mail + * server it comes back as "Character not allowed in mailbox name" and then "Mailbox doesn't + * exist: support", which are sentences about IMAP folder naming that no model turns into "use + * the other argument". The refusal has to carry the fix. + */ + const { calls, built, unlocked } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + folder: "support@example.test", + }); + + expect(result.isError).toBe(true); + expect(result.text).toBe( + "support@example.test looks like an email address, and `folder` names an IMAP folder such as INBOX. Pass the address as `account` instead. Nothing was read and nothing was sent.", + ); + expect(calls).toEqual([]); + expect(built).toEqual([]); + expect(unlocked).toEqual([]); + }); + + test("refuses an address in `folder` on the write tool too, sending nothing", async () => { + // An address this deployment does not have, so there is nothing to adopt. A configured one is + // taken as the account instead: see the adoption cases below. + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + folder: "billing@example.test", + in_reply_to: 42, + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("Pass the address as `account` instead"); + expect(result.text).toContain("nothing was sent"); + expect(calls).toEqual([]); + }); + + test("still reads the old `mailbox` key, so a stale tool list is not a silent wrong folder", async () => { + // A Bot holding a tool list from before the rename would otherwise pass a folder that is + // ignored, and read INBOX while believing it read Archive. + const { calls } = recordingMailbox(); + await callTool(CONNECTION, "list_messages", { mailbox: "Archive" }); + expect(calls).toEqual([ + { method: "recent", mailbox: "Archive", limit: 10 }, + ]); + + // And the guard covers it under the old name as well, so it cannot bring the trap back. + const address = await callTool(CONNECTION, "list_messages", { + mailbox: "support@example.test", + }); + expect(address.isError).toBe(true); + expect(address.text).toContain("Pass the address as `account` instead"); + }); + + test("takes a configured address out of `folder` and reads that account's INBOX", async () => { + /* + * The mistake, adopted rather than refused. A model that put a configured address in `folder` + * has said something unambiguous: there is one mailbox it can mean and it named it. Live runs + * show this on the FIRST mailbox call of a turn, so refusing it spends the turn on a vocabulary + * lesson before any work happens. + */ + const { calls, built } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + folder: "Sales@Example.TEST", + }); + + expect(result.isError).toBe(false); + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + // The folder falls back to INBOX, because the argument that named one was spent on the account. + expect(calls).toEqual([{ method: "recent", mailbox: "INBOX", limit: 10 }]); + // The lesson is taught on the way past rather than instead of the work. + expect(result.text.split("\n")[0]).toBe( + "[folder took the address sales@example.test; it was used as account, reading INBOX.]", + ); + expect(result.text).toContain("uid 42"); + }); + + test("adopts the address on the write path too, threading and sending from that account", async () => { + const { calls, built } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Friday it is.", + in_reply_to: 42, + folder: "sales@example.test", + }); + + expect(result.isError).toBe(false); + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + // The original is fetched from the adopted account's INBOX, and the reply goes out from it. + expect(calls[0]).toEqual({ method: "message", mailbox: "INBOX", uid: 42 }); + expect(calls[1]?.method).toBe("send"); + expect(result.text).toContain( + "[folder took the address sales@example.test; it was used as account, reading INBOX.]", + ); + expect(result.text).toContain("Sent from sales@example.test"); + }); + + test("adopting is not overriding: the same address in both arguments is fine", async () => { + const { built } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + folder: "sales@example.test", + account: "sales@example.test", + }); + + expect(result.isError).toBe(false); + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + }); + + test("refuses two arguments naming two different mailboxes, naming both", async () => { + /* + * Nothing to adopt: a model that named one mailbox in `folder` and another in `account` has + * lost track of which it is reading, and picking either would be picking for it. + */ + const { calls, built, unlocked } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + folder: "sales@example.test", + account: "bot@example.test", + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("folder is sales@example.test"); + expect(result.text).toContain("account is bot@example.test"); + expect(result.text).toContain("Nothing was read and nothing was sent."); + expect(calls).toEqual([]); + expect(built).toEqual([]); + expect(unlocked).toEqual([]); + }); + + test("refuses an account's local part in `folder`, before anything is dialled", async () => { + /* + * The retry after the @ guard fired, as it actually happened: refused `support@example.test`, + * the model tried `support`, then `webmaster`, which are the parts before the @ of configured + * accounts. The mail server answers "Mailbox doesn't exist: support", which reads as a folder + * that happens to be missing rather than as an argument that is wrong. + */ + const { calls, built, unlocked } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + folder: "Sales", + }); + + expect(result.isError).toBe(true); + expect(result.text).toBe( + "Sales is the account sales@example.test, not a folder. Pass account=sales@example.test and leave folder unset to read INBOX. Nothing was read and nothing was sent.", + ); + expect(calls).toEqual([]); + expect(built).toEqual([]); + expect(unlocked).toEqual([]); + }); + + test("resolves the two arguments together, on its own", () => { + const users = ["bot@example.test", "sales@example.test"]; + expect(selectMailbox({}, users)).toEqual({ + account: "bot@example.test", + folder: "INBOX", + }); + expect(selectMailbox({ folder: "Archive" }, users)).toEqual({ + account: "bot@example.test", + folder: "Archive", + }); + expect(selectMailbox({ folder: " SALES@example.test " }, users)).toEqual({ + account: "sales@example.test", + folder: "INBOX", + note: "[folder took the address sales@example.test; it was used as account, reading INBOX.]", + }); + // An address this deployment does not have has nothing to adopt, so the refusal stands. + expect( + selectMailbox({ folder: "billing@example.test" }, users).error, + ).toContain("looks like an email address"); + // A folder named like an account's local part is still refused: `support` is not an address, + // and a folder genuinely called that can exist. + expect(selectMailbox({ folder: "sales" }, users).error).toContain( + "is the account sales@example.test, not a folder", + ); + }); + + test("picks the folder out of the arguments, on its own", () => { + const users = ["bot@example.test", "sales@example.test"]; + expect(selectFolder({}, users)).toEqual({ folder: "INBOX" }); + expect(selectFolder({ folder: " " }, users)).toEqual({ folder: "INBOX" }); + expect(selectFolder({ folder: "Archive" }, users)).toEqual({ + folder: "Archive", + }); + expect(selectFolder({ mailbox: "Sent" }, users)).toEqual({ + folder: "Sent", + }); + expect(selectFolder({ folder: "a@b.test" }, users).error).toContain( + "looks like an email address", + ); + // Case-insensitively and after trimming, which is how a model writes it. + expect(selectFolder({ folder: " BOT " }, users).error).toContain( + "is the account bot@example.test, not a folder", + ); + // A folder that merely starts the same way is still a folder. + expect(selectFolder({ folder: "bot-archive" }, users)).toEqual({ + folder: "bot-archive", + }); + }); + + test("picks the account out of the arguments, on its own", () => { + const users = ["bot@example.test", "sales@example.test"]; + expect(selectAccount({}, users)).toEqual({ account: "bot@example.test" }); + expect(selectAccount({ account: " " }, users)).toEqual({ + account: "bot@example.test", + }); + expect(selectAccount({ account: "SALES@example.test" }, users)).toEqual({ + account: "sales@example.test", + }); + expect( + selectAccount({ account: "nobody@example.test" }, users).error, + ).toContain("not one of this deployment's mailbox accounts"); + }); +}); + +describe("listing", () => { + test("dials the configured mailbox with the vault's password", async () => { + const { built } = recordingMailbox(); + await callTool(CONNECTION, "list_messages", {}); + expect(built).toEqual([ + { config: CONFIG, account: DEFAULT_ACCOUNT, password: PASSWORD }, + ]); + }); + + test("defaults to ten of INBOX", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(result.isError).toBe(false); + expect(calls).toEqual([{ method: "recent", mailbox: "INBOX", limit: 10 }]); + expect(result.text).toContain("uid 42"); + expect(result.text).toContain("Dana Reid"); + expect(result.text).toContain("The Friday numbers"); + expect(result.text).toContain("unread"); + }); + + test("honours a mailbox and a limit that were asked for", async () => { + const { calls } = recordingMailbox(); + await callTool(CONNECTION, "list_messages", { + folder: "Archive", + limit: 3, + }); + expect(calls).toEqual([{ method: "recent", mailbox: "Archive", limit: 3 }]); + }); + + test("caps a huge ask at fifty and says it did", async () => { + // Capped rather than refused: "give me 200" is a clear intention this tool does not serve, and a + // refusal would cost a round trip to be told a smaller number. Said, so a model that asked for + // 200 and got 50 does not conclude the mailbox holds fifty messages. + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { limit: 200 }); + + expect(calls).toEqual([{ method: "recent", mailbox: "INBOX", limit: 50 }]); + expect(result.text).toContain("50 is the most this tool lists at once"); + }); + + test("says how many there were when the mailbox holds more than the page", async () => { + /* + * A page of ten with nothing else on it reads to a model as the whole mailbox, and it will + * answer "you have ten messages" about a mailbox holding four thousand. The count is free (it + * is the mailbox's own `exists`) and it is the difference between a listing and a claim. + */ + recordingMailbox({ + recent: async (_mailbox, limit) => ({ + headers: Array.from({ length: limit }, (_, index) => ({ + ...HEADER, + uid: index + 1, + })), + total: 4_321, + }), + }); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(result.text).toContain( + `showing 10 of 4321 messages in folder INBOX of ${DEFAULT_ACCOUNT}, newest first.`, + ); + // Nothing was capped, so the tool's own limit is not mentioned as well. + expect(result.text).not.toContain("most this tool lists"); + }); + + test("refuses a limit that is not a whole number, rather than rounding it", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + limit: "ten", + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("`limit` has to be a whole number"); + expect(calls).toEqual([]); + }); + + test("an empty mailbox is said in words, not answered with nothing", async () => { + // An empty string reads to a model as "the tool had nothing to say" and gets filled in from + // memory, which for a mailbox means inventing mail. + recordingMailbox({ recent: async () => ({ headers: [], total: 0 }) }); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(result.isError).toBe(false); + expect(result.text).toBe( + `There are no messages in folder INBOX of ${DEFAULT_ACCOUNT}.`, + ); + }); + + test("bounds the whole answer the way every other connector's is", async () => { + const many: MessageHeader[] = Array.from({ length: 50 }, (_, index) => ({ + ...HEADER, + uid: index + 1, + subject: "x".repeat(1_000), + })); + recordingMailbox({ + recent: async () => ({ headers: many, total: many.length }), + }); + const result = await callTool(CONNECTION, "list_messages", { limit: 50 }); + + expect(result.truncated).toBe(true); + expect(result.text).toContain("[truncated:"); + expect(result.text.length).toBeLessThan(MAX_RESULT_CHARS + 200); + }); +}); + +describe("reading one message", () => { + test("reaches the mailbox the uid came from", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "read_message", { + uid: 42, + folder: "Archive", + }); + + expect(result.isError).toBe(false); + expect(calls).toEqual([{ method: "message", mailbox: "Archive", uid: 42 }]); + expect(result.text).toContain("From: Dana Reid "); + expect(result.text).toContain("Can you send the Friday numbers?"); + }); + + test("accepts a uid a model wrote as a string", async () => { + const { calls } = recordingMailbox(); + await callTool(CONNECTION, "read_message", { uid: "42" }); + expect(calls).toEqual([{ method: "message", mailbox: "INBOX", uid: 42 }]); + }); + + test("refuses a missing uid and a fractional one without dialling", async () => { + const { calls } = recordingMailbox(); + const missing = await callTool(CONNECTION, "read_message", {}); + expect(missing.isError).toBe(true); + expect(missing.text).toContain("Say which message to read"); + + const fractional = await callTool(CONNECTION, "read_message", { + uid: 12.7, + }); + expect(fractional.isError).toBe(true); + expect(fractional.text).toContain("whole number"); + expect(calls).toEqual([]); + }); + + test("a uid that is not there is named as such, with why", async () => { + recordingMailbox({ message: async () => null }); + const result = await callTool(CONNECTION, "read_message", { uid: 7 }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("no message with uid 7 in folder INBOX"); + expect(result.text).toContain("per folder"); + }); + + test("a long body is cut and says how long it was", async () => { + recordingMailbox({ + message: async () => ({ + ...MESSAGE, + body: "y".repeat(MAX_BODY_CHARS), + bodyLength: 41_000, + }), + }); + const result = await callTool(CONNECTION, "read_message", { uid: 42 }); + + expect(result.isError).toBe(false); + // The length is stated, because a message cut at 8000 of 9000 has almost certainly said what it + // came to say and one cut at 8000 of 41000 has not. + expect(result.text).toContain("[truncated: the message body is 41000"); + expect(result.text).toContain(String(MAX_BODY_CHARS)); + }); + + test("a message read only in part says the rest was never seen", async () => { + /* + * A different fact from a long body. A cut body means the deployment holds the whole message + * and is showing part of it; a cut source means it never read the rest off the wire, so an + * answer offering to go back for the attachment would be offering something impossible. + */ + recordingMailbox({ + message: async () => ({ + ...MESSAGE, + sourceTruncated: true, + sizeBytes: 4_000_000, + }), + }); + const result = await callTool(CONNECTION, "read_message", { uid: 42 }); + + expect(result.text).toContain(`only the first ${MAX_SOURCE_BYTES} bytes`); + expect(result.text).toContain("4000000 bytes"); + expect(result.text).toContain("attachments"); + }); + + test("a uid too large for IMAP is answered as a message that is not there", async () => { + // Uids are 32-bit (RFC 3501). Passed through, `1e21` reaches imapflow and comes back as + // "Invalid sequence set value", which is a sentence about a data structure rather than about + // the message a model thinks it asked for. + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "read_message", { + uid: 4_294_967_296, + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain( + "no message with uid 4294967296 in folder INBOX", + ); + expect(calls).toEqual([]); + }); + + test("a message with no readable text says that rather than nothing", async () => { + recordingMailbox({ + message: async () => ({ ...MESSAGE, body: "", bodyLength: 0 }), + }); + const result = await callTool(CONNECTION, "read_message", { uid: 42 }); + expect(result.text).toContain("no readable text"); + }); +}); + +describe("searching", () => { + test("passes the query through and defaults to twenty", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "search_messages", { + query: "invoice", + }); + + expect(result.isError).toBe(false); + expect(calls).toEqual([ + { method: "search", mailbox: "INBOX", query: "invoice", limit: 20 }, + ]); + }); + + test("refuses an empty query without dialling", async () => { + const { calls } = recordingMailbox(); + const blank = await callTool(CONNECTION, "search_messages", { query: " " }); + expect(blank.isError).toBe(true); + expect(blank.text).toContain("Say what to search"); + expect(calls).toEqual([]); + }); + + test("says how many matched when it is showing fewer", async () => { + recordingMailbox({ + search: async (_mailbox, _query, limit) => ({ + headers: Array.from({ length: limit }, (_, index) => ({ + ...HEADER, + uid: index + 1, + })), + total: 300, + }), + }); + const result = await callTool(CONNECTION, "search_messages", { + query: "invoice", + }); + + // A search that found 300 and is answering with 20 has to say so, or the model reports 20. + expect(result.text).toContain("showing 20 of 300 matches"); + }); + + test("nothing found is stated, with nothing to answer from", async () => { + recordingMailbox({ search: async () => ({ headers: [], total: 0 }) }); + const result = await callTool(CONNECTION, "search_messages", { + query: "invoice", + }); + + expect(result.isError).toBe(false); + expect(result.text).toContain( + `Nothing in folder INBOX of ${DEFAULT_ACCOUNT} matches "invoice"`, + ); + expect(result.text).toContain("nothing here to answer from"); + }); +}); + +describe("sending", () => { + test("sends what it was given and confirms it", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Attached.", + }); + + expect(result.isError).toBe(false); + expect(calls).toEqual([ + { + method: "send", + message: { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Attached.", + }, + }, + ]); + expect(result.text).toContain( + `Sent from ${DEFAULT_ACCOUNT} to dana@example.test`, + ); + expect(result.text).toContain("starts a new thread"); + // Where to verify it. A Bot that cannot find its own sent mail concludes it was never sent. + expect(result.text).toContain("A copy is in folder Sent."); + }); + + test("refuses a message with no recipient, no subject or no body, sending nothing", async () => { + const { calls } = recordingMailbox(); + for (const args of [ + { subject: "s", body: "b" }, + { to: "dana@example.test", body: "b" }, + { to: "dana@example.test", subject: "s" }, + ]) { + const result = await callTool(CONNECTION, "send_message", args); + expect(result.isError).toBe(true); + } + expect(calls).toEqual([]); + }); + + test("a reply threads on the original's own ids and gains a Re:", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Friday it is.", + in_reply_to: 42, + }); + + expect(result.isError).toBe(false); + // Fetched first, then sent: the ids come from a message that exists rather than from a model. + expect(calls[0]).toEqual({ method: "message", mailbox: "INBOX", uid: 42 }); + expect(calls[1]).toEqual({ + method: "send", + message: { + to: "dana@example.test", + subject: "Re: The Friday numbers", + body: "Friday it is.", + reply: { + messageId: "", + // The original's chain with the original appended, which is what a mail client walks. + references: ["", ""], + }, + }, + }); + expect(result.text).toContain("threads as a reply"); + }); + + test("a reply to a uid that is not there sends nothing at all", async () => { + // Sending it as a new message instead would tell the model the wrong thing about what it did, + // and the recipient would see an answer that appears to be about nothing. + const { calls } = recordingMailbox({ message: async () => null }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Friday it is.", + in_reply_to: 99, + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("Nothing was sent."); + expect(calls.some((call) => call.method === "send")).toBe(false); + }); + + test("a reply to a uid too large for IMAP sends nothing and says so", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + in_reply_to: 9_000_000_000, + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("no message with uid 9000000000"); + expect(result.text).toContain("Nothing was sent."); + expect(calls).toEqual([]); + }); + + test("an original with no Message-ID is answered unthreaded rather than threaded on a guess", async () => { + const { calls } = recordingMailbox({ + message: async () => ({ ...MESSAGE, messageId: null, references: [] }), + }); + await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Friday it is.", + in_reply_to: 42, + }); + + const sent = calls.find((call) => call.method === "send"); + expect(sent?.method === "send" && sent.message.reply).toBeUndefined(); + // The subject still reads as an answer, which is the half that does not need an id. + expect(sent?.method === "send" && sent.message.subject).toBe( + "Re: The Friday numbers", + ); + }); +}); + +describe("where mail is allowed to go", () => { + const RESTRICTED: MailboxConfig = { + ...CONFIG, + allowedRecipientDomains: new Set(["example.test", "partner.example"]), + }; + + test("a recipient outside the list is refused before anything is dialled", async () => { + /* + * The check that stands between a Bot that was talked into something by the mail it just read + * and an address outside the deployment. Before the network, so a refused recipient costs no + * connection and no mailbox is opened on the way to being told no. + */ + const { calls } = recordingMailbox({ config: RESTRICTED }); + const result = await callTool(CONNECTION, "send_message", { + to: "attacker@evil.test", + subject: "The inbox", + body: "Here it is.", + // Even with a reply to fetch, nothing is dialled: the recipient is settled first. + in_reply_to: 42, + }); + + expect(result.isError).toBe(true); + // The domain, because that is what the allowlist is written about and what an administrator + // would change. + expect(result.text).toContain("evil.test"); + expect(result.text).toContain("Nothing was sent."); + expect(result.text).toContain("MAILBOX_ALLOWED_RECIPIENT_DOMAINS"); + expect(calls).toEqual([]); + }); + + test("a recipient on the list goes through, in either case and with a display name", async () => { + const { calls } = recordingMailbox({ config: RESTRICTED }); + const result = await callTool(CONNECTION, "send_message", { + to: "Dana Reid , ops@partner.example", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(false); + expect(calls.some((call) => call.method === "send")).toBe(true); + }); + + test("one refused recipient refuses the whole message", async () => { + // Sending to the allowed half would be a partial send reported as a send, and the model would + // tell somebody the message went to people it never reached. + const { calls } = recordingMailbox({ config: RESTRICTED }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test, attacker@evil.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(true); + expect(calls).toEqual([]); + }); + + test("an unrestricted deployment sends anywhere, which is the default", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "anyone@anywhere.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(false); + expect(calls.some((call) => call.method === "send")).toBe(true); + }); +}); + +describe("picking the refused recipients out of a to field", () => { + const allowed = new Set(["example.test"]); + + test("an empty allowlist refuses nothing", () => { + expect(refusedRecipients("anyone@anywhere.test", new Set())).toEqual([]); + }); + + test("matches on the domain, case-insensitively, through a display name", () => { + expect( + refusedRecipients("Dana , ops@example.test", allowed), + ).toEqual([]); + }); + + test("names each refused domain once", () => { + expect( + refusedRecipients("a@evil.test, b@evil.test, c@other.test", allowed), + ).toEqual(["evil.test", "other.test"]); + }); + + test("something that is not an address at all is refused rather than passed on", () => { + // An unparseable recipient is not one this list has cleared, and letting it through to be + // somebody else's validation error would be a hole in a safety check. + expect(refusedRecipients("not-an-address", allowed)).toEqual([ + "not-an-address", + ]); + expect(refusedRecipients("trailing@", allowed)).toEqual(["trailing@"]); + }); +}); + +describe("a folder that is not there", () => { + test("answers with the folders that are, and says the account is not chosen this way", () => { + /* + * "Mailbox doesn't exist: support" reads as a folder that happens to be missing, so a model + * retries with another folder name. The folders that do exist, plus one sentence about which + * argument picks the account, turn a loop into a correction. + */ + expect( + noSuchFolderSentence("support", "bot@example.test", [ + "INBOX", + "Sent", + "Archive", + ]), + ).toBe( + "No folder named support in bot@example.test. Folders here: INBOX, Sent, Archive. The account is chosen by `account`, not by folder; leave folder unset for INBOX.", + ); + }); + + test("says nothing about folders when the server would not list them", () => { + // A LIST that failed leaves the correction without its examples. Still better than the vendor's + // own sentence, and never a reason to lose the refusal. + const sentence = noSuchFolderSentence("support", "bot@example.test", []); + expect(sentence).not.toContain("Folders here"); + expect(sentence).toContain("The account is chosen by `account`"); + }); + + test("keeps the instruction when there are more folders than fit", () => { + /* + * Trimmed from the end of the LIST rather than the end of the sentence: the closing instruction + * is the half that changes what the model does next, and a failure is capped at 400 characters + * by the transport above this. + */ + const many = Array.from( + { length: MAX_FOLDERS_LISTED + 20 }, + (_, index) => `Folder-Number-${index}`, + ); + const sentence = noSuchFolderSentence("support", "bot@example.test", many); + + expect(sentence.length).toBeLessThanOrEqual(400); + expect(sentence).toContain("Folders here: Folder-Number-0,"); + expect(sentence).toContain(", and more."); + expect(sentence).toContain( + "The account is chosen by `account`, not by folder; leave folder unset for INBOX.", + ); + }); + + test("reaches the model whole, rather than cut by the failure cap", async () => { + const sentence = noSuchFolderSentence("support", "bot@example.test", [ + "INBOX", + "Sent", + "Archive", + "Drafts", + "Junk", + ]); + recordingMailbox({ + recent: async () => { + throw new MailboxError(sentence); + }, + }); + const result = await callTool(CONNECTION, "list_messages", { + folder: "Newsletters", + }); + + expect(result.isError).toBe(true); + expect(result.text).toBe(sentence); + }); +}); + +describe("the reply headers themselves", () => { + test("append the original to its own chain", () => { + expect( + replyFrom( + { messageId: "", references: [""] }, + "The Friday numbers", + ), + ).toEqual({ + subject: "Re: The Friday numbers", + reply: { messageId: "", references: ["", ""] }, + }); + }); + + test("do not stack Re: on a subject that already has one, in any case", () => { + for (const subject of ["Re: numbers", "RE: numbers", "re: numbers"]) { + expect( + replyFrom({ messageId: "", references: [] }, subject).subject, + ).toBe(subject); + } + }); + + test("recognise the reply marker other mail clients write", () => { + // A client that only knew "Re:" is how a thread ends up titled "Re: AW: Re: AW: the numbers". + for (const subject of [ + "AW: numbers", + "SV:numbers", + "Antw: numbers", + "Ref : numbers", + ]) { + expect( + replyFrom({ messageId: "", references: [] }, subject).subject, + ).toBe(subject); + } + }); + + test("still prefix a subject that merely starts with those letters", () => { + // "Reference pricing" is not a reply, and neither is "Software renewal". + expect( + replyFrom({ messageId: "", references: [] }, "Reference pricing") + .subject, + ).toBe("Re: Reference pricing"); + }); + + test("are absent entirely when the original carries no Message-ID", () => { + // An In-Reply-To pointing at nothing is not a reply, and an invented id threads the answer into + // a conversation that does not exist. + expect(replyFrom({ messageId: null, references: [""] }, "n")).toEqual({ + subject: "Re: n", + }); + }); +}); + +describe("what a model is told when the mail server fails", () => { + test("the server's own sentence survives, so the fix is nameable", async () => { + recordingMailbox({ + recent: async () => { + throw new MailboxError("Mailbox does not exist"); + }, + }); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(result.isError).toBe(true); + expect(result.text).toBe("Mailbox does not exist"); + }); + + test("the password never survives, even when the server echoes it back", async () => { + /* + * Some IMAP and SMTP servers quote the offending command on a failed login, and that command is + * `LOGIN user password`. That sentence goes into an audit row and in front of a model, and + * neither is a place for this deployment's mailbox password. + */ + recordingMailbox({ + send: async () => { + throw new MailboxError( + `Invalid credentials for LOGIN bot@example.test ${PASSWORD}`, + ); + }, + }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(true); + expect(result.text).not.toContain(PASSWORD); + expect(result.text).toContain("[redacted]"); + expect(result.text).toContain("Invalid credentials"); + }); + + test("a failure is capped, because a failure is not a promise about length", async () => { + recordingMailbox({ + recent: async () => { + throw new Error("z".repeat(5_000)); + }, + }); + const result = await callTool(CONNECTION, "list_messages", {}); + expect(result.text.length).toBe(400); + }); + + test("a tool this transport does not implement is refused by name", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "delete_message", { uid: 1 }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("delete_message is not a tool Mailbox"); + expect(calls).toEqual([]); + }); +}); + +describe("redaction, on its own", () => { + test("replaces every occurrence", () => { + expect(redacted(`a ${PASSWORD} b ${PASSWORD}`, PASSWORD)).toBe( + "a [redacted] b [redacted]", + ); + }); + + test("catches the base64 forms, which are what the wire actually carries", () => { + /* + * Neither client prefers plaintext LOGIN: imapflow authenticates with AUTH=PLAIN when the + * server offers it and falls back to AUTH=LOGIN, and both put the credential on the wire + * base64-encoded. A redaction that only knew the plaintext would pass the secret straight + * through while looking like it had worked. + */ + const login = Buffer.from(PASSWORD, "utf8").toString("base64"); + const plain = Buffer.from( + `\u0000${DEFAULT_ACCOUNT}\u0000${PASSWORD}`, + "utf8", + ).toString("base64"); + + const message = `A1 BAD failed: A1 AUTHENTICATE PLAIN ${plain} / LOGIN ${login}`; + const scrubbed = redacted(message, PASSWORD, DEFAULT_ACCOUNT); + + expect(scrubbed).not.toContain(login); + expect(scrubbed).not.toContain(plain); + expect(scrubbed).not.toContain(PASSWORD); + expect(scrubbed).toContain("[redacted]"); + // The rest of the server's sentence survives, which is the whole reason it is kept at all. + expect(scrubbed).toContain("BAD failed"); + }); + + test("leaves a message alone when there is no password, or a trivial one", () => { + // Replacing a one-character secret would redact half the alphabet out of an unrelated sentence. + expect(redacted("no such mailbox", null)).toBe("no such mailbox"); + expect(redacted("no such mailbox", "x")).toBe("no such mailbox"); + }); +}); + +describe("the limit bound, on its own", () => { + test("falls back, passes through, and caps", () => { + const bounds = { fallback: 10, max: 50 }; + expect(boundedLimit(undefined, bounds)).toEqual({ + limit: 10, + capped: false, + }); + expect(boundedLimit(3, bounds)).toEqual({ limit: 3, capped: false }); + expect(boundedLimit(50, bounds)).toEqual({ limit: 50, capped: false }); + expect(boundedLimit(200, bounds)).toEqual({ limit: 50, capped: true }); + }); +}); + +describe("reading a body out of a parsed message", () => { + test("prefers the text part, whatever length it is", () => { + // A plain-text alternative saying "this message needs an HTML viewer" is still what the sender + // wrote. Preferring the HTML on length would swap the sender's own words out for markup. + expect( + readBody({ text: "short", html: "

much longer body here

" }), + ).toEqual({ body: "short", bodyLength: 5 }); + }); + + test("falls back to stripped HTML only when there is no text part", () => { + expect( + readBody({ html: "

Hello

there

" }), + ).toEqual({ body: "Hello\nthere", bodyLength: 11 }); + }); + + test("caps the body and reports the length it had", () => { + const long = "y".repeat(MAX_BODY_CHARS + 500); + const { body, bodyLength } = readBody({ text: long }); + expect(body).toHaveLength(MAX_BODY_CHARS); + expect(bodyLength).toBe(MAX_BODY_CHARS + 500); + }); + + test("a message with neither part is empty rather than undefined", () => { + expect(readBody({})).toEqual({ body: "", bodyLength: 0 }); + expect(readBody({ html: false })).toEqual({ body: "", bodyLength: 0 }); + }); +}); + +describe("stripping HTML", () => { + test("drops scripts and styles rather than reading them as prose", () => { + expect(strippedHtml("

Hi

")).toBe("Hi"); + }); + + test("keeps line structure and decodes the entities that matter", () => { + expect(strippedHtml("

a & b

c
d

")).toBe("a & b\nc\nd"); + }); +}); + +describe("the sentence a mail server actually wrote", () => { + test("prefers responseText, which is where imapflow puts it", () => { + /* + * imapflow answers every IMAP NO and BAD with an Error whose message is the fixed string + * "Command failed" and puts the server's own words on `responseText`. Reading `message` turns + * "Invalid credentials", "Mailbox does not exist" and "Over quota" into one useless sentence + * that names none of them, which is the opposite of why the vendor's wording is kept at all. + */ + const error = Object.assign(new Error("Command failed"), { + responseText: "Invalid credentials (Failure)", + responseStatus: "NO", + response: { command: "NO" }, + }); + expect(mailServerSentence(error)).toBe("Invalid credentials (Failure)"); + }); + + test("falls back to a string response, which is where nodemailer puts the SMTP reply", () => { + const error = Object.assign(new Error("Invalid login"), { + response: "535 5.7.8 Authentication credentials invalid", + }); + expect(mailServerSentence(error)).toBe( + "535 5.7.8 Authentication credentials invalid", + ); + }); + + test("never renders a response object, and falls back to the message", () => { + // `String({})` is "[object Object]", which is worse than the generic message it replaced. + const error = Object.assign(new Error("Command failed"), { + response: { command: "BAD" }, + }); + expect(mailServerSentence(error)).toBe("Command failed"); + expect(mailServerSentence(new Error("socket hang up"))).toBe( + "socket hang up", + ); + expect(mailServerSentence("plain string")).toBe("plain string"); + }); +}); + +describe("the wall clock every network operation runs against", () => { + test("ends a call that never finishes, and closes the socket on the way out", async () => { + /* + * The three timeouts imapflow is built with are INACTIVITY timeouts, so a server sending one + * byte every twenty seconds resets all of them forever and the turn never ends. The cleanup is + * the point rather than the rejection: without it the losing work keeps an authenticated + * connection this deployment has stopped waiting for. + */ + let closed = 0; + const expired = withDeadline( + () => new Promise(() => {}), + () => { + closed += 1; + }, + "reading the mailbox", + 5, + ); + + await expect(expired).rejects.toBeInstanceOf(MailboxError); + await expect(expired).rejects.toThrow("did not finish reading the mailbox"); + expect(closed).toBe(1); + }); + + test("leaves a call that finishes in time alone", async () => { + let closed = 0; + const answer = await withDeadline( + async () => "done", + () => { + closed += 1; + }, + "sending the message", + 50, + ); + + expect(answer).toBe("done"); + // Nothing to close, and the timer is cleared rather than left holding the process open. + expect(closed).toBe(0); + }); +}); + +/** + * The copy in Sent, which is the half SMTP does not do. + * + * SMTP delivers; it does not file. Without the append the account's Sent folder stays empty, + * webmail shows nothing sent, and a person checking concludes the mail never went. That happened + * here, to three messages that had all been delivered, which is why the bytes are built once and + * used twice and why a failure to file is never reported as a failure to send. + * + * These cases reach the real client rather than the stub above, through the one seam that exists + * for it: the property worth being sure about is that the bytes handed to SMTP and the bytes + * appended to IMAP are the same bytes, and no stub one level up can see both. + */ +describe("filing a copy in Sent", () => { + type Delivered = { envelope: unknown; raw: unknown }; + type Appended = { path: string; flags: unknown; raw: unknown; date: unknown }; + + /** + * The real clients, wired to a recording IMAP and a recording SMTP. + * + * `folders` is what the server would answer LIST with, and `appendFails` is the server that + * refuses the append after the mail has already gone out. + */ + function wired( + options: { + folders?: { path: string; name?: string; specialUse?: string }[]; + appendFails?: string; + } = {}, + ) { + const delivered: Delivered[] = []; + const appended: Appended[] = []; + const listed = options.folders ?? [ + { path: "INBOX", name: "INBOX" }, + { path: "Sent", name: "Sent" }, + ]; + + const clients = createMailboxClients(CONFIG, DEFAULT_ACCOUNT, PASSWORD, { + imap: () => + ({ + connect: async () => {}, + logout: async () => {}, + close: () => {}, + list: async () => listed, + append: async ( + path: string, + raw: unknown, + flags: unknown, + date: unknown, + ) => { + if (options.appendFails) throw new Error(options.appendFails); + appended.push({ path, raw, flags, date }); + return { destination: path } as never; + }, + }) as unknown as ImapLike, + smtp: () => + ({ + sendMail: async (mail: { envelope?: unknown; raw?: unknown }) => { + delivered.push({ envelope: mail.envelope, raw: mail.raw }); + return {} as never; + }, + close: () => {}, + }) as unknown as SmtpLike, + }); + + return { clients, delivered, appended }; + } + + const MAIL = { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Attached.", + }; + + test("delivers and files the very same bytes", async () => { + /* + * Composed twice would be two messages: different Message-ID, different Date, different + * boundaries. A person reading Sent would be looking at a near-copy of what the recipient got, + * and a Bot verifying its own send by Message-ID would find nothing. + */ + const { clients, delivered, appended } = wired(); + const receipt = await clients.send(MAIL); + + expect(delivered).toHaveLength(1); + expect(appended).toHaveLength(1); + expect(Buffer.isBuffer(delivered[0]?.raw)).toBe(true); + expect(appended[0]?.raw).toBe(delivered[0]?.raw); + + const raw = String(delivered[0]?.raw); + expect(raw).toContain(`From: ${DEFAULT_ACCOUNT}`); + expect(raw).toContain("To: dana@example.test"); + expect(raw).toContain("Subject: The Friday numbers"); + expect(raw).toContain("Attached."); + // The id in the receipt is the id in the bytes, so it can be looked up in Sent afterwards. + expect(receipt.messageId).not.toBeNull(); + expect(raw).toContain(String(receipt.messageId)); + + // The envelope goes with it, so delivery uses the addresses the headers name. + expect(delivered[0]?.envelope).toMatchObject({ + from: DEFAULT_ACCOUNT, + to: ["dana@example.test"], + }); + + expect(appended[0]?.path).toBe("Sent"); + // Seen, because the account wrote this message rather than receiving it. + expect(appended[0]?.flags).toEqual(["\\Seen"]); + expect(appended[0]?.date).toBeInstanceOf(Date); + expect(receipt.filedTo).toBe("Sent"); + expect(receipt.fileError).toBeNull(); + }); + + test("finds Sent by its special-use flag, whatever it is called", async () => { + // A French account's Sent folder is "Éléments envoyés", and no list of English names finds it. + const { clients, appended } = wired({ + folders: [ + { path: "INBOX", name: "INBOX" }, + { + path: "Éléments envoyés", + name: "Éléments envoyés", + specialUse: "\\Sent", + }, + ], + }); + const receipt = await clients.send(MAIL); + + expect(appended[0]?.path).toBe("Éléments envoyés"); + expect(receipt.filedTo).toBe("Éléments envoyés"); + }); + + test("falls back to the name when the server marks nothing", async () => { + const { clients, appended } = wired({ + folders: [ + { path: "INBOX", name: "INBOX" }, + { path: "Sent Items", name: "Sent Items" }, + ], + }); + const receipt = await clients.send(MAIL); + + expect(appended[0]?.path).toBe("Sent Items"); + expect(receipt.filedTo).toBe("Sent Items"); + }); + + test("an account with no Sent folder still sends, and says why there is no copy", async () => { + const { clients, delivered, appended } = wired({ + folders: [{ path: "INBOX", name: "INBOX" }], + }); + const receipt = await clients.send(MAIL); + + expect(delivered).toHaveLength(1); + expect(appended).toEqual([]); + expect(receipt.filedTo).toBeNull(); + expect(receipt.fileError).toBe("this account has no Sent folder"); + }); + + test("an append that fails is not a send that failed", async () => { + /* + * The expensive mistake this prevents: reported as a failure, a Bot sends the message again, + * and mail cannot be recalled. Delivery already happened by the time the append is attempted. + */ + const { clients, delivered } = wired({ + appendFails: "Over quota", + }); + const receipt = await clients.send(MAIL); + + expect(delivered).toHaveLength(1); + expect(receipt.messageId).not.toBeNull(); + expect(receipt.filedTo).toBeNull(); + expect(receipt.fileError).toContain("Over quota"); + }); + + test("picks the folder out of a LIST, on its own", () => { + expect( + sentFolderFrom([ + { path: "INBOX", name: "INBOX" }, + { path: "Archive", name: "Archive" }, + { path: "S", name: "S", specialUse: "\\Sent" }, + ]), + ).toBe("S"); + expect( + sentFolderFrom([ + { path: "INBOX", name: "INBOX" }, + { path: "INBOX.Sent Messages", name: "Sent Messages" }, + ]), + ).toBe("INBOX.Sent Messages"); + // Null rather than a guess: appending to the wrong folder puts outgoing mail where a person + // reads it as incoming. + expect(sentFolderFrom([{ path: "INBOX", name: "INBOX" }])).toBeNull(); + }); + + test("writes both threading headers into the bytes, once", async () => { + const composed = await composeMessage(DEFAULT_ACCOUNT, { + ...MAIL, + reply: { + messageId: "", + references: ["", ""], + }, + }); + const raw = String(composed.raw); + + expect(raw).toContain("In-Reply-To: "); + expect(raw).toContain(""); + expect(raw).toContain(`Message-ID: ${composed.messageId}`); + }); +}); + +describe("what a model is told about the copy", () => { + test("the confirmation names the folder, so a Bot can verify its own send", async () => { + recordingMailbox({ + send: async () => ({ + messageId: "", + filedTo: "Sent Items", + fileError: null, + }), + }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(false); + expect(result.text).toContain("A copy is in folder Sent Items."); + }); + + test("a copy that could not be filed is a note on a successful send, not an error", async () => { + // A model told the send failed sends it again. Mail cannot be recalled. + recordingMailbox({ + send: async () => ({ + messageId: "", + filedTo: null, + fileError: "Over quota", + }), + }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(false); + expect(result.text).toContain("Sent from"); + expect(result.text).toContain("no copy could be filed in the Sent folder"); + expect(result.text).toContain("Over quota"); + expect(result.text).toContain("do not send it again"); + }); + + test("a mail server that quoted the password into the filing failure is scrubbed too", async () => { + // The filing failure is a sentence a server wrote, on an authenticated connection, and it + // reaches a model and an audit row like any other. + recordingMailbox({ + send: async () => ({ + messageId: null, + filedTo: null, + fileError: `NO [AUTHENTICATIONFAILED] LOGIN ${PASSWORD}`, + }), + }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(false); + expect(result.text).not.toContain(PASSWORD); + expect(result.text).toContain("[redacted]"); + }); +}); diff --git a/server/tests/config.test.ts b/server/tests/config.test.ts index 5b1d9ce2a..4d95fa6ab 100644 --- a/server/tests/config.test.ts +++ b/server/tests/config.test.ts @@ -726,3 +726,178 @@ describe("how far a Bot may hand work on", () => { ).toThrow("BOT_HANDOFF_MAX_PER_RUN"); }); }); + +/** + * The mailbox, which is three variables that mean nothing apart and secrets that are not here. + * + * The absent case is the one worth pinning: it is the ordinary state of every deployment that does + * not want this, and it must not be a start-up failure: the catalogue entry stays admissible and + * grantable either way, and a tool call is where a deployment finds out. + */ +describe("the deployment's mailbox", () => { + const MAILBOX = { + MAILBOX_IMAP_HOST: "imap.example.test", + MAILBOX_SMTP_HOST: "smtp.example.test", + MAILBOX_USERS: "bot@example.test", + }; + + test("is absent when none of it is set, without refusing to start", () => { + expect(loadConfig(baseEnvironment).mailbox).toBeUndefined(); + }); + + test("defaults both ports to the implicit-TLS ones", () => { + // 993 and 465 rather than the STARTTLS ports, so the connection is encrypted before the + // password is sent rather than negotiating for it in the clear. + expect(loadConfig({ ...baseEnvironment, ...MAILBOX }).mailbox).toEqual({ + imapHost: "imap.example.test", + imapPort: 993, + smtpHost: "smtp.example.test", + smtpPort: 465, + users: ["bot@example.test"], + // Unset means anywhere, which is the behaviour every deployment had before the list existed. + allowedRecipientDomains: new Set(), + }); + }); + + test("reads several accounts, in order, with the first as the default", () => { + // One pair of hosts, an account each: the shared-hosting shape. The order is a decision, since + // the first is what a tool call that named no account works in. + expect( + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_USERS: + "Support@Example.test, sales@example.test ,billing@example.test", + }).mailbox?.users, + ).toEqual([ + "support@example.test", + "sales@example.test", + "billing@example.test", + ]); + }); + + test("deduplicates accounts that differ only in case", () => { + // The address is also the key of the vault credential holding that account's password, so two + // spellings of one mailbox would be a second account nothing can ever unlock. + expect( + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_USERS: "bot@example.test,BOT@example.test", + }).mailbox?.users, + ).toEqual(["bot@example.test"]); + }); + + test("still reads the singular MAILBOX_USER, as a list of one", () => { + // The variable this feature shipped with. A deployment that already has one should not have to + // be edited to keep working. + expect( + loadConfig({ + ...baseEnvironment, + MAILBOX_IMAP_HOST: "imap.example.test", + MAILBOX_SMTP_HOST: "smtp.example.test", + MAILBOX_USER: "Bot@Example.test", + }).mailbox?.users, + ).toEqual(["bot@example.test"]); + }); + + test("refuses both spellings at once, naming the conflict", () => { + // Two answers to the same question. Merging them or preferring one silently would be guessing + // at an intention nothing here can read. + expect(() => + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_USER: "other@example.test", + }), + ).toThrow("MAILBOX_USERS and MAILBOX_USER are both set"); + }); + + test("refuses an account that is not an address, naming it", () => { + // It would otherwise become an account a model can select, a credential key an administrator + // cannot guess, and a login failure at run time in front of somebody. + expect(() => + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_USERS: "bot@example.test,not-an-address", + }), + ).toThrow('MAILBOX_USERS entry "not-an-address"'); + expect(() => + loadConfig({ + ...baseEnvironment, + MAILBOX_IMAP_HOST: "imap.example.test", + MAILBOX_SMTP_HOST: "smtp.example.test", + MAILBOX_USER: "bot at example.test", + }), + ).toThrow('MAILBOX_USER entry "bot at example.test"'); + }); + + test("reads the recipient allowlist, lower-cased and without the @ people write", () => { + expect( + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_ALLOWED_RECIPIENT_DOMAINS: "Example.com, @partner.example", + }).mailbox?.allowedRecipientDomains, + ).toEqual(new Set(["example.com", "partner.example"])); + }); + + test("an emptied allowlist means anywhere, not nowhere", () => { + // A blanked line in a .env has switched the restriction off. The opposite reading would be a + // mailbox that silently refuses everybody. + expect( + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_ALLOWED_RECIPIENT_DOMAINS: " ", + }).mailbox?.allowedRecipientDomains.size, + ).toBe(0); + }); + + test("refuses an allowlist entry that is not a domain", () => { + // A safety list written as an address would match nothing while looking like a restriction. + for (const entry of ["sales@example.com", "https://example.com", "a b"]) { + expect(() => + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_ALLOWED_RECIPIENT_DOMAINS: entry, + }), + ).toThrow("MAILBOX_ALLOWED_RECIPIENT_DOMAINS"); + } + }); + + test("takes ports a deployment names", () => { + expect( + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_IMAP_PORT: "1993", + MAILBOX_SMTP_PORT: "2465", + }).mailbox, + ).toMatchObject({ imapPort: 1993, smtpPort: 2465 }); + }); + + test("refuses half a mailbox, naming what is missing", () => { + // Booting with a host and no user is a deployment where every mail tool fails at the first + // login, at run time, in front of somebody, with nothing but an auth error to go on. + expect(() => + loadConfig({ + ...baseEnvironment, + MAILBOX_IMAP_HOST: "imap.example.test", + }), + ).toThrow("MAILBOX_SMTP_HOST, MAILBOX_USERS"); + expect(() => + loadConfig({ ...baseEnvironment, MAILBOX_USERS: "bot@example.test" }), + ).toThrow("MAILBOX_IMAP_HOST, MAILBOX_SMTP_HOST"); + }); + + test("refuses a port that is not a port, rather than falling back to the default", () => { + for (const port of ["nine-nine-three", "0", "70000", "993.5"]) { + expect(() => + loadConfig({ ...baseEnvironment, ...MAILBOX, MAILBOX_IMAP_PORT: port }), + ).toThrow("MAILBOX_IMAP_PORT must be a port number between 1 and 65535"); + } + }); +}); diff --git a/server/tests/plugin-catalogue.test.ts b/server/tests/plugin-catalogue.test.ts index d6b533a4a..79fc22f6d 100644 --- a/server/tests/plugin-catalogue.test.ts +++ b/server/tests/plugin-catalogue.test.ts @@ -93,10 +93,15 @@ describe("which servers this deployment will talk to", () => { expect(entry.hostPattern?.startsWith("^")).toBe(true); expect(entry.hostPattern?.endsWith("$")).toBe(true); } else if (entry.auth.kind === "builtin") { - // First-party and in-process: there is no host outside this process to reach, so the - // https requirement below does not apply. Asserted positively instead, so this branch - // cannot quietly become a loophole for a future entry that DOES dial a real host. - expect(entry.host).toBe("builtin://routines"); + /* + * First-party and in-process: there is no host outside this process to reach, so the + * https requirement below does not apply. Asserted against a list rather than against the + * `builtin://` scheme, so this branch cannot quietly become a loophole for a future entry + * that DOES dial a real host: adding one means adding it here, deliberately. + */ + expect(["builtin://routines", "builtin://mailbox"]).toContain( + entry.host, + ); } else { expect(entry.host.startsWith("https://")).toBe(true); } @@ -105,6 +110,19 @@ describe("which servers this deployment will talk to", () => { }); describe("whose credential a server uses", () => { + test("a builtin entry says whether it is reached as the asker or as the deployment", () => { + /* + * The one question the audit trail asks about a call with no vendor. Routines touches the + * asker's own rows; Mailbox opens one mailbox the deployment owns, on a password the deployment + * holds. Recording the second as the asker would put a person's id on a row describing access + * that was never theirs, so every builtin has to state it rather than inherit a guess. + */ + for (const entry of CATALOGUE) { + if (entry.auth.kind !== "builtin") continue; + expect(["actor", "deployment"]).toContain(entry.auth.reachedAs); + } + }); + test("every entry says which, rather than leaving it to be inferred", () => { // The whole point of replacing a `needsCredential` boolean. "Needs a credential" did not say // whose, and a reader who guessed would guess the deployment's, which for a user-oauth vendor @@ -252,7 +270,9 @@ describe("Routines", () => { }); test("has no credential, because there is nothing to authenticate to", () => { - expect(entry?.auth.kind).toBe("builtin"); + // Reached as the person asking, unlike the other builtin: a routine is theirs, and the audit + // trail should say whose rows a call touched. + expect(entry?.auth).toEqual({ kind: "builtin", reachedAs: "actor" }); }); test("is reached through the builtin transport, not a vendor", () => { @@ -285,6 +305,50 @@ describe("Routines", () => { }); }); +describe("Mailbox", () => { + const entry = catalogueEntry("mailbox"); + + test("is in the catalogue and resolves to its own builtin address", () => { + expect(entry).not.toBeNull(); + expect(resolveServerUrl("mailbox")?.url).toBe("builtin://mailbox"); + }); + + test("has no per-person credential, because there is one mailbox and it is the deployment's", () => { + expect(entry?.auth).toEqual({ kind: "builtin", reachedAs: "deployment" }); + // `builtin` takes no credential from the server row, so nothing an administrator points at this + // entry is ever spent. The mailbox password is resolved from the vault by its own key instead. + expect(serverCredentialKind(entry as CatalogueEntry)).toBeNull(); + }); + + test("is reached through its own builtin transport, not a vendor and not Routines", () => { + expect(entry?.transport).toBe("builtin-mailbox"); + }); + + test("pins the exact write list, so a dropped or renamed entry fails here", () => { + // One write, and the reason this entry is grantable per tool: reading mail is recoverable and + // sending it is not. + expect(entry?.writeTools).toEqual(["send_message"]); + }); + + test("classifies its tools the same way every other vendor's are classified", () => { + expect(classifyTool(entry, "send_message", true)).toBe("write"); + expect(classifyTool(entry, "list_messages", true)).toBe("read"); + expect(classifyTool(entry, "read_message", true)).toBe("read"); + expect(classifyTool(entry, "search_messages", true)).toBe("read"); + // A name nothing here has vouched for is a write, the same as for any other vendor. + expect(classifyTool(entry, "delete_message", false)).toBe("write"); + // Every tool, advertised or not, is a write when the server never said it was advertised. + for (const name of [ + "send_message", + "list_messages", + "read_message", + "search_messages", + ]) { + expect(classifyTool(entry, name, false)).toBe("write"); + } + }); +}); + describe("what a tool does", () => { const drive = catalogueEntry("google-drive")!; diff --git a/server/tests/plugin-reached-as.integration.test.ts b/server/tests/plugin-reached-as.integration.test.ts new file mode 100644 index 000000000..dda20c01c --- /dev/null +++ b/server/tests/plugin-reached-as.integration.test.ts @@ -0,0 +1,171 @@ +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { randomUUID } from "node:crypto"; +import { and, eq, inArray } from "drizzle-orm"; +import { createAuditStore } from "../src/audit"; +import type { ActionPolicy } from "../src/computer/policy"; +import { createDatabase } from "../src/db/client"; +import { + agents, + auditEvents, + mcpServers, + pluginGrants, +} from "../src/db/schema"; +import { catalogueEntry } from "../src/plugins/catalogue"; +import { createPluginStore } from "../src/plugins/store"; +import { TEST_POOL } from "./support/database"; + +/** + * Whose access a call ran on, as the audit trail records it, for the two connectors with no vendor. + * + * The catalogue says which each builtin is (`reachedAs` on the entry) and that is asserted where it + * is written. This file asserts the half that actually matters: that the value reaches the ROW. The + * question an investigation asks of a per-person connector is "who did this run reach as", and the + * two builtins answer it differently for a real reason. A routine touches the asking person's own + * rows, so the row names them. The mailbox is one mailbox belonging to the deployment, opened on a + * password the deployment holds, so a row naming the asker would attribute access to somebody who + * never had it. + * + * Nothing is dialled: the vendor is injected, so neither builtin transport runs and no mailbox or + * routine store is needed. What is under test is the row, not the tool. + */ + +const database = createDatabase( + process.env.DATABASE_URL ?? + "postgres://openbot:openbot@localhost:5432/openbot", + TEST_POOL, +); + +const suite = randomUUID().slice(0, 8); +const botId = `agent_reached_as_${suite}`; +const actorId = `person_${suite}@openbot.local`; + +const CALLS = [ + { serverId: "routines", toolName: "list_routines", reachedAs: actorId }, + { serverId: "mailbox", toolName: "list_messages", reachedAs: "deployment" }, +] as const; + +const policy: ActionPolicy = { mode: "enforce", deny: [], allow: ["true"] }; + +/** Which of the two server rows this suite created, so it removes only those. */ +const created: string[] = []; + +const store = createPluginStore({ + database, + auditStore: createAuditStore(database), + credentials: { + // A builtin server row holds no credential, so nothing here is ever read. Loud rather than + // absent: a read would mean this file had started exercising something it does not claim to. + readSecret: async () => { + throw new Error("a builtin server has no credential to read"); + }, + create: async () => { + throw new Error("this suite does not write credentials"); + }, + updateSecret: async () => { + throw new Error("this suite does not write credentials"); + }, + revoke: async () => new Date(), + }, + encryptionKey: "x".repeat(44), + policy: () => policy, + // No transport runs. The row is written from the entry and the actor, and both are known before + // anything would have been dialled. + callVendor: async () => ({ text: "ok", isError: false }), +}); + +beforeAll(async () => { + await database + .insert(agents) + .values({ id: botId, name: botId, type: "remote_ag_ui", configuration: {} }) + .onConflictDoNothing(); + + for (const { serverId, toolName } of CALLS) { + const entry = catalogueEntry(serverId); + if (!entry) throw new Error(`${serverId} is not in the catalogue`); + + const existing = await database + .select({ id: mcpServers.id }) + .from(mcpServers) + .where(eq(mcpServers.id, serverId)); + if (existing.length === 0) created.push(serverId); + + // Written directly rather than through addServer, so the test needs nothing to be reachable. + await database + .insert(mcpServers) + .values({ + id: serverId, + title: entry.title, + vendor: entry.vendor, + url: entry.host as string, + provenance: "first-party", + }) + .onConflictDoNothing(); + + await store.grant( + "mcp", + `${serverId}/${toolName}`, + botId, + "admin@openbot.local", + ); + } +}); + +afterAll(async () => { + // Scoped to this suite's own Bot. The refs name real connectors, so a delete by ref alone would + // take an administrator's grants for a Bot people use. + await database.delete(pluginGrants).where( + and( + inArray( + pluginGrants.ref, + CALLS.map(({ serverId, toolName }) => `${serverId}/${toolName}`), + ), + eq(pluginGrants.agentId, botId), + ), + ); + await database.delete(agents).where(eq(agents.id, botId)); + // A server row is deployment configuration. Removed only where this suite is what added it. + if (created.length > 0) { + await database.delete(mcpServers).where(inArray(mcpServers.id, created)); + } +}); + +describe("whose access a builtin call is recorded as", () => { + for (const { serverId, toolName, reachedAs } of CALLS) { + test(`${serverId} is reached as ${reachedAs === "deployment" ? "the deployment" : "the asker"}`, async () => { + const ref = `${serverId}/${toolName}`; + const result = await store.callTool({ + ref, + args: {}, + botId, + actorId, + }); + expect(result.isError).toBe(false); + + const rows = await database + .select({ + eventType: auditEvents.eventType, + payload: auditEvents.payload, + }) + .from(auditEvents) + .where( + and( + eq(auditEvents.targetType, "mcp_tool"), + eq(auditEvents.targetId, ref), + ), + ); + + const mine = rows.filter( + (row) => + row.eventType === "mcp.call_succeeded" && + (row.payload as { bot?: string }).bot === botId, + ); + expect(mine.length).toBeGreaterThan(0); + expect((mine[0].payload as { reachedAs?: string }).reachedAs).toBe( + reachedAs, + ); + // The actor is on the row either way. What `reachedAs` settles is whether the access was + // theirs, which is a different question from who asked. + expect((mine[0].payload as { actor?: string }).actor).toBe(actorId); + }); + } +});