diff --git a/.env.example b/.env.example index 20128287..b4debe2a 100644 --- a/.env.example +++ b/.env.example @@ -330,3 +330,43 @@ AGENT_TOOL_TOKEN= # Do not accept a default in production. WORKER_SHARED_SECRET= +# The deployment's mailbox, which a Bot reads and answers through the Mailbox connector. All three +# are needed together: set one and the server refuses to start naming the others, rather than booting +# with half a mailbox that fails at the first login in front of somebody. Leave all three unset and +# the connector is still there, still grantable, and every call answers with what to set. +# +# MAILBOX_USERS is a comma-separated list of addresses, all on the hosts above, which is the shape a +# shared host gives you: support@, sales@ and billing@ are three mailboxes on one IMAP and one SMTP +# server. The FIRST is the default, the one a tool call that named no account works in, so the order +# is a decision rather than a detail. One address is a perfectly good list. +# +# MAILBOX_USER, the singular this feature shipped with, is still read as a list of one, so an +# existing deployment keeps working unedited. Setting both refuses to start: they are two answers to +# the same question. +# +# THE PASSWORDS ARE NOT HERE, and that is deliberate. They are the only secrets in this feature, so +# they live where this deployment's other secrets live: the encrypted vault. Store one per account +# at /admin/credentials with kind `mcp`, provider `mailbox` and key id the address itself, and +# rotate them there. A password in this file is a password in a repository, a compose file and every +# process list that ever read it. +MAILBOX_IMAP_HOST= +MAILBOX_SMTP_HOST= +MAILBOX_USERS= +# MAILBOX_USERS=support@example.com,sales@example.com +# +# Both default to the implicit-TLS ports, 993 and 465, so the connection is encrypted before the +# password is sent rather than negotiating for it in the clear. Set them only for a server that +# listens elsewhere. +# MAILBOX_IMAP_PORT=993 +# MAILBOX_SMTP_PORT=465 +# +# Where a Bot may send mail, as a comma-separated list of domains. Unset or empty means anywhere, +# which is the default and the behaviour every deployment had before this existed. +# +# Worth setting, because the policy engine cannot do this job: a rule sees a tool call's name and +# effect, never its arguments, so the only rule you can write about send_message covers all of it or +# none of it. Meanwhile the reading tools bring text somebody else wrote into a model's context, so +# a Bot that can read and send without limits is one persuasive message away from mailing the inbox +# to whoever asked for it. This bounds where anything can go; an approval rule on send_message in +# your boundaries decides whether it goes at all. Deployments that care should have both. +# MAILBOX_ALLOWED_RECIPIENT_DOMAINS=example.com,partner.example diff --git a/CHANGELOG.md b/CHANGELOG.md index d446e043..a830b7a6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,48 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged. ## Unreleased +### A Bot can read and answer the deployment's own mailbox + +A new catalogue entry, Mailbox, appears at `/admin/plugins/mailbox`. It gives a Bot four tools +against the deployment's own mail: `list_messages`, `read_message`, `search_messages` and +`send_message`, the last of which threads a reply when it is given the uid of the message being +answered, and files a copy of what it sent in the account's Sent folder so webmail and the Bot both +find the outgoing mail. The mailbox belongs to the deployment rather than to whoever is asking, so +everybody granted the tools reads the same mail. Decide that before granting it, because it is the +whole of the access model. + +Configure it with `MAILBOX_IMAP_HOST`, `MAILBOX_SMTP_HOST` and `MAILBOX_USERS`, which are needed +together: set one of the three and the server refuses to start naming the others, rather than +booting with half a mailbox that fails at the first login in front of somebody. `MAILBOX_USERS` is +a comma-separated list, so several accounts on one shared host are one deployment, and the first +listed is the default that a call naming no `account` works in. `MAILBOX_USER` is still read as a +list of one for a deployment that already had it, and setting both refuses to start. +`MAILBOX_IMAP_PORT` and `MAILBOX_SMTP_PORT` default to the implicit-TLS ports, 993 and 465. Leave +the three required variables unset and the connector is still listed and still grantable, and every +call answers with the sentence naming what to set. + +The passwords are not environment variables. Each account's password is a row in the encrypted +credential vault at `/admin/credentials`, kind `mcp`, provider `mailbox`, key id the address itself, +so a deployment with three mailboxes holds three rows and rotates or revokes each on its own. A +password is read at the moment a call needs it, so a rotation takes effect on the next tool call +rather than on the next restart, and a revocation stops that account within a call. Nothing prints +one: IMAP command logging is off, and a mail server's failure sentence is scrubbed of the plaintext +and of the base64 forms before it reaches an audit row, a transcript or a model. + +`MAILBOX_ALLOWED_RECIPIENT_DOMAINS` bounds where mail may go, refusing a recipient outside the list +before any connection is opened. Unset means anywhere, which is the behaviour every deployment had +before this existed. It is there because a policy rule cannot do this job: a rule sees a tool call's +name and its effect, never its arguments, so the only thing it can express about `send_message` is +whether it happens at all. An approval rule decides that mail goes; the allowlist decides where it +may go. A deployment that cares should set both. + +Grants are per tool and not per account, so "may read the mail" and "may answer it" are two separate +decisions, while "may read support@ but not billing@" is not one this deployment can express: if an +account must stay out of a Bot's reach, do not configure it here. Every call takes the same route as +any other connector's. The Bot's grant is checked, the policy is evaluated with the tool's effect +(`send_message` as a write, the other three as reads), and an audit row is written, before any mail +server is dialled. + ### Coworkers are made in a wizard and managed in a dialog Creating a coworker is now a three-step wizard — who it is, who may see it, then where it runs, diff --git a/bun.lock b/bun.lock index 415b9ce6..d4f03be1 100644 --- a/bun.lock +++ b/bun.lock @@ -72,6 +72,9 @@ "cron-parser": "^5", "drizzle-orm": "^0.45.2", "hono": "^4.10.0", + "imapflow": "^1.7.8", + "mailparser": "^3.9.20", + "nodemailer": "^9.1.1", "postgres": "^3.4.9", "rxjs": "7.8.1", "yaml": "^2.9.0", @@ -79,6 +82,8 @@ }, "devDependencies": { "@copilotkit/aimock": "1.39.0", + "@types/mailparser": "^3.4.6", + "@types/nodemailer": "^7.0.4", "drizzle-kit": "^0.31.10", "eventsource": "3.0.7", }, @@ -589,6 +594,8 @@ "@segment/analytics-node": ["@segment/analytics-node@2.3.0", "", { "dependencies": { "@lukeed/uuid": "^2.0.0", "@segment/analytics-core": "1.8.2", "@segment/analytics-generic-utils": "1.2.0", "buffer": "^6.0.3", "jose": "^5.1.0", "node-fetch": "^2.6.7", "tslib": "^2.4.1" } }, "sha512-fOXLL8uY0uAWw/sTLmezze80hj8YGgXXlAfvSS6TUmivk4D/SP0C0sxnbpFdkUzWg2zT64qWIZj26afEtSnxUA=="], + "@selderee/plugin-htmlparser2": ["@selderee/plugin-htmlparser2@0.12.0", "", { "dependencies": { "domelementtype": "~2.3.0", "domhandler": "~5.0.3" }, "peerDependencies": { "selderee": "~0.12.0" } }, "sha512-oELmoyA6ML9jDRMV3kgcMQFKxUfBU0yFVn6yTctVaLT5ygXnxH52I3TZEgV9EhXJC68/uFvE5Daj1/25c0Xa/A=="], + "@shadcn/react": ["@shadcn/react@0.3.0", "", { "peerDependencies": { "@types/react": ">=19", "react": ">=19" }, "optionalPeers": ["@types/react", "react"] }, "sha512-iKN0NuYe850VDHlxEfvppFrpa7CMV3zArTHusXlaSmeFeQhohGbIqclv6WwPTs/44I8EkXQpcuRt6sXEVKkWqQ=="], "@shikijs/core": ["@shikijs/core@3.23.0", "", { "dependencies": { "@shikijs/types": "3.23.0", "@shikijs/vscode-textmate": "^10.0.2", "@types/hast": "^3.0.4", "hast-util-to-html": "^9.0.5" } }, "sha512-NSWQz0riNb67xthdm5br6lAkvpDJRTgB36fxlo37ZzM2yq0PQFFzbd8psqC2XMPgCzo1fW6cVi18+ArJ44wqgA=="], @@ -803,12 +810,16 @@ "@types/katex": ["@types/katex@0.16.8", "", {}, "sha512-trgaNyfU+Xh2Tc+ABIb44a5AYUpicB3uwirOioeOkNPPbmgRNtcWyDeeFRzjPZENO9Vq8gvVqfhaaXWLlevVwg=="], + "@types/mailparser": ["@types/mailparser@3.4.6", "", { "dependencies": { "@types/node": "*", "iconv-lite": "^0.6.3" } }, "sha512-wVV3cnIKzxTffaPH8iRnddX1zahbYB1ZEoAxyhoBo3TBCBuK6nZ8M8JYO/RhsCuuBVOw/DEN/t/ENbruwlxn6Q=="], + "@types/mdast": ["@types/mdast@4.0.4", "", { "dependencies": { "@types/unist": "*" } }, "sha512-kGaNbPh1k7AFzgpud/gMdvIm5xuECykRR+JnWKQno9TAXVa6WIVCGTPvYGekIDL4uwCZQSYbUxNBSb1aUo79oA=="], "@types/ms": ["@types/ms@2.1.0", "", {}, "sha512-GsCCIZDE/p3i96vtEqx+7dBUGXrc7zeSK3wwPHIaRThS+9OhWIXRqzs4d6k1SVU8g91DrNRWxWUGhp5KXQb2VA=="], "@types/node": ["@types/node@26.2.0", "", { "dependencies": { "undici-types": "~8.3.0" } }, "sha512-5IviulTZeRNp2vAJ514cc/HUlY5nZ9fCbq9DMyC52BrhFZACo3nI0R7qBxhQmo/d27NFe96ur/b7Wwxklda+kg=="], + "@types/nodemailer": ["@types/nodemailer@7.0.12", "", { "dependencies": { "@types/node": "*" } }, "sha512-80vKwiIsVSyFA1rRovH59jNPLBOuc6dRZIHEu40gXTkBkZnQv8vog1xSGEb9j5q/tdMAs5ivvDR2pLTU0hGHXA=="], + "@types/prop-types": ["@types/prop-types@15.7.15", "", {}, "sha512-F6bEyamV9jKGAFBEmlQnesRPGOQqS2+Uwi0Em15xenOxHaf2hv6L8YCVn3rPdPJOiJfPiCnLIRyvwVaqMY3MIw=="], "@types/qs": ["@types/qs@6.15.1", "", {}, "sha512-GZHUBZR9hckSUhrxmp1nG6NwdpM9fCunJwyThLW1X3AyHgd9IlHb6VANpQQqDr2o/qQp6McZ3y/IA2rVzKzSbw=="], @@ -869,6 +880,8 @@ "@xmldom/xmldom": ["@xmldom/xmldom@0.9.12", "", {}, "sha512-5AXjrcMClTryPe9LgZrygpB1lj7s0S9E0+W+AHaVKAVyHanafK86iPSvG5xHVSp/jC+VH1UXu0TAEmY279xH7A=="], + "@zone-eu/mailsplit": ["@zone-eu/mailsplit@5.4.16", "", { "dependencies": { "libbase64": "1.3.0", "libmime": "5.4.3", "libqp": "2.1.1" } }, "sha512-zQ9iXvlT3Wi/hazeC1MdI4rQc1UJwJ6IQ6QzSZ5KDxLZZWQSazWLOzImLFluXadKShJ9WJvI1xH+AyVS8b9azg=="], + "abort-controller": ["abort-controller@3.0.0", "", { "dependencies": { "event-target-shim": "^5.0.0" } }, "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg=="], "accepts": ["accepts@1.3.8", "", { "dependencies": { "mime-types": "~2.1.34", "negotiator": "0.6.3" } }, "sha512-PYAthTa2m2VKxuvSD3DPC/Gy+U+sOA1LAuT8mkmRuvw+NACSaeXEQ+NHcVF7rONl6qcaxV3Uuemwawk+7+SJLw=="], @@ -1129,6 +1142,8 @@ "deepmerge": ["deepmerge@4.3.1", "", {}, "sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A=="], + "deepmerge-ts": ["deepmerge-ts@8.0.2", "", {}, "sha512-uqbvqLUMrc6p0MO+WBRtTxY55hmyh94WRwI5a++PZe54X+bfVh59FSN7uWCBCW1CCVjzjnrwzfI8zidE2obMMw=="], + "default-browser": ["default-browser@5.5.1", "", { "dependencies": { "bundle-name": "^4.1.0", "default-browser-id": "^5.0.0" } }, "sha512-m1pAzaJgZ/gssEqlOhJkPJp8Xly7QyW6xcrkUa2KKcDeDSEMP7X8xipU3snUcfisTQx0w1AGae+9UtJSfVnXGw=="], "default-browser-id": ["default-browser-id@5.0.1", "", {}, "sha512-x1VCxdX4t+8wVfd1so/9w+vQ4vx7lKd2Qp5tDRutErwmR85OgmfX7RlLRMWafRMY7hbEiXIbudNrjOAPa/hL8Q=="], @@ -1157,8 +1172,16 @@ "dom-accessibility-api": ["dom-accessibility-api@0.5.16", "", {}, "sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg=="], + "dom-serializer": ["dom-serializer@2.0.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.2", "entities": "^4.2.0" } }, "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg=="], + + "domelementtype": ["domelementtype@2.3.0", "", {}, "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw=="], + + "domhandler": ["domhandler@5.0.3", "", { "dependencies": { "domelementtype": "^2.3.0" } }, "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w=="], + "dompurify": ["dompurify@3.4.14", "", { "optionalDependencies": { "@types/trusted-types": "^2.0.7" } }, "sha512-dVoH9z+MY+C9IilgGCk3YfFqjLi3fChm2OiKJMzh6axrJ5qwxqWaZamgmHrpv22CN/KdbZJuGEGgfQoL00LTdg=="], + "domutils": ["domutils@3.2.2", "", { "dependencies": { "dom-serializer": "^2.0.0", "domelementtype": "^2.3.0", "domhandler": "^5.0.3" } }, "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw=="], + "dot-prop": ["dot-prop@6.0.1", "", { "dependencies": { "is-obj": "^2.0.0" } }, "sha512-tE7ztYzXHIeyvc7N+hR3oi7FIbf/NIjVP9hmAt3yMXzrQ072/fpjGLx2GxNxGxUl5V73MEqYzioOMoVhGMJ5cA=="], "dotenv": ["dotenv@17.4.2", "", {}, "sha512-nI4U3TottKAcAD9LLud4Cb7b2QztQMUEfHbvhTH09bqXTxnSie8WnjPALV/WMCrJZ6UV/qHJ6L03OqO3LcdYZw=="], @@ -1181,6 +1204,8 @@ "encodeurl": ["encodeurl@2.0.0", "", {}, "sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg=="], + "encoding-japanese": ["encoding-japanese@2.3.0", "", {}, "sha512-eQyh1vzHz13DUkZcJO+0IOAoKXRQwKV5IBffeuYsWZyRLGiSzfzXObCqWvqFXdX0UU8qOk+lBXbkUhMCpdJe4Q=="], + "end-of-stream": ["end-of-stream@1.4.5", "", { "dependencies": { "once": "^1.4.0" } }, "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg=="], "enhanced-resolve": ["enhanced-resolve@5.24.5", "", { "dependencies": { "graceful-fs": "^4.2.4", "tapable": "^2.3.3" } }, "sha512-L1l8TNvomm6UVW5B253AGxQagSQr+vGwhMlrrfRS2qmhx46AMpMVJKQYLvWYbysTMY8VoicOvzHzoHMbyzB+4A=="], @@ -1367,14 +1392,20 @@ "hastscript": ["hastscript@9.0.1", "", { "dependencies": { "@types/hast": "^3.0.0", "comma-separated-tokens": "^2.0.0", "hast-util-parse-selector": "^4.0.0", "property-information": "^7.0.0", "space-separated-tokens": "^2.0.0" } }, "sha512-g7df9rMFX/SPi34tyGCyUBREQoKkapwdY/T04Qn9TDWfHhAYt4/I0gMVirzK5wEzeUqIjEB+LXC/ypb7Aqno5w=="], + "he": ["he@1.2.0", "", { "bin": { "he": "bin/he" } }, "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw=="], + "help-me": ["help-me@5.0.0", "", {}, "sha512-7xgomUX6ADmcYzFik0HzAxh/73YlKR9bmFzf51CZwR+b6YtzU2m0u49hQCqV6SvlqIqsaxovfwdvbnsw3b/zpg=="], "hono": ["hono@4.13.4", "", {}, "sha512-AGEwKIyRMHRv1t8Wjwa3LHxQ61X5CqrdFT+4BRNTpqS5aJNnpl5WLjADb7vFlJzI/8uK7T5QLVApCMQKNa3LgQ=="], + "html-to-text": ["html-to-text@10.0.1", "", { "dependencies": { "@selderee/plugin-htmlparser2": "~0.12.0", "deepmerge-ts": "^8.0.1", "dom-serializer": "^2.0.0", "htmlparser2": "^10.1.0", "selderee": "~0.12.0" } }, "sha512-GiVhRI1BatGARSCmlXWNCjDT0cWrwBWoeduLoV0WSKAgaV/wa+hUWy5LiQLUs4UwiUrE52ZCMfBGiKD87TDPrg=="], + "html-url-attributes": ["html-url-attributes@3.0.1", "", {}, "sha512-ol6UPyBWqsrO6EJySPz2O7ZSr856WDrEzM5zMqp+FJJLGMW35cLYmmZnl0vztAZxRUoNZJFTCohfjuIJ8I4QBQ=="], "html-void-elements": ["html-void-elements@3.0.0", "", {}, "sha512-bEqo66MRXsUGxWHV5IP0PUiAWwoEjba4VCzg0LjFJBpchPaTfyfCKTG6bc5F8ucKec3q5y6qOdGyYTSBEvhCrg=="], + "htmlparser2": ["htmlparser2@10.1.0", "", { "dependencies": { "domelementtype": "^2.3.0", "domhandler": "^5.0.3", "domutils": "^3.2.2", "entities": "^7.0.1" } }, "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ=="], + "http-errors": ["http-errors@2.0.1", "", { "dependencies": { "depd": "~2.0.0", "inherits": "~2.0.4", "setprototypeof": "~1.2.0", "statuses": "~2.0.2", "toidentifier": "~1.0.1" } }, "sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ=="], "http-proxy-agent": ["http-proxy-agent@7.0.2", "", { "dependencies": { "agent-base": "^7.1.0", "debug": "^4.3.4" } }, "sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig=="], @@ -1383,12 +1414,14 @@ "human-signals": ["human-signals@8.0.1", "", {}, "sha512-eKCa6bwnJhvxj14kZk5NCPc6Hb6BdsU9DZcOnmQKSnO1VKrfV0zCvtttPZUsBvjmNDn8rpcJfpwSYnHBjc95MQ=="], - "iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "iconv-lite": ["iconv-lite@0.6.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw=="], "ieee754": ["ieee754@1.2.1", "", {}, "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA=="], "ignore": ["ignore@5.3.2", "", {}, "sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g=="], + "imapflow": ["imapflow@1.7.8", "", { "dependencies": { "@zone-eu/mailsplit": "5.4.16", "encoding-japanese": "2.3.0", "iconv-lite": "0.7.3", "libbase64": "1.3.0", "libmime": "5.4.3", "libqp": "2.1.1", "pino": "10.3.1", "socks": "2.8.9" } }, "sha512-dJoCIdZOJh26Rn2PdwEzwj0bRDgGBxxX38pio534FagIHVuR2l0SAfLr6sJo32YfuJHiSs/U2ntNDHnMg3/Hlg=="], + "import-fresh": ["import-fresh@3.3.1", "", { "dependencies": { "parent-module": "^1.0.0", "resolve-from": "^4.0.0" } }, "sha512-TR3KfrTZTYLPB6jUjfx6MF9WcWrHL9su5TObK4ZkYgBdWKPOFoSoQIdEuTuR82pmtxH2spWG9h6etwfr1pLBqQ=="], "import-meta-resolve": ["import-meta-resolve@4.2.0", "", {}, "sha512-Iqv2fzaTQN28s/FwZAoFq0ZSs/7hMAHJVX+w8PZl3cY19Pxk6jFFalxQoIfW2826i/fDLXv8IiEZRIT0lDuWcg=="], @@ -1503,8 +1536,16 @@ "layout-base": ["layout-base@1.0.2", "", {}, "sha512-8h2oVEZNktL4BH2JCOI90iD1yXwL6iNW7KcCKT2QZgQJR2vbqDsldCTPRU9NifTCqHZci57XvQQ15YTu+sTYPg=="], + "leac": ["leac@0.7.0", "", {}, "sha512-qMrZeyEekgdRQ9o6a4NAB2EQZrv827GJdn1vnapwSJ90hWRB4TzUSunvacPkxQ2TnNqHNI1/zSt0hlo0crG8Jw=="], + + "libbase64": ["libbase64@1.3.0", "", {}, "sha512-GgOXd0Eo6phYgh0DJtjQ2tO8dc0IVINtZJeARPeiIJqge+HdsWSuaDTe8ztQ7j/cONByDZ3zeB325AHiv5O0dg=="], + + "libmime": ["libmime@5.4.3", "", { "dependencies": { "encoding-japanese": "2.3.0", "iconv-lite": "0.7.3", "libbase64": "1.3.0", "libqp": "2.1.1" } }, "sha512-di9BoDabBUMqjeD/wGj+hHpSgdqAph5ui7w6OdY6NpzU6O6VFLQsMOg9tqCjm/zf9OHzAM9EZxSOF7uIb8O8Hw=="], + "libphonenumber-js": ["libphonenumber-js@1.13.11", "", {}, "sha512-ETER2kMaIFTI/Nh1a8Gk03dUF/SL0VZqtI+CcVHZxp5WIHYwNS7S+uiYZDYCvLy3lOR4/DAD5jf0h5WkePPpqg=="], + "libqp": ["libqp@2.1.1", "", {}, "sha512-0Wd+GPz1O134cP62YU2GTOPNA7Qgl09XwCqM5zpBv87ERCXdfDtyKXvV7c9U22yWJh44QZqBocFnXN11K96qow=="], + "lightningcss": ["lightningcss@1.32.0", "", { "dependencies": { "detect-libc": "^2.0.3" }, "optionalDependencies": { "lightningcss-android-arm64": "1.32.0", "lightningcss-darwin-arm64": "1.32.0", "lightningcss-darwin-x64": "1.32.0", "lightningcss-freebsd-x64": "1.32.0", "lightningcss-linux-arm-gnueabihf": "1.32.0", "lightningcss-linux-arm64-gnu": "1.32.0", "lightningcss-linux-arm64-musl": "1.32.0", "lightningcss-linux-x64-gnu": "1.32.0", "lightningcss-linux-x64-musl": "1.32.0", "lightningcss-win32-arm64-msvc": "1.32.0", "lightningcss-win32-x64-msvc": "1.32.0" } }, "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ=="], "lightningcss-android-arm64": ["lightningcss-android-arm64@1.32.0", "", { "os": "android", "cpu": "arm64" }, "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg=="], @@ -1533,6 +1574,8 @@ "lines-and-columns": ["lines-and-columns@1.2.4", "", {}, "sha512-7ylylesZQ/PV29jhEDl3Ufjo6ZX7gCqJr5F7PKrqc93v7fzSymt1BpwEU8nAUXs8qzzvqhbjhK5QZg6Mt/HkBg=="], + "linkify-it": ["linkify-it@5.0.2", "", { "dependencies": { "uc.micro": "^2.0.0" } }, "sha512-ONTm2jCMAVZjgQa/Fy1kScXsuOoF5NPTsoFBdE1KVIZ2vAh/r9+Bqo+0jINCBYnavTPQZz38QzFTme79ENoN3Q=="], + "lit": ["lit@3.3.3", "", { "dependencies": { "@lit/reactive-element": "^2.1.0", "lit-element": "^4.2.0", "lit-html": "^3.3.0" } }, "sha512-fycuvZg/hkpozL00lm1pEJH5nN/lr9ZXd6mJI2HSN4+Bzc+LDNdEApJ6HFbPkdFNHLvOplIIuJvxkS4XUxqirw=="], "lit-element": ["lit-element@4.2.2", "", { "dependencies": { "@lit-labs/ssr-dom-shim": "^1.5.0", "@lit/reactive-element": "^2.1.0", "lit-html": "^3.3.0" } }, "sha512-aFKhNToWxoyhkNDmWZwEva2SlQia+jfG0fjIWV//YeTaWrVnOxD89dPKfigCUspXFmjzOEUQpOkejH5Ly6sG0w=="], @@ -1581,6 +1624,8 @@ "magic-string": ["magic-string@0.30.21", "", { "dependencies": { "@jridgewell/sourcemap-codec": "^1.5.5" } }, "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ=="], + "mailparser": ["mailparser@3.9.20", "", { "dependencies": { "@zone-eu/mailsplit": "5.4.16", "encoding-japanese": "2.3.0", "he": "1.2.0", "html-to-text": "10.0.1", "iconv-lite": "0.7.3", "libmime": "5.4.3", "linkify-it": "5.0.2", "nodemailer": "9.1.1", "punycode.js": "2.3.1", "tlds": "1.261.0" } }, "sha512-PZ9RD6B7SkmyQ9rj8JvYNS18rL6pWoNkSw9KGVuJQrrdouFxFIB05ugnR3ubInlnjLXV3KHXhGyz8MlTz1VGzw=="], + "markdown-table": ["markdown-table@3.0.4", "", {}, "sha512-wiYz4+JrLyb/DqW2hkFJxP7Vd7JuTDm77fvbM8VfEQdmSMqcImWeeRbHwZjBjIFki/VaMK2BhFi7oUUZeM5bqw=="], "marked": ["marked@17.0.6", "", { "bin": { "marked": "bin/marked.js" } }, "sha512-gB0gkNafnonOw0obSTEGZTT86IuhILt2Wfx0mWH/1Au83kybTayroZ/V6nS25mN7u8ASy+5fMhgB3XPNrOZdmA=="], @@ -1739,6 +1784,8 @@ "node-rsa": ["node-rsa@1.1.1", "", { "dependencies": { "asn1": "^0.2.4" } }, "sha512-Jd4cvbJMryN21r5HgxQOpMEqv+ooke/korixNNK3mGqfGJmy0M77WDDzo/05969+OkMy3XW1UuZsSmW9KQm7Fw=="], + "nodemailer": ["nodemailer@9.1.1", "", {}, "sha512-izw9mVKFix6YSnC9eLgV6g1opl9DUlRio9ZNcq+Wu9Ujn2UwF+8Nl0B8nz22kEC+CTZCvinkxwJ0DeFbb6NwcQ=="], + "npm-run-path": ["npm-run-path@6.0.0", "", { "dependencies": { "path-key": "^4.0.0", "unicorn-magic": "^0.3.0" } }, "sha512-9qny7Z9DsQU8Ou39ERsPU4OZQlSTP47ShQzuKZ6PRXpYLtIFgl/DEBYEXKlvcEa+9tHVcK8CF81Y2V72qaZhWA=="], "object-assign": ["object-assign@4.1.1", "", {}, "sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg=="], @@ -1791,6 +1838,8 @@ "parse5": ["parse5@7.3.0", "", { "dependencies": { "entities": "^6.0.0" } }, "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw=="], + "parseley": ["parseley@0.13.1", "", { "dependencies": { "leac": "^0.7.0", "peberminta": "^0.10.0" } }, "sha512-uNBJZzmb60l6p6VWLTmevizNAGnE0xoSf1n0B4q3ntegDNzcS68NRCcBDZTcyXHxt2XhBChsCuqj4M+nChvE/A=="], + "parseurl": ["parseurl@1.3.3", "", {}, "sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ=="], "partial-json": ["partial-json@0.1.7", "", {}, "sha512-Njv/59hHaokb/hRUjce3Hdv12wd60MtM9Z5Olmn+nehe0QDAsRtRbJPvJ0Z91TusF0SuZRIvnM+S4l6EIP8leA=="], @@ -1809,6 +1858,8 @@ "pathe": ["pathe@2.0.3", "", {}, "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w=="], + "peberminta": ["peberminta@0.10.0", "", {}, "sha512-80B2AsU+I4Qdb0ZAPSfe9UwvGzwkM37IKIFEvdS3D/3Ndgv2bsuJ0bfG1+iEYO+l7Gfd4EUJmuRyq7efLgRMzQ=="], + "phoenix": ["phoenix@1.8.12", "", {}, "sha512-svUniHb83aGh1XpYWHe4fkbfrGd6xa26TWO/3clkuNplXwgjxh7RI7GOWxf28VAgDWwQ2/yN8PxIa5dqjXOcAA=="], "picocolors": ["picocolors@1.1.1", "", {}, "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="], @@ -1863,6 +1914,8 @@ "pump": ["pump@3.0.4", "", { "dependencies": { "end-of-stream": "^1.1.0", "once": "^1.3.1" } }, "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA=="], + "punycode.js": ["punycode.js@2.3.1", "", {}, "sha512-uxFIHU0YlHYhDQtV4R9J6a52SLx28BCjT+4ieh7IGbgwVJWO+km431c4yRlREUAsAmt/uMjQUyQHNEPf0M39CA=="], + "qs": ["qs@6.15.3", "", { "dependencies": { "es-define-property": "^1.0.1", "side-channel": "^1.1.1" } }, "sha512-O9gl3zCl5h5blw1KGUzQKhA5oUXSl8rwUIM5o0S3nCXMliSvy5Dzx7/DJcI+SwgICv+IneSZwhBh1oSyEHA71A=="], "queue-microtask": ["queue-microtask@1.2.3", "", {}, "sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A=="], @@ -1979,6 +2032,8 @@ "secure-json-parse": ["secure-json-parse@2.7.0", "", {}, "sha512-6aU+Rwsezw7VR8/nyvKTx8QpWH9FrcYiXXlqC4z5d5XQBDRqtbfsRjnwGyqbi3gddNtWHuEk9OANUotL26qKUw=="], + "selderee": ["selderee@0.12.0", "", { "dependencies": { "parseley": "~0.13.1" } }, "sha512-b1YMh3+DHZp59DLna3qVwQ5iOla/nrI6mLBNW02XxU77M3046Df6VLkoaJyFz20VsGIG5kkp+FK0kg4K4HnUFw=="], + "semver": ["semver@6.3.1", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA=="], "send": ["send@0.19.2", "", { "dependencies": { "debug": "2.6.9", "depd": "2.0.0", "destroy": "1.2.0", "encodeurl": "~2.0.0", "escape-html": "~1.0.3", "etag": "~1.8.1", "fresh": "~0.5.2", "http-errors": "~2.0.1", "mime": "1.6.0", "ms": "2.1.3", "on-finished": "~2.4.1", "range-parser": "~1.2.1", "statuses": "~2.0.2" } }, "sha512-VMbMxbDeehAxpOtWJXlcUS5E8iXh6QmN+BkRX1GARS3wRaXEEgzCcB10gTQazO42tpNIya8xIyNx8fll1OFPrg=="], @@ -2081,6 +2136,8 @@ "tinyglobby": ["tinyglobby@0.2.17", "", { "dependencies": { "fdir": "^6.5.0", "picomatch": "^4.0.4" } }, "sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g=="], + "tlds": ["tlds@1.261.0", "", { "bin": { "tlds": "bin.js" } }, "sha512-QXqwfEl9ddlGBaRFXIvNKK6OhipSiLXuRuLJX5DErz0o0Q0rYxulWLdFryTkV5PkdZct5iMInwYEGe/eR++1AA=="], + "tldts": ["tldts@7.4.11", "", { "dependencies": { "tldts-core": "^7.4.11" }, "bin": { "tldts": "bin/cli.js" } }, "sha512-aBiNayCfTQxuIJBm06M+xR14cYaYlDlSXZbgsnKzKNxDKUVq7KFwTjwBSsb7m9Y5xO8WfPnBc63WaYFMTGlvqw=="], "tldts-core": ["tldts-core@7.4.11", "", {}, "sha512-CW3WN2rIIE/Of21mulhgnGOwoDyEFNygyIBOONSdyAuSATgMMUCpLeUlB+E8sAwA5xRV9hYPl+kyZ9citHCaKg=="], @@ -2115,6 +2172,8 @@ "typescript": ["typescript@5.9.3", "", { "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" } }, "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw=="], + "uc.micro": ["uc.micro@2.1.0", "", {}, "sha512-ARDJmphmdvUk6Glw7y9DQ2bFkKBHwQHLi2lsaH6PPmz/Ka9sFOBsBluozhDltWmnv9u/cF6Rt87znRTPV+yp/A=="], + "undici": ["undici@7.29.0", "", {}, "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw=="], "undici-types": ["undici-types@8.3.0", "", {}, "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ=="], @@ -2361,12 +2420,12 @@ "d3-dsv/commander": ["commander@7.2.0", "", {}, "sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw=="], - "d3-dsv/iconv-lite": ["iconv-lite@0.6.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw=="], - "d3-sankey/d3-array": ["d3-array@2.12.1", "", { "dependencies": { "internmap": "^1.0.0" } }, "sha512-B0ErZK/66mHtEsR1TkPEEkwdy+WDesimkM5gpZr5Dsg54BiTA5RXtYW5qTLIAcekaS9xfZrzBLF/OAkB3Qn1YQ=="], "d3-sankey/d3-shape": ["d3-shape@1.3.7", "", { "dependencies": { "d3-path": "1" } }, "sha512-EUkvKjqPFUAZyOlhY5gzCxCeI0Aep04LwIRpsZ/mLFelJiUfnK56jo5JMDSE7yyP2kLSb6LtF+S5chMk7uqPqw=="], + "dom-serializer/entities": ["entities@4.5.0", "", {}, "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw=="], + "dot-prop/is-obj": ["is-obj@2.0.0", "", {}, "sha512-drqDG3cbczxxEJRoOXcOjtdp1J/lyp1mNn0xaznRs8+muBhgQcrnbspox5X5fOw0HnMnbfDzvnEMEtqDEJEo8w=="], "drizzle-kit/esbuild": ["esbuild@0.25.12", "", { "optionalDependencies": { "@esbuild/aix-ppc64": "0.25.12", "@esbuild/android-arm": "0.25.12", "@esbuild/android-arm64": "0.25.12", "@esbuild/android-x64": "0.25.12", "@esbuild/darwin-arm64": "0.25.12", "@esbuild/darwin-x64": "0.25.12", "@esbuild/freebsd-arm64": "0.25.12", "@esbuild/freebsd-x64": "0.25.12", "@esbuild/linux-arm": "0.25.12", "@esbuild/linux-arm64": "0.25.12", "@esbuild/linux-ia32": "0.25.12", "@esbuild/linux-loong64": "0.25.12", "@esbuild/linux-mips64el": "0.25.12", "@esbuild/linux-ppc64": "0.25.12", "@esbuild/linux-riscv64": "0.25.12", "@esbuild/linux-s390x": "0.25.12", "@esbuild/linux-x64": "0.25.12", "@esbuild/netbsd-arm64": "0.25.12", "@esbuild/netbsd-x64": "0.25.12", "@esbuild/openbsd-arm64": "0.25.12", "@esbuild/openbsd-x64": "0.25.12", "@esbuild/openharmony-arm64": "0.25.12", "@esbuild/sunos-x64": "0.25.12", "@esbuild/win32-arm64": "0.25.12", "@esbuild/win32-ia32": "0.25.12", "@esbuild/win32-x64": "0.25.12" }, "bin": { "esbuild": "bin/esbuild" } }, "sha512-bbPBYYrtZbkt6Os6FiTLCTFxvq4tt3JKall1vRwshA3fdVztsLAatFaZobhkBC8/BrPetoa0oksYoKXoG4ryJg=="], @@ -2437,18 +2496,26 @@ "hastscript/property-information": ["property-information@7.2.0", "", {}, "sha512-IAtzIB6sUiWaJYrX9smp3V46pBGbBeLFRGdh25kg1334VcBlD8HzhPeNIWQH9zhGmo2itIe25EHt9dQP7G5hmg=="], + "imapflow/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + + "imapflow/pino": ["pino@10.3.1", "", { "dependencies": { "@pinojs/redact": "^0.4.0", "atomic-sleep": "^1.0.0", "on-exit-leak-free": "^2.1.0", "pino-abstract-transport": "^3.0.0", "pino-std-serializers": "^7.0.0", "process-warning": "^5.0.0", "quick-format-unescaped": "^4.0.3", "real-require": "^0.2.0", "safe-stable-stringify": "^2.3.1", "sonic-boom": "^4.0.1", "thread-stream": "^4.0.0" }, "bin": { "pino": "bin.js" } }, "sha512-r34yH/GlQpKZbU1BvFFqOjhISRo1MNx1tWYsYvmj6KIRHSPMT2+yHOEb1SG6NMvRoHRF0a07kCOox/9yakl1vg=="], + "is-inside-container/is-docker": ["is-docker@3.0.0", "", { "bin": { "is-docker": "cli.js" } }, "sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ=="], "jsonwebtoken/semver": ["semver@7.8.5", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA=="], "katex/commander": ["commander@8.3.0", "", {}, "sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww=="], + "libmime/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "log-symbols/chalk": ["chalk@5.6.2", "", {}, "sha512-7NzBL0rN6fMUW+f7A6Io4h40qQlG+xGmtMxfbnH/K7TAtt8JQWVQK+6g0UXKMeVJoyV5EkkNsErQ8pVD3bLHbA=="], "log-symbols/is-unicode-supported": ["is-unicode-supported@1.3.0", "", {}, "sha512-43r2mRvz+8JRIKnWJ+3j8JtjRKZ6GmjzfaE/qiBJnikNnYv/6bagRJ1kUhNk8R5EX/GkobD+r+sfxCPJsiKBLQ=="], "lru-memoizer/lru-cache": ["lru-cache@11.5.2", "", {}, "sha512-4pfM1Ff0x50o0tQwb5ucw/RzNyD0/YJME6IVcStalZuMWxdt3sR3huStTtxz4PUmvZfRguvDejasvQ2kifR11g=="], + "mailparser/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "mdast-util-definitions/@types/mdast": ["@types/mdast@3.0.15", "", { "dependencies": { "@types/unist": "^2" } }, "sha512-LnwD+mUEfxWMa1QpDraczIn6k0Ee3SMicuYSSzS6ZYl2gKS09EClnJYGd8Du6rfc5r/GZEk5o1mRb8TaTj03sQ=="], "mdast-util-definitions/unist-util-visit": ["unist-util-visit@4.1.2", "", { "dependencies": { "@types/unist": "^2.0.0", "unist-util-is": "^5.0.0", "unist-util-visit-parents": "^5.1.1" } }, "sha512-MSd8OUGISqHdVvfY9TPhyK2VdUrPgxkUtWSuMHF6XAAFuL4LokseigBnZtPnJMu+FbynTkFNnFlyjxpVKujMRg=="], @@ -2493,6 +2560,8 @@ "prop-types/react-is": ["react-is@16.13.1", "", {}, "sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ=="], + "raw-body/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "react-markdown/remark-parse": ["remark-parse@10.0.2", "", { "dependencies": { "@types/mdast": "^3.0.0", "mdast-util-from-markdown": "^1.0.0", "unified": "^10.0.0" } }, "sha512-3ydxgHa/ZQzG8LvC7jTXccARYDcRld3VfcgIIFs7bI6vbRSxJJmzgLEIIoYKyrfhaY+ujuWaf/PJiMZXoiCXgw=="], "react-markdown/remark-rehype": ["remark-rehype@10.1.0", "", { "dependencies": { "@types/hast": "^2.0.0", "@types/mdast": "^3.0.0", "mdast-util-to-hast": "^12.1.0", "unified": "^10.0.0" } }, "sha512-EFmR5zppdBp0WQeDVZ/b66CWJipB2q2VLNFMabzDSGR66Z2fQii83G5gTBbgGEnEEA0QRussvrFHxk1HWGJskw=="], @@ -2787,6 +2856,10 @@ "hastscript/@types/hast/@types/unist": ["@types/unist@3.0.3", "", {}, "sha512-ko/gIFJRv177XgZsZcBwnqJN5x/Gien8qNOn0D5bQU/zAzVf9Zt3BlcUiLqhV9y4ARk0GbT3tnUiPNgnTXzc/Q=="], + "imapflow/pino/pino-abstract-transport": ["pino-abstract-transport@3.0.0", "", { "dependencies": { "split2": "^4.0.0" } }, "sha512-wlfUczU+n7Hy/Ha5j9a/gZNy7We5+cXp8YL+X+PG8S0KXxw7n/JXA3c46Y0zQznIJ83URJiwy7Lh56WLokNuxg=="], + + "imapflow/pino/thread-stream": ["thread-stream@4.2.0", "", { "dependencies": { "real-require": "^1.0.0" } }, "sha512-e2zZ96wSChazBsbENf/Pcm/4swHt2cEKQ92rhUjkL9GCKiTDJIaTBenjE/m9DXi0QBmTMDkFDdOomUy20A1tDQ=="], + "mdast-util-definitions/unist-util-visit/unist-util-is": ["unist-util-is@5.2.1", "", { "dependencies": { "@types/unist": "^2.0.0" } }, "sha512-u9njyyfEh43npf1M+yGKDGVPbY/JWEemg5nH05ncKPfi+kBbKBJoTdsogMu33uhytuLlv9y0O7GH7fEdwLdLQw=="], "mdast-util-definitions/unist-util-visit/unist-util-visit-parents": ["unist-util-visit-parents@5.1.3", "", { "dependencies": { "@types/unist": "^2.0.0", "unist-util-is": "^5.0.0" } }, "sha512-x6+y8g7wWMyQhL1iZfhIPhDAs7Xwbn9nRosDXl7qoPTSCy0yNxnKc+hWokFifWQIDGi154rdUqKvbCa4+1kLhg=="], @@ -2839,6 +2912,8 @@ "@modelcontextprotocol/sdk/express/body-parser/content-type": ["content-type@2.1.0", "", {}, "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag=="], + "@modelcontextprotocol/sdk/express/body-parser/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "@modelcontextprotocol/sdk/express/type-is/content-type": ["content-type@2.1.0", "", {}, "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag=="], "@modelcontextprotocol/sdk/express/type-is/media-typer": ["media-typer@1.1.1", "", {}, "sha512-yz3xRaG20c6/BOzvYoDaGtPmGscs7YivItZEEqe6GbwNfHuxu9YNmvnEkMzKldAGY4/80pRcQRZSEnhquk9XuQ=="], @@ -2847,10 +2922,14 @@ "@slack/bolt/express/body-parser/content-type": ["content-type@2.1.0", "", {}, "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag=="], + "@slack/bolt/express/body-parser/iconv-lite": ["iconv-lite@0.7.3", "", { "dependencies": { "safer-buffer": ">= 2.1.2 < 3.0.0" } }, "sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ=="], + "@slack/bolt/express/type-is/content-type": ["content-type@2.1.0", "", {}, "sha512-mj7UPXE0jaqaOsukNZRUEfEi2AcL7C/vwmwcHV0O97eO1E1pxBZuyjlZrx5seTaNBg1U6+o35wpa35Qfcc+7ag=="], "@slack/bolt/express/type-is/media-typer": ["media-typer@1.1.1", "", {}, "sha512-yz3xRaG20c6/BOzvYoDaGtPmGscs7YivItZEEqe6GbwNfHuxu9YNmvnEkMzKldAGY4/80pRcQRZSEnhquk9XuQ=="], + "imapflow/pino/thread-stream/real-require": ["real-require@1.0.0", "", {}, "sha512-P4nbQYQfePJxRSmY+v/KINxVucm4NF3p3s7pJveMTtom52FR4YGltUQLB8idDXwDDWW+eYrWDFbuzUnjoWHF7g=="], + "react-markdown/remark-parse/mdast-util-from-markdown/mdast-util-to-string": ["mdast-util-to-string@3.2.0", "", { "dependencies": { "@types/mdast": "^3.0.0" } }, "sha512-V4Zn/ncyN1QNSqSBxTrMOLpjr+IKdHl2v3KVLoWmDPscP4r9GcCi71gjgvUV1SFSKh92AjAG4peFuBl2/YgCJg=="], "react-markdown/remark-parse/mdast-util-from-markdown/micromark": ["micromark@3.2.0", "", { "dependencies": { "@types/debug": "^4.0.0", "debug": "^4.0.0", "decode-named-character-reference": "^1.0.0", "micromark-core-commonmark": "^1.0.1", "micromark-factory-space": "^1.0.0", "micromark-util-character": "^1.0.0", "micromark-util-chunked": "^1.0.0", "micromark-util-combine-extensions": "^1.0.0", "micromark-util-decode-numeric-character-reference": "^1.0.0", "micromark-util-encode": "^1.0.0", "micromark-util-normalize-identifier": "^1.0.0", "micromark-util-resolve-all": "^1.0.0", "micromark-util-sanitize-uri": "^1.0.0", "micromark-util-subtokenize": "^1.0.0", "micromark-util-symbol": "^1.0.0", "micromark-util-types": "^1.0.1", "uvu": "^0.5.0" } }, "sha512-uD66tJj54JLYq0De10AhWycZWGQNUvDI55xPgk2sQM5kn1JYlhbCMTtEeT27+vAhW2FBQxLlOmS3pmA7/2z4aA=="], diff --git a/docs/README.md b/docs/README.md index 6b7c6668..80d6ca5d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -7,6 +7,7 @@ Start with the root [README](../README.md), then use these references: - [Development](development.md): local setup, migrations, ports, and quality checks. - [Coworkers](coworkers.md): durable Bot profiles, channels, visibility, deletion, and external AG-UI registration. - [Routines](routines.md): standing instructions a Bot runs on a schedule, the worker that fires them, and who they run as. +- [Mailbox](mailbox.md): the deployment's own mailbox, the four tools a Bot reads and answers it with, and where the password lives. - Plugins, one connector per page — what an administrator registers, what each person consents to, and what the failures mean: - [Google Drive](plugins/google-drive.md) - [Notion](plugins/notion.md) diff --git a/docs/configuration.md b/docs/configuration.md index 738d1a1d..6fe138f0 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -57,6 +57,13 @@ at `agent-langgraph` on a laptop. | `AUDIT_RETENTION_DAYS` | unset | Whole number of days to keep audit rows; older ones are removed. Unset keeps the trail forever. | | `WORKER_SHARED_SECRET` | unset; `start.sh` uses a fixed local default | The secret the routines worker presents to fire a due routine. Without it the server refuses every handoff, whether or not a worker exists to send one. | | `OPENBOT_GENERATIVE_UI` | unset (capability off) | `true` or `1` lets a Bot answer with an interface it wrote itself. | +| `MAILBOX_IMAP_HOST` | unset | IMAP server for the deployment's mailbox. Needed together with the SMTP host and the user; see [mailbox.md](mailbox.md). | +| `MAILBOX_SMTP_HOST` | unset | SMTP server the mailbox sends through. | +| `MAILBOX_USERS` | unset | Comma-separated addresses on those hosts, one per mailbox account. The first is the default account. Passwords are not environment variables: each is a vault credential keyed by the address. | +| `MAILBOX_USER` | unset | The singular this feature shipped with, still read as a list of one. Setting it together with `MAILBOX_USERS` refuses to start. | +| `MAILBOX_IMAP_PORT` | `993` | Implicit TLS. Set only for a server that listens elsewhere. | +| `MAILBOX_SMTP_PORT` | `465` | Implicit TLS. Set only for a server that listens elsewhere. | +| `MAILBOX_ALLOWED_RECIPIENT_DOMAINS` | unset (anywhere) | Comma-separated domains a Bot may send mail to. A policy rule cannot do this, because rules see a tool's name and effect and not its arguments; see [mailbox.md](mailbox.md). | **`OPENBOT_GENERATIVE_UI`** turns on generated interfaces. Set it, and a Bot may answer by writing the markup, styles and script for an interface and streaming it into the transcript, where it renders diff --git a/docs/mailbox.md b/docs/mailbox.md new file mode 100644 index 00000000..fe04efe8 --- /dev/null +++ b/docs/mailbox.md @@ -0,0 +1,240 @@ +# Mailbox + +The Mailbox connector gives a Bot the deployment's mail: it can list what has arrived, open a message, search +for one, and send mail, including a threaded reply. "Check the support mailbox and tell me what came +in overnight" is a mailbox question, and so is "reply to Dana and say we will have it by Friday." + +The mailbox belongs to the deployment. It is not the mailbox of whoever is asking, and no person +connects an account to it. That is the difference from Google Drive or Notion, where each person +consents for themselves and sees only their own; here everybody granted the tools reads the same +mail. Decide that before granting it, because it is the whole of the access model. + +A deployment may configure several accounts on one pair of hosts, which is what a shared host gives +you: `support@`, `sales@` and `billing@` are three mailboxes on one IMAP and one SMTP server, with a +password each. Every tool takes an optional `account` to say which one to work in; leaving it out +works in the first configured, the default. + +**Grants are per tool, not per account.** A Bot granted `list_messages` can list every configured +account, and one granted `send_message` can send from any of them. There is no per-account grant and +no way to give a Bot one mailbox out of three. If an account must stay out of a Bot's reach, do not +configure it on this deployment. + +## The prerequisite + +Two things, and both are an administrator's: + +1. **Configuration**, which says where the mailbox is and which accounts are on it (below). +2. **Grants**, which say which Bots may use it. Mailbox is a catalogue entry like any other, at + `/admin/plugins/mailbox`, and enabling the entry hands no Bot anything. Each of the four tools is + granted per Bot, so "may read the mail" and "may answer it" are two separate decisions. + +`send_message` is the connector's only write tool, and it is a write in the strongest sense this +product has: it reaches people who never agreed to talk to a Bot, and there is nothing to recall once +it has run. Grant it deliberately, and consider a policy rule that requires approval for it the same +way one would for any other irreversible action. + +## Configuration + +| Variable | Default | What it is | +| -------------------- | ------- | ----------------------------------------------------------------- | +| `MAILBOX_IMAP_HOST` | unset | The IMAP server messages are read from. | +| `MAILBOX_SMTP_HOST` | unset | The SMTP server mail is sent through. | +| `MAILBOX_USERS` | unset | Comma-separated addresses on those hosts. The first is the default account. | +| `MAILBOX_USER` | unset | The singular this shipped with, read as a list of one. Both set refuses to start. | +| `MAILBOX_IMAP_PORT` | `993` | Implicit TLS. Set only for a server that listens elsewhere. | +| `MAILBOX_SMTP_PORT` | `465` | Implicit TLS. Set only for a server that listens elsewhere. | +| `MAILBOX_ALLOWED_RECIPIENT_DOMAINS` | unset (anywhere) | Comma-separated domains a Bot may send to. See [Bounding where mail can go](#bounding-where-mail-can-go). | + +The three hosts-and-users variables are needed together. Set one or two and the server refuses to +start, naming the ones that are missing, rather than booting with half a mailbox that fails at the +first login, at run time, in front of somebody, with nothing but an authentication error from a +server that will not say which half was wrong. Leave all three unset and the connector is still +listed, still grantable, and every tool call answers with the sentence naming what to set. + +`MAILBOX_USERS` is a list, so `support@example.com,sales@example.com` is two accounts and +`support@example.com` is one. The order matters: the first is the default. Addresses are lower-cased +and deduplicated, and an entry that is not an address refuses to start, naming it, rather than +becoming an account a model can select and nothing can unlock. `MAILBOX_USER` is still read as a +list of one for a deployment that already had it; setting both refuses to start, because they are +two answers to the same question. + +Both ports default to the implicit-TLS ones rather than the STARTTLS ones, so the connection is +encrypted before the password is sent rather than negotiating for it in the clear. + +### The passwords are not environment variables + +They are the only secrets in this feature, so they live where this deployment's other secrets live: +the encrypted credential vault. Store **one credential per account** at `/admin/credentials` as: + +- **kind** `mcp`, the vault's name for "the one token this deployment holds for this server", which + is exactly what a mailbox password is: one secret, the deployment's, used for every Bot granted the + tools, never anybody's own grant. +- **provider** `mailbox` +- **key id** the address exactly as it is configured, lower-cased: `support@example.com`. + +So a deployment with three accounts holds three rows, each rotated and revoked on its own. An +account with no stored password is refused by name, naming that account and the key id to store it +under, while the accounts that do have one keep working. + +Rotate in the same place. A password is read from the vault at the moment a call needs it and thrown +away after, so a rotation takes effect on the next tool call rather than on the next restart, and +revoking it stops that account within a call. + +Nothing prints one. IMAP logging is off at the client, which matters because one of the commands it +would log is the authentication one; and a failure sentence from a mail server that quoted the login +back is scrubbed before it reaches an audit row, a transcript or a model. The scrub covers the +base64 forms as well as the plaintext, because neither client sends the password as typed: IMAP +authenticates with `AUTH=PLAIN` or `AUTH=LOGIN`, both base64 on the wire, so a quoted command +carries an encoding of it rather than the password itself. + +## The four tools + +Every one of them takes two arguments about where to look, and they are different things: + +- **`account`** is an email address, one of the configured ones, and it says which mailbox to open. + It defaults to the first configured. An account this deployment does not have is refused before + the vault is read and before anything is dialled, and the refusal lists the ones that exist. +- **`folder`** is an IMAP folder inside that account, such as `INBOX`, `Sent` or `Archive`, and it + defaults to `INBOX`. + +**A configured address in `folder` is adopted rather than refused.** If the value is one of this +deployment's own accounts, and `account` is either unset or the same address, it is taken as the +account, the folder falls back to `INBOX`, and the answer opens with one line saying so: "[folder +took the address support@example.com; it was used as account, reading INBOX.]" There is exactly one +mailbox that value can mean and the model named it, so refusing would spend a whole turn teaching +vocabulary before any work happens, and live runs show the mistake on the first mailbox call of a +turn. The note teaches the same lesson on the way past. + +Two neighbouring cases are still refused before anything is dialled, because neither is unambiguous: + +- **An address in `folder` that is not a configured account.** There is nothing to adopt, so the + refusal says to pass it as `account` instead. +- **`folder` holding one configured address while `account` names a different one.** Two arguments + naming two mailboxes is a model that has lost track of which it is reading. The refusal names + both. + +**The part before the @ of a configured account is refused too**, and deliberately not adopted: +`support` is not an address, and a folder genuinely called `support` can exist. + +None of this is hypothetical. A smaller model given a `mailbox` argument and an `account` argument +put the address in the first one, was answered "Character not allowed in mailbox name" by the IMAP +server, and never tried the second; refused that, it retried with `support` and then `webmaster`, +which are the local parts of two configured accounts, and was answered "Mailbox doesn't exist: +support". The argument is named `folder`, says in its own description that it is neither an address +nor half of one, and is listed after `account` so that is the argument a model meets first. + +A folder that genuinely is not there is answered with the folders that are: "No folder named +Newsletters in support@example.com. Folders here: INBOX, Sent, Archive. The account is chosen by +`account`, not by folder; leave folder unset for INBOX." The listing costs one `LIST` on the +connection that was already open, inside the same deadline, and it is what turns the vendor's +"Mailbox doesn't exist" into something a model can act on rather than retry against. + +Every answer names both, so a turn that reads two accounts cannot merge them and a model reading the +result learns the vocabulary: "showing 10 of 236 messages in folder INBOX of support@example.com, +newest first." + +- **`list_messages`**: the newest messages in a mailbox, newest first: uid, date, sender, subject + and whether it has been read. No bodies. `limit` defaults to 10 and is capped at 50. +- **`read_message`**: one message by `uid`, with its headers and its text. +- **`search_messages`**: messages whose subject, sender or text match `query`, newest first. + `limit` defaults to 20. The match is the mail server's own IMAP `SEARCH`: a plain substring, with + no ranking and no boolean syntax. +- **`send_message`**: sends `to`, `subject` and `body` from the deployment's mailbox, and files a + copy in the account's Sent folder. Give `in_reply_to` as the uid of a message and the reply + threads: the original is fetched, its `Message-ID` becomes `In-Reply-To`, its own `References` + chain plus that id becomes `References`, and `Re: ` goes in front of the subject if it is not + already there. Without it, the message opens a new thread however the subject is worded. + +The sender is always the selected account, and the confirmation says which. There is no `from` +field, so a Bot cannot send as somebody else. + +### The copy in Sent + +SMTP delivers a message; it does not file one. So a send that did nothing else would leave the +account's Sent folder empty, webmail showing nothing sent, and a Bot unable to find its own outgoing +mail. That is not hypothetical: a person checking the mailbox concluded three messages had never +been sent, and all three had been delivered. + +So the message is built once as raw bytes, those bytes are what SMTP delivers, and the same bytes +are appended to the account's Sent folder, marked `\Seen`. Delivered and stored are then byte for +byte the same message, down to the `Message-ID`, which is what makes a send verifiable: the +confirmation names the folder ("A copy is in folder Sent.") and the message id, so a Bot can list +that folder and find what it just sent. + +The folder is resolved by its IMAP special-use flag first, which is the answer that survives a +localised server, and by the names `Sent`, `Sent Items` and `Sent Messages` on a server that marks +nothing. An account where neither finds one files no copy rather than guessing, since appending to +the wrong folder would put outgoing mail where a person reads it as incoming. + +**A copy that could not be filed is never reported as a send that failed.** Filing is a separate +operation against a separate server and happens after delivery, so it can fail on a message that has +already gone. The confirmation then says the mail was sent, names the reason no copy exists, and +says not to send it again. A model told only that something failed would send the message a second +time, and mail cannot be recalled. + +**Uids are per folder, and a folder belongs to one account.** A uid from a listing of `Archive` +names a different message in `INBOX`, and a uid from `support@`'s INBOX names a different message in +`sales@`'s, so every tool that takes one also takes the `folder` and the `account` it came from. A +uid that is not there is refused by name rather than guessed at, and for `send_message` nothing is +sent. + +**Every result is bounded.** At most 512 KB of a message is read off the wire, so one mail with a +large attachment cannot become a gigabyte in this process; a message body is then cut at 8,000 +characters and says so, with the full length, so a model can tell a message that has said what it +came to say from one that has not; and the whole result is capped again at the same 20,000 +characters every other connector's is. A listing or a search that had more behind it says +"showing 10 of 4321" rather than presenting a page as the whole mailbox. Truncation is always +visible, never silent. + +**Every call has a deadline.** Sixty seconds of wall clock per network operation, on top of the +thirty-second inactivity timeouts, and the socket is closed when it expires. The inactivity timeouts +alone would let a server that drips one byte at a time hold a turn open forever. + +**Nothing is changed by reading.** Opening a message does not mark it read, move it or delete it, and +there is no tool that does. A connection is opened, used and closed per call; no session is kept. + +## Bounding where mail can go + +`MAILBOX_ALLOWED_RECIPIENT_DOMAINS` is a comma-separated list of domains (`example.com,partner.example`; +a leading `@` is accepted and dropped). Set it and `send_message` refuses any recipient outside the +list before it opens a connection, naming the domain that was refused and saying nothing was sent. +Unset or empty means anywhere, so a deployment that has not set it behaves exactly as before. + +**Why this exists rather than a policy rule.** The policy engine sees a tool call's name and its +effect. It does not see the arguments, so no rule can say "may email the company and nobody else": +the only thing a rule can express about `send_message` is whether it happens at all. That leaves a +deployment two controls, and they do different jobs: + +1. **An approval rule on `send_message`** in your boundaries. Wholesale, per send, with a person in + the loop. This is the control that decides whether mail goes out. +2. **This allowlist.** Bounds where anything can go, with nobody in the loop. + +**The shape worth protecting against.** The read tools pull text that somebody else wrote into a +model's context, and that text can contain instructions. A Bot holding the read tools and an +unconstrained `send_message` can therefore be talked into mailing the mailbox out, by an email +addressed to it, with no person involved at any point. That is why `send_message` should be +approval-gated wherever the mailbox holds anything worth keeping, and why the allowlist is worth +setting even when it is gated: it is the half that still holds if a rule is edited or a mode is +switched to dry-run. + +## Governance + +Nothing about a Mailbox tool call is special. It goes through `plugins/store.ts` like every other +connector: the Bot's grant is checked, the policy is evaluated with the tool's effect (`send_message` +as a write, the other three as reads), an audit row is written, and only then is any mail server +dialled. There is no second path to the mailbox and no bypass. + +The grant is per tool. It is not per account, and the policy engine cannot make it one, for the same +reason it cannot bound recipients: a rule sees a tool call's name and effect, never its arguments. +A Bot granted the mailbox tools reaches every configured account. + +The audit trail records every call, and for a failure it keeps the mail server's own sentence, which +is usually the most useful thing available: "Invalid credentials", "Mailbox does not exist" and +"Relay access denied" each name a different fix. + +## See also + +- [Configuration](configuration.md): every environment variable, in one table. +- [Architecture](architecture.md): where a plugin call is decided, recorded and made. +- [Routines](routines.md): the other first-party connector that runs in-process, and the one to read + next if you want a Bot to check the mailbox on a schedule. diff --git a/server/package.json b/server/package.json index 4b6fa4c1..ca551e1b 100644 --- a/server/package.json +++ b/server/package.json @@ -22,6 +22,9 @@ "cron-parser": "^5", "drizzle-orm": "^0.45.2", "hono": "^4.10.0", + "imapflow": "^1.7.8", + "mailparser": "^3.9.20", + "nodemailer": "^9.1.1", "postgres": "^3.4.9", "rxjs": "7.8.1", "yaml": "^2.9.0", @@ -29,6 +32,8 @@ }, "devDependencies": { "@copilotkit/aimock": "1.39.0", + "@types/mailparser": "^3.4.6", + "@types/nodemailer": "^7.0.4", "drizzle-kit": "^0.31.10", "eventsource": "3.0.7" } diff --git a/server/src/config.ts b/server/src/config.ts index 1c235f9c..2e249192 100644 --- a/server/src/config.ts +++ b/server/src/config.ts @@ -61,6 +61,53 @@ export type ComputerConfig = | SharedComputerConfig | SandboxComputerConfig; +/** + * The deployment's own mailbox: where it is and which accounts it signs in as. + * + * NO PASSWORD HERE, and that is the point of the shape. Everything in this file comes from the + * environment, which means a `.env` file on a laptop, a compose file in a repository and whatever a + * cluster hands a container as plain text. The mailbox passwords are the only secrets in this + * feature, so they live where the other secrets this deployment holds live: the encrypted + * credential vault, one credential per account, keyed by the address. See + * `plugins/builtin-mailbox.ts` for how one is resolved. + * + * Both ports default to the implicit-TLS ones (993 and 465) rather than to the STARTTLS ones, so a + * deployment that sets only the two hosts gets an encrypted connection rather than one that + * negotiates for it in the clear. + */ +export type MailboxConfig = { + imapHost: string; + imapPort: number; + smtpHost: string; + smtpPort: number; + /** + * The accounts on these hosts, lower-cased and in the order they were configured. + * + * Never empty: a mailbox with no account is refused at start-up rather than carried around as a + * shape every caller has to check. The FIRST is the default, which is the one a tool call that + * named no account works in, so the order in `MAILBOX_USERS` is a decision rather than a detail. + * + * Several accounts, one set of hosts: this is the shared-hosting shape, where `support@`, + * `sales@` and `billing@` are all mailboxes on the same IMAP and SMTP servers with a password + * each. Each password is its own vault credential, keyed by the address. + */ + users: readonly string[]; + /** + * The domains a Bot may send to. Empty means anywhere, which is the default. + * + * WHY A DEPLOYMENT MIGHT WANT THIS. The policy engine sees a tool call's NAME and its effect, not + * its arguments, so no rule can say "may email the company and nobody else": the only thing a + * rule can do about `send_message` is require approval for all of it or none of it. That leaves a + * gap this closes: the read tools bring text somebody else wrote into a model's context, and a + * Bot holding read plus unconstrained send is one persuasive message away from mailing the inbox + * to whoever asked. An allowlist bounds where anything can go, without a person in the loop. + * + * It is a floor and not a substitute for the approval rule. Inside the allowed domains the Bot + * can still send whatever it was talked into, so a deployment that cares should have both. + */ + allowedRecipientDomains: ReadonlySet; +}; + /** * Who a deployment lets in, and through which front door. * @@ -258,6 +305,16 @@ export type DeploymentConfig = { * mounted and failing: a capability that is not configured should be missing, not broken. */ computer?: ComputerConfig; + /** + * The deployment's mailbox. Absent means the Mailbox tools refuse rather than fail. + * + * Unlike {@link DeploymentConfig.computer}, absence does not unmount anything: the catalogue entry + * stays admissible and its tools stay grantable, because a Bot's grants are an administrator's + * decision and should not evaporate because a variable was unset during a deploy. What absence + * changes is what a call answers: a sentence naming the four things to set, rather than a + * connection attempt to nowhere. + */ + mailbox?: MailboxConfig; /** How far one Bot handing work to another may go. */ handoff: HandoffCaps; /** @@ -819,6 +876,152 @@ function generativeUiEnabled(environment: Environment): boolean { return on === "true" || on === "1"; } +/** + * A port from the environment, or the protocol's default. + * + * Refused rather than coerced, the same as every other number in this file. A deployment that typed + * `993 ` with a stray character and silently got 993 anyway is fine; one that typed `9993` and got + * the default would be talking to the right host on the wrong port with nothing saying so. + */ +function mailboxPort( + environment: Environment, + name: string, + fallback: number, +): number { + const raw = optional(environment, name); + if (!raw) return fallback; + + const port = Number(raw); + if (!Number.isInteger(port) || port < 1 || port > 65535) { + throw new Error(`${name} must be a port number between 1 and 65535`); + } + return port; +} + +/** + * The deployment's mailbox, if it has one. + * + * Absent when none of the three are set, which is the ordinary state of a deployment that does not + * want this. Set one or two of them and it refuses to start naming what is missing, rather than + * booting with half a mailbox: a deployment with a host and no user is one where every mail tool + * fails at the first login, at run time, in front of somebody, and the only evidence is an auth + * failure from a server that will not say which half was wrong. + * + * The passwords are deliberately not read here. See {@link MailboxConfig}. + */ +function mailboxConfig(environment: Environment): MailboxConfig | undefined { + const imapHost = optional(environment, "MAILBOX_IMAP_HOST"); + const smtpHost = optional(environment, "MAILBOX_SMTP_HOST"); + const users = mailboxUsers(environment); + if (!imapHost && !smtpHost && users.length === 0) return undefined; + + const missing = [ + imapHost ? null : "MAILBOX_IMAP_HOST", + smtpHost ? null : "MAILBOX_SMTP_HOST", + users.length > 0 ? null : "MAILBOX_USERS", + ].filter((name): name is string => name !== null); + if (missing.length > 0) { + throw new Error( + `${missing.join(", ")} must be set as well: a mailbox needs an IMAP host, an SMTP host and at least one user. Unset all three to switch the mailbox off.`, + ); + } + + return { + imapHost: imapHost as string, + // Implicit TLS on both, rather than the STARTTLS ports. See MailboxConfig. + imapPort: mailboxPort(environment, "MAILBOX_IMAP_PORT", 993), + smtpHost: smtpHost as string, + smtpPort: mailboxPort(environment, "MAILBOX_SMTP_PORT", 465), + users, + allowedRecipientDomains: allowedRecipientDomains(environment), + }; +} + +/** + * The accounts on this deployment's mail hosts, in the order they were written. + * + * `MAILBOX_USERS` is the list and its first entry is the default account. `MAILBOX_USER`, the + * singular this feature shipped with, is still read as a list of one, so a deployment that already + * has one does not have to be edited to keep working. Both set is refused rather than merged or + * silently preferred: they are two answers to the same question, and a deployment holding both has + * an intention nobody here can read. + * + * Lower-cased and deduplicated, because an address is case-insensitive and the address is also the + * key of the vault credential holding that account's password. `Support@` and `support@` written as + * two entries are one mailbox, and treating them as two would be a second account nothing can ever + * unlock. + * + * An entry that is not an address refuses to start, naming it. It would otherwise become an account + * a model can select, a credential key an administrator cannot guess, and a login failure at run + * time in front of somebody. + */ +function mailboxUsers(environment: Environment): string[] { + const list = optional(environment, "MAILBOX_USERS"); + const legacy = optional(environment, "MAILBOX_USER"); + if (list && legacy) { + throw new Error( + "MAILBOX_USERS and MAILBOX_USER are both set and they are the same setting. Keep MAILBOX_USERS, which holds the whole list, and unset MAILBOX_USER.", + ); + } + + const name = list ? "MAILBOX_USERS" : "MAILBOX_USER"; + const entries = list + ? commaSeparated(environment, "MAILBOX_USERS") + : legacy + ? [legacy] + : []; + + const users: string[] = []; + for (const entry of entries) { + if (!/^[^\s@,]+@[^\s@,]+$/.test(entry)) { + throw new Error( + `${name} entry "${entry}" must be an email address such as bot@example.com`, + ); + } + const address = entry.toLowerCase(); + if (!users.includes(address)) users.push(address); + } + return users; +} + +/** + * Where mail from this deployment may go, read from the environment. + * + * Unset and empty are the same answer, and it is "anywhere": a deployment that has not said + * otherwise keeps the behaviour it had, and one that empties the variable has switched the + * restriction off rather than switched every send off. The opposite reading would turn a blanked + * line in a `.env` into a mailbox that silently refuses everybody. + * + * Lower-cased, because a domain is case-insensitive and an allowlist that misses `Example.test` is + * an allowlist somebody will debug at the wrong end. A leading `@` is accepted and dropped, since + * that is how a person writes a domain when they are thinking of addresses. + * + * Refused rather than ignored when an entry is not a domain. `MAILBOX_ALLOWED_RECIPIENT_DOMAINS` is + * a safety list, and one written as `sales@example.test` that silently matched nothing would be a + * deployment that believes it is restricted and is not. + */ +function allowedRecipientDomains( + environment: Environment, +): ReadonlySet { + const domains = commaSeparated( + environment, + "MAILBOX_ALLOWED_RECIPIENT_DOMAINS", + ).map((entry) => entry.replace(/^@/, "").toLowerCase()); + + for (const domain of domains) { + if ( + !/^[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)*$/.test( + domain, + ) + ) { + throw new Error( + `MAILBOX_ALLOWED_RECIPIENT_DOMAINS entry "${domain}" must be a domain such as example.com, not an address or a URL`, + ); + } + } + return new Set(domains); +} + /** * How long the audit trail is kept. * @@ -861,6 +1064,7 @@ export function loadConfig( const auth = authConfig(environment, google); const managedAgent = managedAgentConfig(environment); const workerSharedSecret = optional(environment, "WORKER_SHARED_SECRET"); + const mailbox = mailboxConfig(environment); return { databaseUrl: required(environment, "DATABASE_URL"), @@ -894,6 +1098,7 @@ export function loadConfig( ? { appDistDir: optional(environment, "APP_DIST_DIR") as string } : {}), computer: computerConfig(environment), + ...(mailbox ? { mailbox } : {}), handoff: handoffCaps(environment), ...(optional(environment, "AGENT_TOOL_TOKEN") ? { agentToolToken: optional(environment, "AGENT_TOOL_TOKEN") as string } diff --git a/server/src/index.ts b/server/src/index.ts index 6828cb9a..18c74c38 100644 --- a/server/src/index.ts +++ b/server/src/index.ts @@ -57,12 +57,14 @@ import { import { createCredentialAdminService, createCredentialStore, + decryptCredentialForUse, resolveModelApiKey, } from "./credentials"; import { createDatabase } from "./db/client"; import { intelligenceChannelMappings } from "./db/schema"; import { createOnboardingStore } from "./people/onboarding"; import { createPeopleStore } from "./people/store"; +import { mailboxCredentialFor, useMailbox } from "./plugins/builtin-mailbox"; import { useRoutineTools } from "./plugins/builtin-routines"; import { redirectUriFor } from "./plugins/oauth"; import { createPluginStore } from "./plugins/store"; @@ -338,6 +340,50 @@ const pluginStore = createPluginStore({ const routineStore = createRoutineStore(database); useRoutineTools(routineStore); +/** + * The mailbox, and where its password comes from. + * + * Installed the same way and for the same reason as the routine store above: the transport is a + * module, so this is the one seam it has. + * + * WHAT IS INSTALLED IS A WAY TO READ A PASSWORD, NOT A PASSWORD. Nothing here decrypts anything at + * boot. The closure runs when a tool call needs the secret, so an administrator who rotates an + * account's credential is obeyed by the next call rather than by the next restart, and one who + * revokes it stops the tools within a call, because `decryptCredentialForUse` refuses a revoked + * row, which `builtin-mailbox` reports as a failure rather than treating as an absent password. + * + * ONE CREDENTIAL PER ACCOUNT, keyed by the address, so a deployment with several mailboxes on a + * shared host holds several rows and each is rotated and revoked on its own. + * + * `mcp` is the kind, which is the vault's name for "the one token this deployment holds for this + * server", and it is exactly what a mailbox password is here: one secret, held by the deployment, + * used for everybody, never anybody's own grant. The provider and key id are the connector's key, so + * an administrator stores it at `/admin/credentials` the same way they store a custom server's + * token. + * + * Nothing is installed when no mailbox is configured. The catalogue entry stays admissible and its + * tools stay grantable either way (see `DeploymentConfig.mailbox`), and a call then answers with + * the sentence naming what to set. + */ +useMailbox( + config.mailbox + ? { + config: config.mailbox, + password: async (account) => { + const held = await credentialStore.findLiveByKey( + mailboxCredentialFor(account), + ); + if (!held) return null; + return decryptCredentialForUse( + config.keyEncryptionKey, + credentialStore, + held.id, + ); + }, + } + : null, +); + /** * Where a Bot handing work to another gets decided. * diff --git a/server/src/mailbox/client.ts b/server/src/mailbox/client.ts new file mode 100644 index 00000000..64ba1f60 --- /dev/null +++ b/server/src/mailbox/client.ts @@ -0,0 +1,866 @@ +import { ImapFlow } from "imapflow"; +import { simpleParser } from "mailparser"; +import { createTransport, type Transporter } from "nodemailer"; +import MailComposer from "nodemailer/lib/mail-composer"; +import type { MailboxConfig } from "../config"; + +/** + * The only place in this deployment that speaks IMAP and SMTP. + * + * One door, for the same reason `plugins/mcp.ts` is one door for MCP: every call out carries the + * mailbox password and brings back text a model will read, so both directions want a single place to + * be careful in. A second client somewhere else would be a second place to forget the timeout, the + * size cap, or the rule that the password never appears in anything anybody reads. + * + * NO SESSION IS KEPT. A connection is opened, used and closed for every listing, every read, every + * search and every send. A pooled IMAP session would be a long-lived authenticated socket that + * several Bots' calls arrive on, which is the shape of bug `mcp.ts` declines to have. IMAP is worse + * for it than HTTP, because a session carries SELECTED-mailbox state, so one call's `mailbox` + * argument would silently decide what another call read. + * + * WHAT THIS MODULE IS NOT. It makes no decision about whether a call is allowed, whose it is, or + * what a model should be told. It speaks the two protocols and hands back plain values. + * `plugins/builtin-mailbox.ts` holds the tools, the argument checking and the sentences; the split + * is the same one Routines has between `routines/store.ts` and `plugins/builtin-routines.ts`. + */ + +/** + * How long a server gets before we give up on it. + * + * One number for connecting, for the greeting and for socket inactivity, because a mailbox that is + * slow in any of those three ways is a turn that is hanging, and the person is waiting either way. + * Well under the 60s an MCP call is given: a mail server that has said nothing for half a minute is + * not about to. + */ +const TIMEOUT_MS = 30_000; + +/** + * The wall clock one network operation gets, whatever it is doing. + * + * The three timeouts above are all INACTIVITY timeouts, which is a different promise: a server that + * sends one byte every twenty seconds resets all three forever, and the call it is holding open is a + * turn nobody can end. This one does not care whether bytes are arriving. When it expires the socket + * is closed rather than left to the garbage collector, because an abandoned IMAP connection is an + * authenticated socket that the server keeps until its own idle timer notices. + * + * Per operation rather than per tool call, and the difference is visible in exactly one place: + * `send_message` with `in_reply_to` fetches over IMAP and then sends over SMTP, so its worst case is + * two deadlines rather than one. Both end, which is the property that matters. + */ +const CALL_DEADLINE_MS = 60_000; + +/** + * The most of one message this deployment will pull off the wire. + * + * imapflow will happily accept a literal up to a gigabyte and `simpleParser` has no cap of its own, + * so without this a single mail with a large attachment is a gigabyte in this process's heap to + * produce at most {@link MAX_BODY_CHARS} of text. 512 KB is far more prose than the body cap can + * ever show, so the only thing it truncates is content that was never going to be read out. + * + * A truncated source is still parsed, because the headers and the first text part are at the front + * of a message and are what the tools want. That it was cut is carried out with the message and + * said in the answer rather than left to look like a message that ended there. + */ +export const MAX_SOURCE_BYTES = 512 * 1024; + +/** One message as a listing shows it. No body: a listing of ten bodies is not a listing. */ +export type MessageHeader = { + uid: number; + /** The From: header rendered as `Name
`, or just the address. */ + from: string; + to: string; + subject: string; + /** The message date as an ISO string, or null when the server sent none. */ + date: string | null; + seen: boolean; +}; + +/** One message, opened. */ +export type FullMessage = MessageHeader & { + /** + * The Message-ID header, which is the only thing that makes a reply thread. + * + * Null for a message that arrived without one. A reply to such a message is still a reply; it just + * cannot be threaded, and {@link OutgoingMessage.reply} is left off rather than filled with a + * guess that would thread it into the wrong conversation. + */ + messageId: string | null; + /** The References header, oldest first, as the reply has to repeat it. */ + references: readonly string[]; + /** The text of the message, already bounded by the caller's cap. */ + body: string; + /** How long the body was before the cap, so the caller can say it was cut. */ + bodyLength: number; + /** + * True when only the first {@link MAX_SOURCE_BYTES} of the message were read off the wire. + * + * Separate from a body that was merely long: this one says the deployment never saw the rest, so + * an answer that says "that is the whole message" would be wrong rather than abbreviated. + */ + sourceTruncated: boolean; + /** What the server says the whole message weighs, in bytes, when it says. */ + sizeBytes: number | null; +}; + +/** A message on its way out. */ +export type OutgoingMessage = { + to: string; + subject: string; + body: string; + /** + * The message this one replies to, when it is a reply. + * + * Both headers together, never one: `In-Reply-To` alone threads in some clients and not others, + * and `References` alone loses which message was actually answered. They are computed from a + * message that was fetched, so a reply is threaded against something that exists rather than + * against an id a model produced. + */ + reply?: { messageId: string; references: readonly string[] }; +}; + +/** + * One open IMAP connection, as the four tools need it. + * + * Deliberately narrow. Nothing a model calls has any business deleting a message, moving one, or + * setting a flag, so none of that is here. Same reasoning that keeps `RoutineTools` down to four of + * `RoutineStore`'s methods. + */ +export type MailboxSession = { + /** The newest `limit` messages in `mailbox`, newest first. */ + recent(mailbox: string, limit: number): Promise; + /** One message by uid, or null when that mailbox holds no such uid. */ + message(mailbox: string, uid: number): Promise; + /** Messages whose subject, sender or text matches, newest first, at most `limit`. */ + search(mailbox: string, query: string, limit: number): Promise; +}; + +/** + * Some messages, and how many there were to choose from. + * + * `total` is the whole point of the shape. A page of ten headers with nothing else said reads to a + * model as the whole mailbox, and it will answer "you have ten messages" about a mailbox holding + * four thousand. The count is free at both call sites (the mailbox's own `exists`, and the length of + * what SEARCH returned) and it is the difference between a listing and a claim. + */ +export type MessagePage = { + headers: MessageHeader[]; + /** How many messages the mailbox or the search had, before `limit` was applied. */ + total: number; +}; + +/** + * The two protocols, as something the tools can be tested against. + * + * A seam rather than a direct import, and it is the same seam `PluginStoreOptions.callVendor` is: + * the properties worth asserting about this module are which arguments a call went out with and + * what a model is told, and asserting either otherwise would need a reachable mail server, which + * means the properties most worth testing would be the ones never tested. + */ +export type MailboxClients = { + /** Open a connection, run the work, close it. Closed whatever happened. */ + withSession(use: (session: MailboxSession) => Promise): Promise; + send(message: OutgoingMessage): Promise; +}; + +/** + * What happened to one outgoing message: it went, and whether a copy was filed. + * + * TWO OUTCOMES RATHER THAN ONE, and keeping them apart is the whole point of the shape. SMTP + * delivery and the IMAP copy are separate operations against separate servers, and the second can + * fail after the first has succeeded. Reported as one failure, that is a Bot telling somebody the + * mail did not go and sending it again, which is the expensive mistake: mail cannot be recalled and + * the recipient gets it twice. + */ +export type SendReceipt = { + messageId: string | null; + /** The folder the copy was appended to, or null when no copy was filed. */ + filedTo: string | null; + /** Why no copy was filed, when none was. Null when one was. */ + fileError: string | null; +}; + +/** + * The message as bytes, built once. + * + * WHY RAW RATHER THAN LETTING NODEMAILER COMPOSE AT SEND TIME. What SMTP delivers and what IMAP + * stores have to be the same message, and composing twice is two messages: different `Message-ID`, + * different `Date`, different boundaries. A person looking at Sent in webmail would be reading a + * near-copy of what the recipient got, and a Bot trying to verify its own send by `Message-ID` + * would find nothing. + */ +export type ComposedMessage = { + raw: Buffer; + /** The SMTP envelope, so the delivery uses the same addresses the headers name. */ + envelope: ReturnType< + ReturnType["compile"]>["getEnvelope"] + >; + messageId: string; +}; + +/** + * Build one outgoing message into the bytes that will be both delivered and stored. + * + * Exported because it is worth asserting on its own: it is the only place the `From` is decided, + * and the only place the two threading headers are written. + */ +export async function composeMessage( + account: string, + message: OutgoingMessage, +): Promise { + const composed = new MailComposer({ + // The selected account, never an address from the arguments. A `from` a model could name is a + // Bot sending mail as somebody else. + from: account, + to: message.to, + subject: message.subject, + text: message.body, + ...(message.reply + ? { + inReplyTo: message.reply.messageId, + references: [...message.reply.references], + } + : {}), + }).compile(); + + return { + raw: await composed.build(), + envelope: composed.getEnvelope(), + // `compile` has already ensured one, so this is the id that is in the bytes above rather than a + // second one generated here. + messageId: composed.messageId(), + }; +} + +/** The special-use flag every IMAP server that has a Sent folder marks it with (RFC 6154). */ +const SENT_SPECIAL_USE = "\\Sent"; + +/** What a Sent folder is called on a server that marks nothing. Lower-cased. */ +const SENT_NAMES = new Set(["sent", "sent items", "sent messages"]); + +/** + * Which folder a copy of a sent message belongs in, out of what the server listed. + * + * SPECIAL-USE FIRST, because it is the answer that survives a localised server: a French account's + * Sent folder is `Éléments envoyés` and no list of English names will ever find it. The names are + * the fallback for the servers that mark nothing, and they are the three spellings in the wild. + * + * Null rather than a guess when neither finds one. Appending to the wrong folder is worse than not + * appending: it puts outgoing mail somewhere a person will read it as incoming. + */ +export function sentFolderFrom( + boxes: readonly { path: string; name?: string; specialUse?: string }[], +): string | null { + const marked = boxes.find((box) => box.specialUse === SENT_SPECIAL_USE); + if (marked) return marked.path; + + const named = boxes.find((box) => + SENT_NAMES.has((box.name ?? box.path).toLowerCase()), + ); + return named ? named.path : null; +} + +/** The most of one body that is ever read out of a message. See {@link readBody}. */ +export const MAX_BODY_CHARS = 8_000; + +export class MailboxError extends Error { + constructor(message: string) { + super(message); + this.name = "MailboxError"; + } +} + +/** `Name
`, or the address alone, or an empty string. Never `undefined` in a listing. */ +function addressLine( + addresses: readonly { name?: string; address?: string }[] | undefined, +): string { + if (!addresses || addresses.length === 0) return ""; + return addresses + .map((one) => { + const address = one.address ?? ""; + const name = one.name?.trim(); + return name ? `${name} <${address}>` : address; + }) + .join(", "); +} + +/** + * A header line out of what the server sent. + * + * Every field is optional in IMAP's envelope and several are optional in practice: a message with no + * Subject, no Date, or a From the server could not parse is an ordinary message, not an error. Empty + * strings rather than absent fields, so the rendering above this never has to branch. + */ +function headerOf(message: { + uid: number; + flags?: Set; + envelope?: { + date?: Date; + subject?: string; + from?: { name?: string; address?: string }[]; + to?: { name?: string; address?: string }[]; + }; +}): MessageHeader { + const envelope = message.envelope ?? {}; + return { + uid: message.uid, + from: addressLine(envelope.from), + to: addressLine(envelope.to), + subject: envelope.subject ?? "", + date: envelope.date ? new Date(envelope.date).toISOString() : null, + seen: message.flags?.has("\\Seen") ?? false, + }; +} + +/** + * HTML as something worth putting in front of a model, for a message that carries no text part. + * + * Deliberately crude, and only ever a fallback. `mailparser` already hands back `text` for anything + * multipart/alternative, which is nearly everything; what is left is the marketing mail that ships + * HTML alone, where the choice is between this and telling the model the message was empty. Scripts + * and styles go first, because their contents are not prose and would otherwise arrive as prose. + */ +export function strippedHtml(html: string): string { + return ( + html + .replace(/<(script|style)\b[^>]*>[\s\S]*?<\/\1>/gi, " ") + .replace(//gi, "\n") + .replace(/<\/(p|div|tr|li|h[1-6])>/gi, "\n") + .replace(/<[^>]+>/g, " ") + .replace(/ /gi, " ") + .replace(/&/gi, "&") + .replace(/</gi, "<") + .replace(/>/gi, ">") + .replace(/"/gi, '"') + .replace(/'/gi, "'") + .replace(/[ \t]+/g, " ") + // The space every dropped tag left behind, where it now sits at the start or end of a line. A + // stripper that keeps them indents the whole message by one space and nobody can see why. + .replace(/[ \t]*\n[ \t]*/g, "\n") + .replace(/\n{3,}/g, "\n\n") + .trim() + ); +} + +/** + * The readable text of a parsed message, and how long it was before the cap. + * + * The text part wins whenever there is one. Falling back to the HTML is for the message that has no + * text part at all, rather than for the one whose text part is short: a plain-text alternative that + * says "this message needs an HTML viewer" is still what the sender wrote, and preferring the HTML + * on length would swap the sender's own words out for markup on any short message. + */ +export function readBody(parsed: { text?: string; html?: string | false }): { + body: string; + bodyLength: number; +} { + const text = parsed.text?.trim(); + const source = + text && text.length > 0 + ? text + : typeof parsed.html === "string" + ? strippedHtml(parsed.html) + : ""; + return { + body: + source.length > MAX_BODY_CHARS ? source.slice(0, MAX_BODY_CHARS) : source, + bodyLength: source.length, + }; +} + +/** The References header as a list, oldest first. Whitespace-separated, per RFC 5322. */ +function referencesOf(value: string | string[] | undefined): string[] { + if (!value) return []; + const parts = Array.isArray(value) ? value : value.split(/\s+/); + return parts.map((one) => one.trim()).filter((one) => one.length > 0); +} + +/** + * The sentence a mail server actually wrote, out of the error that carries it. + * + * WHY THIS IS NOT `error.message`. imapflow answers every IMAP `NO` and `BAD` with an Error whose + * message is the fixed string "Command failed" and puts the server's own words on `responseText` + * (`imapflow/lib/imap-flow.js`). So the naive reading turns "Invalid credentials", "Mailbox does not + * exist" and "Over quota" into one useless sentence that names none of them, which is the opposite + * of the reason this deployment keeps the vendor's wording at all: each of those has a different fix + * and only the server can tell them apart. + * + * nodemailer is the other half of the same story, from the other protocol: it puts the SMTP reply on + * `response`. Both are read, and both are read only when they are strings, because imapflow also + * uses `response` for the parsed response OBJECT, and `String(anObject)` is `[object Object]`. + */ +export function mailServerSentence(error: unknown): string { + if (error !== null && typeof error === "object") { + const carried = error as { responseText?: unknown; response?: unknown }; + if ( + typeof carried.responseText === "string" && + carried.responseText.trim() !== "" + ) { + return carried.responseText.trim(); + } + if ( + typeof carried.response === "string" && + carried.response.trim() !== "" + ) { + return carried.response.trim(); + } + } + return error instanceof Error ? error.message : String(error); +} + +/** The most folder names one refusal will list. See {@link noSuchFolderSentence}. */ +export const MAX_FOLDERS_LISTED = 40; + +/** + * Whether the server refused because the folder is not there, rather than for any other reason. + * + * Only ever asked about a failed SELECT, which narrows it a lot: at that point the connection is up + * and authenticated, so the interesting failures are "no such folder" and permission. Every server + * words the first differently ("Mailbox doesn't exist: support", "[NONEXISTENT] Unknown Mailbox", + * "NO Mailbox does not exist"), so the wordings are matched rather than a code that not every + * server sends. + */ +function looksLikeMissingFolder(error: unknown): boolean { + const sentence = mailServerSentence(error).toLowerCase(); + return ( + sentence.includes("doesn't exist") || + sentence.includes("does not exist") || + sentence.includes("nonexistent") || + sentence.includes("unknown mailbox") || + sentence.includes("no such mailbox") || + sentence.includes("no such folder") + ); +} + +/** + * What a model is told when it asked for a folder that is not there. + * + * THE VENDOR'S OWN SENTENCE IS A TRAP HERE, which is why this is the one place the rule about + * keeping it is broken. "Mailbox doesn't exist: support" reads as a folder that happens to be + * missing, so a model retries with another folder name, and the live failure this exists for was + * exactly that: refused an address in `folder`, it tried `support`, then `webmaster`, which are the + * local parts of two configured accounts. The folders that DO exist, and one sentence saying the + * account is not chosen this way, turn a loop into a correction. + * + * Trimmed to fit the 400 characters a failure is capped at by `plugins/builtin-mailbox.ts`, from + * the end of the list rather than the end of the sentence: the closing instruction is the half that + * changes what the model does next, so it is the half that must survive. + */ +export function noSuchFolderSentence( + folder: string, + account: string, + folders: readonly string[], +): string { + const head = `No folder named ${folder} in ${account}.`; + const tail = + "The account is chosen by `account`, not by folder; leave folder unset for INBOX."; + + const candidates = folders.slice(0, MAX_FOLDERS_LISTED); + const room = 380 - head.length - tail.length; + const shown: string[] = []; + let used = " Folders here: .".length; + for (const name of candidates) { + if (used + name.length + 2 > room) break; + shown.push(name); + used += name.length + 2; + } + + const middle = + shown.length === 0 + ? "" + : ` Folders here: ${shown.join(", ")}${shown.length < folders.length ? ", and more" : ""}.`; + return `${head}${middle} ${tail}`; +} + +/** + * Run one network operation against a wall clock, and shut the socket if the clock wins. + * + * The cleanup is the point rather than the rejection. A `Promise.race` that only rejects leaves the + * losing work running, holding an authenticated connection this deployment has stopped waiting for; + * `onExpiry` is what makes the deadline mean the operation is over rather than merely unwatched. The + * timer is cleared in a `finally` so a fast call does not hold the process open for a minute. + */ +export async function withDeadline( + work: () => Promise, + onExpiry: () => void, + what: string, + // A parameter with a default rather than a constant read inside, so the behaviour can be asserted + // in a test that finishes in milliseconds instead of one that takes a minute to prove a minute. + deadlineMs: number = CALL_DEADLINE_MS, +): Promise { + let timer: ReturnType | undefined; + const expiry = new Promise((_resolve, reject) => { + timer = setTimeout(() => { + // Best effort: a socket that is already gone throws here, and that is not a failure of the + // deadline, which has already decided what this call answers. + try { + onExpiry(); + } catch {} + reject( + new MailboxError( + `The mail server did not finish ${what} within ${Math.round(deadlineMs / 1000)} seconds.`, + ), + ); + }, deadlineMs); + }); + + try { + // Both promises are raced, so the loser's rejection is handled here rather than surfacing later + // as an unhandled one. + return await Promise.race([work(), expiry]); + } finally { + if (timer) clearTimeout(timer); + } +} + +/** + * The real thing: imapflow for reading, nodemailer for sending. + * + * Built per call rather than once, because it holds the password. Nothing here caches a connection, + * a transport or the secret itself beyond the call that asked for it. + * + * `account` is one of `config.users`, already chosen and checked by the caller, and it is both what + * the two protocols authenticate as and what the mail is from. It is a parameter rather than a + * field of the configuration because a deployment's hosts are one thing and the account a + * particular call is working in is another: the same hosts serve every account, and which one this + * client speaks for is decided per call. + */ +export type MailboxWire = { + /** How an IMAP client is built. Defaults to imapflow. */ + imap?: (options: ConstructorParameters[0]) => ImapLike; + /** How an SMTP transport is built. Defaults to nodemailer. */ + smtp?: (options: SmtpOptions) => SmtpLike; +}; + +/** As much of imapflow as this module speaks. */ +export type ImapLike = Pick< + ImapFlow, + | "connect" + | "logout" + | "close" + | "getMailboxLock" + | "mailbox" + | "fetch" + | "fetchOne" + | "search" + | "list" + | "append" +>; + +/** As much of nodemailer as this module speaks. */ +export type SmtpLike = Pick; + +/** What this module asks an SMTP transport to be built with. */ +export type SmtpOptions = { + host: string; + port: number; + secure: boolean; + auth: { user: string; pass: string }; + connectionTimeout: number; + greetingTimeout: number; + socketTimeout: number; +}; + +export function createMailboxClients( + config: MailboxConfig, + account: string, + password: string, + /* + * Injected only by tests, and only for the one property that cannot be asserted otherwise: that + * the bytes handed to SMTP and the bytes appended to Sent are the same bytes. Everything else + * about this module is asserted through `MailboxAccess.clients`, one level up. + */ + wire: MailboxWire = {}, +): MailboxClients { + /** + * Build, use and close a client. + * + * The `finally` logs out whatever happened, because a thrown error is the case where a leaked + * connection is most likely and least noticed. `logout` is given the same swallow `mcp.ts` gives + * `close`: a server that will not say goodbye has not failed the work that just succeeded. + */ + async function withClient( + use: (client: ImapLike) => Promise, + what = "reading the mailbox", + ) { + const build = wire.imap ?? ((options) => new ImapFlow(options)); + const client = build({ + host: config.imapHost, + port: config.imapPort, + // Implicit TLS, always. STARTTLS is negotiated in the clear, so a server that answers without + // it gets this deployment's mailbox password over a plain socket. + secure: true, + auth: { user: account, pass: password }, + connectionTimeout: TIMEOUT_MS, + greetingTimeout: TIMEOUT_MS, + socketTimeout: TIMEOUT_MS, + /* + * No logging at all, and this is a decision rather than noise control. imapflow's logger + * writes the commands it sends, and one of them is LOGIN. A deployment that turned this on + * would put the mailbox password into whatever collects its stdout. + */ + logger: false, + // The connection lives for one call, so idling would only ever be a command the next call has + // to interrupt. + disableAutoIdle: true, + }); + + try { + return await withDeadline( + async () => { + await client.connect(); + return await use(client); + }, + // Closed rather than logged out: a deadline has expired because the server is not answering, + // and LOGOUT is another command to wait for. + () => client.close(), + what, + ); + } catch (error) { + // Rewrapped so a caller never has to care whether the failure came from the socket, the + // login or the command, and so what reaches an audit row is one sentence rather than a stack. + throw new MailboxError(mailServerSentence(error)); + } finally { + await client.logout().catch(() => { + client.close(); + }); + } + } + + /** + * Hold the mailbox lock for exactly the work that reads it. + * + * imapflow's own rule, and the reason it offers a lock at all: SELECT is connection state, so two + * pieces of work sharing a connection can otherwise read each other's mailbox. Released in a + * `finally` for the same reason the client is closed in one. + */ + async function withMailbox( + client: ImapLike, + mailbox: string, + use: () => Promise, + ): Promise { + let lock: Awaited>; + try { + lock = await client.getMailboxLock(mailbox); + } catch (error) { + /* + * A folder that is not there is answered with the folders that are, on the same connection + * and inside the same deadline. See noSuchFolderSentence for why the server's own words are + * not enough here. + */ + if (!looksLikeMissingFolder(error)) throw error; + let folders: string[] = []; + try { + folders = (await client.list()) + .map((box) => box.path) + .filter( + (path): path is string => typeof path === "string" && path !== "", + ); + } catch { + // A LIST that fails leaves the correction without its examples, which is still better than + // the vendor's sentence. Never a reason to lose the refusal itself. + } + throw new MailboxError(noSuchFolderSentence(mailbox, account, folders)); + } + + try { + return await use(); + } finally { + lock.release(); + } + } + + return { + async withSession(use: (session: MailboxSession) => Promise) { + return withClient(async (client) => { + const session: MailboxSession = { + async recent(mailbox, limit) { + return withMailbox(client, mailbox, async () => { + const open = client.mailbox; + const exists = open ? open.exists : 0; + if (exists === 0) return { headers: [], total: 0 }; + + /* + * The tail of the mailbox by SEQUENCE, which is what "newest" means here. + * + * Sequence numbers are ordered by arrival and are contiguous, so the last `limit` of + * them is one range and one round trip. Uids are neither ordered by arrival in any + * guaranteed way nor contiguous, so asking for "the highest N uids" would be a fetch + * of the whole mailbox first. + */ + const first = Math.max(1, exists - limit + 1); + const headers: MessageHeader[] = []; + for await (const message of client.fetch(`${first}:${exists}`, { + uid: true, + envelope: true, + flags: true, + })) { + headers.push(headerOf(message)); + } + // Newest first, which is the order a person reads a mailbox in. `exists` is the + // whole mailbox, so the caller can say ten of four thousand rather than ten. + return { headers: headers.reverse(), total: exists }; + }); + }, + + async message(mailbox, uid) { + return withMailbox(client, mailbox, async () => { + const message = await client.fetchOne( + String(uid), + { + uid: true, + envelope: true, + flags: true, + // Bounded on the WIRE, not after the fact. See MAX_SOURCE_BYTES. + source: { maxLength: MAX_SOURCE_BYTES }, + // What the whole message weighs, which is the only way to tell a message that + // ended at the cap from one that happens to be exactly that long. + size: true, + }, + { uid: true }, + ); + // `false` is imapflow's "no such message", and it is a different fact from a + // message that arrived without a source. Compared rather than falsy-checked so the + // type narrows: `message?.source` leaves `false` in the union below. + if (message === false || !message.source) return null; + + const parsed = await simpleParser(message.source); + const { body, bodyLength } = readBody(parsed); + const sizeBytes = + typeof message.size === "number" ? message.size : null; + return { + ...headerOf(message), + messageId: parsed.messageId ?? null, + references: referencesOf(parsed.references), + body, + bodyLength, + /* + * Either the server said the message is bigger than what we asked for, or it did not + * say and we got exactly the cap. The second is a message that MIGHT be exactly + * 512 KB, and saying it was cut when it was not is the better of the two errors: it + * understates what we hold rather than overstating it. + */ + sourceTruncated: + sizeBytes !== null + ? sizeBytes > MAX_SOURCE_BYTES + : message.source.length >= MAX_SOURCE_BYTES, + sizeBytes, + }; + }); + }, + + async search(mailbox, query, limit) { + return withMailbox(client, mailbox, async () => { + /* + * One IMAP SEARCH over the three fields a person means by "find the mail about X". + * `text` already covers headers and body, and subject and from are named beside it + * anyway: a server that indexes headers separately answers those far faster, and the + * OR of the three is what an unindexed server would have scanned regardless. + */ + const uids = await client.search( + { + or: [{ subject: query }, { from: query }, { text: query }], + }, + { uid: true }, + ); + if (!uids || uids.length === 0) return { headers: [], total: 0 }; + + // Highest uid last, so the tail is the newest matches. Reversed after fetching so the + // answer reads newest first, like the listing. + const wanted = uids.slice(-limit); + const headers: MessageHeader[] = []; + for await (const message of client.fetch( + wanted, + { uid: true, envelope: true, flags: true }, + { uid: true }, + )) { + headers.push(headerOf(message)); + } + // How many matched, not how many are being shown. A search that found 300 and is + // answering with 20 has to be able to say so. + return { headers: headers.reverse(), total: uids.length }; + }); + }, + }; + + return use(session); + }); + }, + + async send(message) { + /* + * COMPOSED ONCE, USED TWICE. `raw` goes to SMTP and the same `raw` is appended to Sent, so + * what the recipient holds and what the account's Sent folder holds are byte for byte the + * same message, down to the Message-ID a Bot would verify its own send by. + */ + const composed = await composeMessage(account, message); + + const build = + wire.smtp ?? ((options) => createTransport(options) as SmtpLike); + const transport = build({ + host: config.smtpHost, + port: config.smtpPort, + // Implicit TLS on 465, for the same reason IMAP uses it: SMTP AUTH sends the password. + secure: true, + auth: { user: account, pass: password }, + connectionTimeout: TIMEOUT_MS, + greetingTimeout: TIMEOUT_MS, + socketTimeout: TIMEOUT_MS, + }); + + try { + await withDeadline( + () => + transport.sendMail({ + envelope: composed.envelope, + raw: composed.raw, + }), + () => transport.close(), + "sending the message", + ); + } catch (error) { + throw new MailboxError(mailServerSentence(error)); + } finally { + transport.close(); + } + + const filed = await fileInSent(composed.raw); + return { + messageId: composed.messageId, + filedTo: filed.folder, + fileError: filed.error, + }; + }, + }; + + /** + * Put a copy of a message that has already gone out into the account's Sent folder. + * + * AFTER THE SEND AND NEVER IN FRONT OF IT. The mail is already delivered by the time this runs, + * so nothing it does can stop or duplicate a delivery, and nothing it fails at is a failed send. + * That is why every failure here is caught and RETURNED rather than thrown: a throw would reach + * the tools as "the send failed", and a Bot told that resends a message that cannot be recalled. + * + * Why it is needed at all: SMTP delivers, it does not file. Without this the account's Sent + * folder stays empty, webmail shows nothing sent, and a person checking concludes the mail was + * never sent. That happened, to three messages that had all been delivered. + * + * On its own connection, inside the same per-operation deadline as everything else here, and + * marked `\\Seen` because the account did not receive this message, it wrote it. + */ + async function fileInSent( + raw: Buffer, + ): Promise<{ folder: string | null; error: string | null }> { + try { + return await withClient(async (client) => { + const folder = sentFolderFrom(await client.list()); + if (!folder) { + return { folder: null, error: "this account has no Sent folder" }; + } + await client.append(folder, raw, ["\\Seen"], new Date()); + return { folder, error: null }; + }, "filing the copy in Sent"); + } catch (error) { + return { folder: null, error: mailServerSentence(error) }; + } + } +} diff --git a/server/src/plugins/builtin-mailbox.ts b/server/src/plugins/builtin-mailbox.ts new file mode 100644 index 00000000..c2353dda --- /dev/null +++ b/server/src/plugins/builtin-mailbox.ts @@ -0,0 +1,1048 @@ +import type { MailboxConfig } from "../config"; +import { + createMailboxClients, + type FullMessage, + MAX_BODY_CHARS, + MAX_SOURCE_BYTES, + type MailboxClients, + MailboxError, + type MessageHeader, + type MessagePage, + type OutgoingMessage, +} from "../mailbox/client"; +import { MAX_RESULT_CHARS, type McpCallResult, type McpTool } from "./mcp"; + +/** + * The builtin transport for the Mailbox: a Bot reading and answering the deployment's mail, without + * leaving the building. + * + * WHAT MAKES THIS DIFFERENT FROM ROUTINES, the other builtin. Routines has no credential at all, + * because it acts on this deployment's own tables and the ACTOR is the authorization. This one does + * have a credential, a mailbox password, and it is the deployment's rather than anybody's: there is one + * mailbox, every Bot granted these tools reads the same one, and no person consents to it. So the + * authorization here is the GRANT: an administrator decides which Bots may read the mail and which + * may answer it, per tool, on the Plugins page, exactly as they would for a vendor's connector. + * + * That is why nothing below reads `connection.actorId` as permission. A run that reaches here has + * already passed the grant check and the policy decision in `plugins/store.ts`, and there is no + * per-person narrowing left to do: the mailbox does not have somebody's half. + * + * THE PASSWORD IS NEVER IN THE ENVIRONMENT and never in an answer. The hosts and the accounts come + * from `config.ts`; each account's password comes from the encrypted credential vault, as that + * account's own credential, resolved through {@link MailboxAccess.password} at the moment a call + * needs it and thrown away after. {@link redacted} is the last line of that: a failure sentence + * from a mail server that echoed the login back is scrubbed before anybody reads it. + * + * SEVERAL ACCOUNTS, ONE DEPLOYMENT. `support@`, `sales@` and `billing@` on the same shared host are + * one configuration with one host pair and a password each. Which one a call works in is the + * `account` argument, defaulting to the first configured. It changes nothing about the access + * model: the accounts are all the deployment's, the grant is still per tool rather than per + * account, and a Bot granted these tools reaches every one of them. + * + * It implements the same interface as every other transport, as module-level exports, because that + * is the shape {@link ./transport} resolves: a `TransportKind` maps to a MODULE. Which is also why + * the configuration and the vault arrive through {@link useMailbox} rather than a constructor: the + * registry is built at import time, long before `index.ts` has either. + */ + +/** + * What the tools act on: where the mailbox is, how to unlock it, and how the protocols are spoken. + * + * `password` is a function rather than a string because a secret read once at boot is a secret held + * in memory for the life of the process and stale the moment an administrator rotates it. Read per + * call, a rotation takes effect on the next call and a revocation refuses it. + */ +export type MailboxAccess = { + config: MailboxConfig; + /** + * That account's password from the vault, or null when this deployment holds none for it. + * + * Per account rather than per deployment, because a shared host gives each mailbox its own + * login. A deployment can hold the password for one account and not another, and that is a + * working deployment for the account it has: the answer names the account that is missing one. + */ + password: (account: string) => Promise; + /** + * How IMAP and SMTP are actually spoken. Defaults to imapflow and nodemailer. + * + * Injected so a test can assert what a call was about to go out with, which is the same reason + * `PluginStoreOptions.callVendor` exists: the reply headers, the caps and the argument checking + * are the properties worth being sure about, and asserting them otherwise would need a reachable + * mail server. + */ + clients?: ( + config: MailboxConfig, + account: string, + password: string, + ) => MailboxClients; +}; + +/** + * Which credential in the vault is one account's password. + * + * Here rather than at the one call site in `index.ts`, because it is a contract with two other + * parties: the administrator who types these three values at `/admin/credentials`, and + * `docs/mailbox.md`, which tells them to. Three strings agreeing across three places by convention + * is how a deployment ends up holding the right secret under a key nothing reads. + * + * THE KEY ID IS THE ADDRESS, which is what makes several accounts possible at all: one credential + * per configured address, so a rotation, a revocation and a missing password are each one + * account's rather than the whole mailbox's. Lower-cased on the way in, matching what `config.ts` + * stores, so an administrator who typed `Support@` and a caller who asked for `support@` reach the + * same row. + * + * `mcp` is the kind because that is the vault's name for "the one token this deployment holds for + * this server", which is the same kind a custom MCP server's own bearer token is stored under and + * the same thing a mailbox password is: one secret, the deployment's, used for every Bot granted + * the tools. + */ +export function mailboxCredentialFor(account: string): { + kind: "mcp"; + provider: "mailbox"; + keyId: string; +} { + return { kind: "mcp", provider: "mailbox", keyId: account.toLowerCase() }; +} + +let installed: MailboxAccess | null = null; + +/** + * Hand this module the mailbox, once, from the place that has the configuration and the vault. + * + * A module-level binding rather than a constructor argument, for the reason {@link + * ./builtin-routines.useRoutineTools} gives: `transportFor` resolves a kind to a MODULE and there is + * no seam to pass anything through. `null` is a supported argument, and not only for symmetry: the + * suite is one process, so a test that installs a stub has to be able to take it back out again, + * and a deployment with no mailbox configured installs nothing. + */ +export function useMailbox(access: MailboxAccess | null): void { + installed = access; +} + +/** The IMAP folder a tool reads when the call did not name one. */ +const DEFAULT_FOLDER = "INBOX"; + +/** Bounds on how much mail one call may return. See {@link boundedLimit}. */ +const LIST_LIMIT = { fallback: 10, max: 50 } as const; +const SEARCH_LIMIT = { fallback: 20, max: 50 } as const; + +/** + * The largest number that can be a uid, which is what IMAP's own 32-bit field allows (RFC 3501). + * + * Checked here so a model that produced `1e21` is told there is no such message, in the same words + * as any other uid that is not there. Without it the number reaches imapflow, which refuses to + * compile a sequence set out of it, and the model is handed "Invalid sequence set value" about an + * argument it thinks of as a message number. + */ +const MAX_UID = 4_294_967_295; + +/** + * What a call answers when this deployment has no mailbox configured. + * + * Names all four things, including the one that is not an environment variable, because the half a + * reader is most likely to be missing is the half that is not in `.env`. The catalogue entry stays + * admissible and grantable without any of it (see `DeploymentConfig.mailbox`), so this sentence, + * rather than a missing connector, is how a deployment finds out. + */ +const NOT_CONFIGURED = + "Mailbox is not configured. Set MAILBOX_IMAP_HOST, MAILBOX_SMTP_HOST, MAILBOX_USERS and store each account's password as its mailbox credential."; + +/** + * What a call answers when the hosts are configured and the vault holds no password for the account. + * + * Its own sentence rather than the one above, because it is a different job with a different fix: an + * administrator has already set the three variables and has one step left, at a different screen. + * Telling them to set variables they can see are already set is how a correct instruction gets read + * as a broken deployment. + * + * IT NAMES THE ACCOUNT, and with several of them that is the whole content of the message. A + * deployment that stored `support@` and forgot `billing@` is working for one account and broken for + * the other, and a sentence saying only "the mailbox" would send an administrator to look at the + * credential that is already there. + */ +function noPassword(account: string): string { + return `This deployment holds no password for the mailbox ${account}. An administrator has to store it as that account's mailbox credential (kind \`mcp\`, provider \`mailbox\`, key id \`${account}\`) before mail can be read or sent.`; +} + +/** + * The four tools, as the same shape a server would have answered `tools/list` with. + * + * THE DESCRIPTIONS CARRY THE THINGS A MODEL CANNOT SEE. Two of them matter enough to be spelled out + * rather than implied by a field name. The first is that a uid belongs to one mailbox: a uid read + * out of a listing of `Archive` names a different message in `INBOX`, and a model that carries one + * across will confidently open the wrong mail. The second is that this is ONE shared mailbox rather + * than the mailbox of whoever is asking. A Bot that believes it is reading the person's own mail + * will summarize somebody else's inbox to them without either party ever saying so. + */ +/** + * The `folder` argument, in the one wording all four tools use. + * + * IT SAYS WHAT IT IS NOT, and that sentence is the whole point of this constant. A smaller model + * given two arguments that both read as "which mailbox" puts the email address in the wrong one: + * observed live, a Bot passed `support@example.com` as the folder and was answered "Character not + * allowed in mailbox name" by the IMAP server, then "Mailbox doesn't exist: support", and never + * tried `account` at all. Naming the folder `folder`, saying it is not an address, and pointing at + * the argument that is one costs three lines here and saves a run. + */ +const FOLDER_ARGUMENT = Object.freeze({ + type: "string", + description: `IMAP folder to read, such as ${DEFAULT_FOLDER}, Sent or Archive. Default ${DEFAULT_FOLDER}. Leave it unset unless the person names a folder. This is not an email address, and it is not the part before the @ of one; to choose the account, use \`account\`.`, +} as const); + +const TOOLS: readonly McpTool[] = Object.freeze([ + { + name: "list_messages", + description: [ + "List the newest messages in this deployment's mailbox, newest first: each one's uid, when it", + "arrived, who sent it, its subject and whether it has been read. No bodies: open one with", + "`read_message`.", + "", + "This is a single shared mailbox belonging to the deployment, not the mailbox of the person you are", + "talking to. Say whose it is if it is not obvious from the conversation, and never present its", + "contents as their own mail.", + "", + "`uid` values are per folder. A uid from a listing of one folder names a different message in", + "another, so pass the same `folder` to `read_message` that you listed.", + ].join("\n"), + inputSchema: { + type: "object", + properties: { + limit: { + type: "integer", + description: `How many messages to list. Default ${LIST_LIMIT.fallback}, at most ${LIST_LIMIT.max}.`, + }, + folder: FOLDER_ARGUMENT, + }, + }, + }, + { + name: "read_message", + description: [ + "Open one message and read it: its headers and its text, by the uid from `list_messages` or", + "`search_messages`.", + "", + "A long message is cut off and says so. Nothing is marked read, moved or deleted by opening it.", + "", + "Pass the same `folder` the uid came from. Uids are per folder, so a uid listed in one names a", + "different message in another.", + ].join("\n"), + inputSchema: { + type: "object", + properties: { + uid: { + type: "integer", + description: "The message's uid, from a listing or a search.", + }, + folder: FOLDER_ARGUMENT, + }, + required: ["uid"], + }, + }, + { + name: "search_messages", + description: [ + "Find messages whose subject, sender or text matches `query`, newest first. Answers with the same", + "header lines `list_messages` does; open one with `read_message`.", + "", + "The match is the mail server's own, which is a plain substring search rather than a search engine:", + "one or two distinctive words find more than a sentence does, and there is no ranking, no stemming", + "and no boolean syntax. A search that finds nothing says so.", + ].join("\n"), + inputSchema: { + type: "object", + properties: { + query: { + type: "string", + description: + "What to look for, matched against subject, sender and message text.", + }, + limit: { + type: "integer", + description: `How many matches to return. Default ${SEARCH_LIMIT.fallback}, at most ${SEARCH_LIMIT.max}.`, + }, + folder: FOLDER_ARGUMENT, + }, + required: ["query"], + }, + }, + { + name: "send_message", + description: [ + "Send mail from this deployment's mailbox. It goes out immediately and there is nothing to recall,", + "so read what you are about to send back to the person first whenever the wording is theirs to", + "approve.", + "", + "To answer a message rather than start a new conversation, give `in_reply_to` as the uid of the", + "message you are answering. The reply is then threaded properly in the recipient's mail client, and", + '"Re: " is put in front of the subject if it is not already there. Without it the message opens a new', + "thread, however the subject is worded.", + "", + "The sender is always the account this call works in, which is one of the deployment's own", + "mailboxes. There is no field for it, and the mail says who it is from, so do not sign it as", + "somebody else.", + "", + "Some deployments only allow mail to certain domains. If this is one of them, a recipient outside", + "them is refused, nothing is sent, and the refusal names the domain: report that plainly rather", + "than trying another address.", + ].join("\n"), + inputSchema: { + type: "object", + properties: { + to: { + type: "string", + description: + "The recipient's address. Several are separated by commas.", + }, + subject: { type: "string", description: "The subject line." }, + body: { + type: "string", + description: "The message itself, as plain text.", + }, + in_reply_to: { + type: "integer", + description: + "The uid of the message this answers, so the reply threads. Omit for a new conversation.", + }, + folder: FOLDER_ARGUMENT, + }, + required: ["to", "subject", "body"], + }, + }, +]); + +/** + * Who this call is for, and which Bot is making it. + * + * The whole shared connection shape, all of it unused: there is no host to dial from a URL, no token + * to send, and the actor is not what authorizes this one. Declared anyway because it is the + * transport interface, and named here so the reason is written down where somebody would look for + * a missing check. + */ +type Connection = { + url: string; + token?: string; + actorId?: string; + botId?: string; +}; + +/** + * The list is static and needs neither a credential nor a configured mailbox. + * + * The four definitions are schemas in this file: nothing to discover, nobody to ask. Listing them + * without a mailbox configured is deliberate rather than an oversight. An administrator sets a + * connector up and grants its tools before, or instead of, the deployment ever having the secret, + * and a tool list that emptied itself when a variable was unset would revoke grants by accident. + */ +export async function listTools(): Promise { + const users = installed?.config.users ?? null; + return TOOLS.map((tool) => withAccount(tool, users)); +} + +/** + * One tool definition, plus the `account` argument and the sentence explaining it. + * + * ADDED HERE RATHER THAN WRITTEN INTO {@link TOOLS} because the choices are a deployment's, not this + * file's. A model that is told the addresses can pick one; a model told only that an `account` + * argument exists has to guess at a string, and a guessed address is a refusal at best. The names + * are already in front of it in every listing this connector answers with, so naming them in the + * description reveals nothing a granted Bot could not already read. + * + * The generic wording is for the deployment with no mailbox configured, where the list is still + * answered (see {@link listTools}) and there are no addresses to name yet. + */ +function withAccount(tool: McpTool, users: readonly string[] | null): McpTool { + const configured = users && users.length > 0 ? users : null; + const oneOf = configured + ? `One of: ${configured.join(", ")}.` + : "One of the deployment's configured addresses."; + const fallback = configured + ? `Default ${configured[0]}.` + : "Default is the first of them."; + + /* + * ACCOUNT FIRST, deliberately. A model reads a schema in order, and the argument it meets first is + * the one it reaches for when it wants to say "the support mailbox". Meeting `folder` first is how + * an address, and then the local part of an address, ends up in it. + */ + const properties = { + account: { + type: "string", + description: `Which mailbox account to use, as its email address. ${oneOf} ${fallback}`, + }, + ...((tool.inputSchema.properties as Record | undefined) ?? + {}), + }; + + return { + ...tool, + description: [ + tool.description, + "", + `\`account\` is the email address of the mailbox to work in. ${oneOf} ${fallback} It is a different argument from \`folder\`, which names an IMAP folder such as ${DEFAULT_FOLDER}: an address belongs in \`account\` and never in \`folder\`. A uid, like a folder, belongs to one account, so do not carry one across.`, + ].join("\n"), + inputSchema: { ...tool.inputSchema, properties }, + }; +} + +export const listNeedsCredential = false; + +const failure = (message: string): McpCallResult => ({ + text: message, + isError: true, + truncated: false, +}); + +/** Success as a result, with the same visible cap the vendor transports use. */ +function asResult(text: string): McpCallResult { + if (text.length <= MAX_RESULT_CHARS) { + return { text, isError: false, truncated: false }; + } + return { + text: `${text.slice(0, MAX_RESULT_CHARS)}\n\n[truncated: the tool returned ${text.length} characters]`, + isError: false, + truncated: true, + }; +} + +/** A string argument that was actually given, or nothing. Blank is not a value. */ +function stringArg( + args: Record, + key: string, +): string | undefined { + const value = args[key]; + return typeof value === "string" && value.trim() !== "" + ? value.trim() + : undefined; +} + +/** + * A whole number argument, or nothing, or a refusal. + * + * A string of digits counts. Models produce `"42"` for an integer field often enough that refusing + * it would be refusing a correct intention over a JSON type, and there is nothing ambiguous about + * it. Anything else that is not a positive whole number is refused rather than rounded or coerced: + * a uid is an identity, and `12.7` silently becoming 12 is a different message. + */ +function integerArg( + args: Record, + key: string, +): { value?: number; error?: string } { + const raw = args[key]; + if (raw === undefined || raw === null || raw === "") return {}; + + const value = + typeof raw === "number" + ? raw + : typeof raw === "string" + ? Number(raw.trim()) + : Number.NaN; + if (!Number.isInteger(value) || value < 1) { + return { error: `\`${key}\` has to be a whole number, at least 1.` }; + } + return { value }; +} + +/** + * How many messages one call returns, and whether the ask was cut down. + * + * Capped rather than refused, which is the opposite of what this file does with a malformed number, + * and the difference is what the number means. A malformed uid is a mistake only the model can fix; + * "give me 200 messages" is a perfectly clear intention that this tool simply does not serve, and + * refusing it would cost a whole extra round trip to be told a smaller number. The cap is SAID, in + * the answer, so a model that asked for 200 and got 50 knows there may be more rather than + * concluding the mailbox holds fifty messages. + */ +export function boundedLimit( + asked: number | undefined, + bounds: { fallback: number; max: number }, +): { limit: number; capped: boolean } { + if (asked === undefined) return { limit: bounds.fallback, capped: false }; + if (asked > bounds.max) return { limit: bounds.max, capped: true }; + return { limit: asked, capped: false }; +} + +/** + * How a mailbox is named in an answer: the folder, and which account's folder it is. + * + * One function so every sentence says it the same way. The account is in all of them rather than + * only in the ambiguous ones, because a model reading "showing 10 of 236 messages in INBOX" across + * two accounts in one turn has no way to tell which INBOX either page came from, and will merge + * them. + * + * The word "folder" is in it for the model rather than for the reader: every answer this connector + * gives then models the vocabulary the arguments use, so the thing before "of" reads as a folder + * and the thing after it reads as an account, in the same sentence. + */ +function where(folder: string, account: string): string { + return `folder ${folder} of ${account}`; +} + +/** + * The sentence for a uid that names nothing, wherever it was noticed. + * + * One function because there are two ways to arrive at it and they are the same fact to whoever is + * reading: the mailbox answered with no such message, or the number was never one a mailbox could + * hold. A model told two different things about one situation will try two different fixes. + */ +function noSuchMessage(uid: number, place: string): string { + return `There is no message with uid ${uid} in ${place}. Uids are per folder, so check the listing this one came from.`; +} + +/** + * The lines that go under a listing when there is more than it showed. + * + * Two separate facts, and a call can have both. "There are 4000 messages and you are seeing 10" is + * about the mailbox; "50 is the most this tool will list" is about the tool, and only appears when + * somebody asked for more than that. Said, because a page of ten headers with nothing else on it + * reads to a model as the whole mailbox, and it will answer "you have ten messages" about a mailbox + * holding four thousand. + */ +function pageNotes( + page: MessagePage, + capped: boolean, + max: number, + what: string, +): string[] { + const notes: string[] = []; + if (page.total > page.headers.length) { + notes.push( + `[showing ${page.headers.length} of ${page.total} ${what}, newest first.]`, + ); + } + if (capped) { + notes.push( + `[${max} is the most this tool lists at once, so there may be more than these.]`, + ); + } + return notes.length > 0 ? ["", ...notes] : []; +} + +/** One message as a line in a listing. Empty fields are named, never left blank. */ +function headerLine(header: MessageHeader): string { + return [ + `uid ${header.uid}`, + header.date ?? "no date", + `from ${header.from || "an unnamed sender"}`, + `"${header.subject || "(no subject)"}"`, + header.seen ? "read" : "unread", + ].join(" · "); +} + +/** + * One message, opened. + * + * The cut is stated in the body's own terms (how long it was, where it stopped) rather than as a + * generic truncation note, because the model's next move depends on it: a message cut at 8000 of + * 9000 characters has almost certainly said what it came to say, and one cut at 8000 of 400000 has + * not. Same reasoning as `mcp.ts`'s cap note, applied one level down, since the whole result is + * capped again above this. + */ +function messageInWords(message: FullMessage, place: string): string { + const lines = [ + `uid ${message.uid} in ${place}`, + `From: ${message.from || "an unnamed sender"}`, + `To: ${message.to || "nobody named"}`, + `Date: ${message.date ?? "not stated"}`, + `Subject: ${message.subject || "(no subject)"}`, + "", + message.body || "This message has no readable text.", + ]; + if (message.bodyLength > MAX_BODY_CHARS) { + lines.push( + "", + `[truncated: the message body is ${message.bodyLength} characters and the first ${MAX_BODY_CHARS} are shown]`, + ); + } + /* + * A different fact from a long body, and it has to be said separately. + * + * A cut body means this deployment holds the whole message and is showing part of it. A cut + * SOURCE means it never read the rest off the wire, so what is missing is missing everywhere: + * later parts, attachments, and anything a model might otherwise offer to go back for. + */ + if (message.sourceTruncated) { + const weight = + message.sizeBytes === null + ? "" + : ` of the message's ${message.sizeBytes} bytes`; + lines.push( + `[only the first ${MAX_SOURCE_BYTES} bytes${weight} were read, so anything later in it, including attachments, was not seen]`, + ); + } + return lines.join("\n"); +} + +/** + * The subject and the two headers that make a reply a reply. + * + * WHY THE ORIGINAL IS FETCHED RATHER THAN NAMED. Threading is done on `Message-ID`, and a model + * cannot know one: it is a header a person never sees. Given the uid of a message that exists, the + * ids come from the message itself, so a reply is threaded against something real or is not + * threaded at all. + * + * `References` is the original's own chain with the original appended, which is RFC 5322 §3.6.4's + * rule and the thing mail clients actually walk. A message with no `Message-ID` gets neither header: + * an `In-Reply-To` pointing at nothing is not a reply, and inventing an id would thread the answer + * into a conversation that does not exist. + * + * The subject keeps whatever the caller wrote and only gains a prefix. A reply marker already + * there is left alone rather than stacked, and the check is not just for English: `AW:` is German, + * `SV:` Swedish, `Antw:` Dutch and `Ref:` Italian, and a client that only knew `Re:` is how a thread + * ends up titled `Re: AW: Re: AW: the numbers`. Case-insensitive, and tolerant of the space some + * clients put before the colon. + */ +const REPLY_PREFIX = /^(re|aw|sv|antw|ref)\s*:/i; +export function replyFrom( + original: Pick, + subject: string, +): { subject: string; reply?: OutgoingMessage["reply"] } { + const prefixed = REPLY_PREFIX.test(subject.trim()) + ? subject + : `Re: ${subject}`.trim(); + if (!original.messageId) return { subject: prefixed }; + return { + subject: prefixed, + reply: { + messageId: original.messageId, + references: [...original.references, original.messageId], + }, + }; +} + +/** + * A sentence with the password taken out of it, in every spelling it can appear in. + * + * Belt and braces over a rule already kept: imapflow is built with `logger: false` and nothing in + * this deployment prints the secret. What this covers is the sentence a SERVER wrote. Both protocols + * quote the offending command back on a failed login, and nodemailer appends the raw SMTP reply to + * its error message, so the credential can arrive here inside somebody else's words. That text goes + * into an audit row and in front of a model, and neither is a place for the mailbox password. + * + * THE BASE64 FORMS ARE NOT PARANOIA, they are the common case. Neither client prefers plaintext + * `LOGIN`: imapflow authenticates with `AUTH=PLAIN` when the server offers it and falls back to + * `AUTH=LOGIN`, and both put the credential on the wire base64-encoded. So a quoted command carries + * `base64("\0user\0password")` or `base64(password)` rather than the password as typed, and a + * redaction that only knew the plaintext would pass it through unchanged while looking like it + * worked. + * + * A short or empty password is skipped rather than replaced everywhere, since replacing a + * one-character string would redact half the alphabet out of an unrelated message. + */ +export function redacted( + message: string, + password: string | null, + user?: string, +): string { + if (!password || password.length < 4) return message; + + const base64 = (value: string) => + Buffer.from(value, "utf8").toString("base64"); + const forms = [ + password, + // AUTH=LOGIN sends the password on its own line. + base64(password), + // AUTH=PLAIN sends authzid, authcid and password as one NUL-separated blob. + ...(user ? [base64(`\u0000${user}\u0000${password}`)] : []), + ]; + + let scrubbed = message; + for (const form of forms) { + if (form.length < 4) continue; + scrubbed = scrubbed.split(form).join("[redacted]"); + } + return scrubbed; +} + +/** + * The recipient domains this deployment will not send to, out of a `to` field. + * + * Returned rather than thrown, and returned as the DOMAINS rather than as a yes or no, because the + * refusal has to name what was wrong: a model told only that the recipient was refused will try + * another address, and one told the domain will say plainly that this deployment does not mail + * outside it. + * + * `to` is what the model wrote, so this parses defensively: comma-separated, each part either a bare + * address or `Name
`. Anything with no `@`, or nothing after it, is reported as an offender + * too, because an unparseable recipient is not a recipient this list has cleared, and letting it + * through to be somebody else's validation error would be a hole in a safety check. + * + * An empty allowlist means unrestricted and is answered before any parsing. + */ +export function refusedRecipients( + to: string, + allowed: ReadonlySet, +): string[] { + if (allowed.size === 0) return []; + + const refused: string[] = []; + for (const part of to.split(",")) { + const trimmed = part.trim(); + if (trimmed === "") continue; + const angled = /<([^>]*)>/.exec(trimmed); + const address = (angled ? angled[1] : trimmed).trim(); + const at = address.lastIndexOf("@"); + const domain = at === -1 ? "" : address.slice(at + 1).toLowerCase(); + if (domain === "" || !allowed.has(domain)) { + refused.push(domain === "" ? address : domain); + } + } + return [...new Set(refused)]; +} + +/** + * Which account this call works in. + * + * Unset is the first configured account, which is what makes `account` an argument a model may + * ignore: the common deployment has one mailbox and never sees this. + * + * AN ACCOUNT THAT IS NOT CONFIGURED IS REFUSED HERE, before a password is read and long before + * anything is dialled, and the refusal lists the ones that exist. Two reasons. A model that + * invented an address should be corrected rather than handed a login failure from a mail server, + * which is a sentence about credentials for a mailbox that does not exist; and `send_message` must + * not reach the network on a mistaken argument, since the interesting mistake is a Bot that was + * talked into naming an account by the mail it just read. + * + * Matched case-insensitively, because the configured list is lower-cased and an address is. + */ +export function selectAccount( + args: Record, + users: readonly string[], +): { account?: string; error?: string } { + const asked = stringArg(args, "account"); + if (asked === undefined) return { account: users[0] }; + + const wanted = asked.toLowerCase(); + if (!users.includes(wanted)) { + return { + error: `${asked} is not one of this deployment's mailbox accounts. It has ${users.join(", ")}. Nothing was read and nothing was sent.`, + }; + } + return { account: wanted }; +} + +/** + * Which IMAP folder this call reads, and the one mistake worth catching by hand. + * + * AN ADDRESS IN `folder` IS REFUSED HERE, before the vault and before the network. It is the + * mistake a smaller model actually makes: two arguments that both read as "which mailbox", and the + * address goes in the wrong one. Left to the mail server it comes back as "Character not allowed in + * mailbox name: '.'" and then "Mailbox doesn't exist: support", which are sentences about IMAP + * folder naming that no model turns into "use the other argument". Said here, the fix is in the + * refusal. + * + * THE LOCAL PART IS THE SECOND HALF OF THE SAME MISTAKE, and it was made live: refused an address + * in `folder`, the model retried with `support`, then `webmaster`, which are the parts before the @ + * of two configured accounts. That is not an address any more, so the check above lets it through, + * and the mail server answers "Mailbox doesn't exist: support", which reads as a folder that is + * merely missing rather than as an argument that is wrong. A folder genuinely named after an + * account's local part is possible and is given up here on purpose: the mistake is common and the + * collision is not. + * + * `mailbox` is still read as a name for the same argument. This connector shipped with that key, + * and a Bot holding a tool list from before the rename would otherwise pass a folder that is + * silently ignored and read INBOX while believing it read Archive. Both checks cover it either + * way, so the old name cannot reintroduce the trap. + */ +export function selectFolder( + args: Record, + users: readonly string[], +): { folder?: string; error?: string } { + const asked = stringArg(args, "folder") ?? stringArg(args, "mailbox"); + if (asked === undefined) return { folder: DEFAULT_FOLDER }; + if (asked.includes("@")) { + return { + error: `${asked} looks like an email address, and \`folder\` names an IMAP folder such as ${DEFAULT_FOLDER}. Pass the address as \`account\` instead. Nothing was read and nothing was sent.`, + }; + } + + const wanted = asked.toLowerCase(); + const named = users.find((user) => user.split("@")[0] === wanted); + if (named) { + return { + error: `${asked} is the account ${named}, not a folder. Pass account=${named} and leave folder unset to read ${DEFAULT_FOLDER}. Nothing was read and nothing was sent.`, + }; + } + return { folder: asked }; +} + +/** + * Which account and which folder, together, because the interesting case is a mistake across both. + * + * THE ADOPTION. A model that puts a configured address in `folder` has said something unambiguous: + * there is exactly one mailbox it can mean, and it named it. Refusing that costs a whole turn to + * learn a vocabulary lesson, and live runs show the FIRST mailbox call of a turn making this + * mistake, so the lesson is paid for before any work happens. So the address is taken as the + * account, the folder falls back to {@link DEFAULT_FOLDER}, and the answer says what was done. The + * note teaches the same lesson the refusal did, on the way past rather than instead of the work. + * + * WHAT IS STILL REFUSED, because neither is unambiguous: + * + * - An address in `folder` that is NOT a configured account. There is nothing to adopt: the model + * is asking for a mailbox this deployment does not have, and guessing which one it meant would + * read somebody else's mail to answer a question about a mailbox that is not there. + * - `folder` holding one configured address while `account` names a different one. Two arguments + * naming two mailboxes is a model that has lost track of which it is reading, and picking either + * would be picking for it. The refusal names both. + * + * The local-part refusal in {@link selectFolder} is deliberately left alone. `support` is not an + * address, and a folder genuinely called `support` can exist, so adopting it would be guessing + * where the address case is certain. + */ +export function selectMailbox( + args: Record, + users: readonly string[], +): { account?: string; folder?: string; note?: string; error?: string } { + const askedFolder = stringArg(args, "folder") ?? stringArg(args, "mailbox"); + const adopted = askedFolder?.toLowerCase(); + + if (adopted !== undefined && users.includes(adopted)) { + const askedAccount = stringArg(args, "account")?.toLowerCase(); + if (askedAccount !== undefined && askedAccount !== adopted) { + return { + error: `folder is ${askedFolder}, which is the account ${adopted}, while account is ${askedAccount}. Those are two different mailboxes and this call names no folder at all. Pass the one you mean as \`account\` and leave \`folder\` unset. Nothing was read and nothing was sent.`, + }; + } + return { + account: adopted, + folder: DEFAULT_FOLDER, + note: `[folder took the address ${adopted}; it was used as account, reading ${DEFAULT_FOLDER}.]`, + }; + } + + const account = selectAccount(args, users); + if (account.error || !account.account) return { error: account.error }; + const folder = selectFolder(args, users); + if (folder.error || !folder.folder) return { error: folder.error }; + return { account: account.account, folder: folder.folder }; +} + +/** + * Call one tool. + * + * The grant and the policy are already settled by the time anything gets here: `plugins/store.ts` + * checks what this Bot was given, evaluates the policy against the tool's effect, and writes the + * audit row, exactly as it does for a vendor's server. There is no second path to the mailbox and + * nothing here re-decides any of that. + * + * Nothing thrown escapes. A mail server that refused, timed out or answered nonsense comes back as + * an `isError` result rather than as a throw, which is what the vendor transports do and what + * `plugins/tools.ts` expects: it prefixes the sentence with "The vendor reported an error: " and the + * sentence survives intact, which is the part that matters to whoever reads the transcript. + */ +export async function callTool( + _connection: Connection, + toolName: string, + args: Record, +): Promise { + const access = installed; + if (!access) return failure(NOT_CONFIGURED); + + const chosen = selectMailbox(args, access.config.users); + if (chosen.error || !chosen.account || !chosen.folder) { + return failure(chosen.error ?? NOT_CONFIGURED); + } + const { account, folder, note } = chosen; + + const result = await runTool(access, account, folder, toolName, args); + if (!note) return result; + /* + * The note rides on the answer rather than replacing it. A model that is told what it did wrong + * AND handed the mail it asked for learns the argument without spending a turn on the lesson, + * which is the whole point of adopting the address instead of refusing it. + */ + return { ...result, text: `${note}\n${result.text}` }; +} + +/** The work itself, once the account and the folder are settled. */ +async function runTool( + access: MailboxAccess, + account: string, + folder: string, + toolName: string, + args: Record, +): Promise { + let password: string | null = null; + try { + password = await access.password(account); + if (!password) return failure(noPassword(account)); + + const clients = (access.clients ?? createMailboxClients)( + access.config, + account, + password, + ); + // The folder and whose folder, which is how every sentence below names it. See `where`. + const place = where(folder, account); + + if (toolName === "list_messages") { + const asked = integerArg(args, "limit"); + if (asked.error) return failure(asked.error); + const { limit, capped } = boundedLimit(asked.value, LIST_LIMIT); + + const page = await clients.withSession((session) => + session.recent(folder, limit), + ); + if (page.headers.length === 0) { + // Said in words rather than returned as an empty string: an empty result reads to a model + // as "the tool had nothing to say" and gets filled in from memory. + return asResult(`There are no messages in ${place}.`); + } + return asResult( + [ + ...page.headers.map((header) => `- ${headerLine(header)}`), + ...pageNotes(page, capped, LIST_LIMIT.max, `messages in ${place}`), + ].join("\n"), + ); + } + + if (toolName === "read_message") { + const uid = integerArg(args, "uid"); + if (uid.error) return failure(uid.error); + if (uid.value === undefined) { + return failure( + "Say which message to read, by the uid from list_messages or search_messages.", + ); + } + // A number no mailbox could hold is a message that is not there, and is answered as one + // rather than dialled and turned into a sequence-set complaint. See MAX_UID. + if (uid.value > MAX_UID) { + return failure(noSuchMessage(uid.value, place)); + } + + const message = await clients.withSession((session) => + session.message(folder, uid.value as number), + ); + if (!message) { + return failure(noSuchMessage(uid.value, place)); + } + return asResult(messageInWords(message, place)); + } + + if (toolName === "search_messages") { + const query = stringArg(args, "query"); + if (!query) return failure("Say what to search the mailbox for."); + const asked = integerArg(args, "limit"); + if (asked.error) return failure(asked.error); + const { limit, capped } = boundedLimit(asked.value, SEARCH_LIMIT); + + const page = await clients.withSession((session) => + session.search(folder, query, limit), + ); + if (page.headers.length === 0) { + return asResult( + `Nothing in ${place} matches "${query}". There is nothing here to answer from.`, + ); + } + return asResult( + [ + ...page.headers.map((header) => `- ${headerLine(header)}`), + ...pageNotes(page, capped, SEARCH_LIMIT.max, `matches in ${place}`), + ].join("\n"), + ); + } + + if (toolName === "send_message") { + const to = stringArg(args, "to"); + if (!to) return failure("Say who the message is to."); + const subject = stringArg(args, "subject"); + if (!subject) return failure("A message needs a subject."); + const body = stringArg(args, "body"); + if (!body) return failure("A message needs something in it."); + + /* + * WHERE IT IS GOING IS DECIDED BEFORE ANYTHING IS DIALLED. + * + * First, so a refused recipient costs no connection and touches no mailbox: this is the check + * that stands between a Bot that was talked into something by the mail it just read and an + * address outside the deployment. Naming the domain rather than the address, because the + * domain is the thing the allowlist is written about and the thing an administrator would + * change. + */ + const refused = refusedRecipients( + to, + access.config.allowedRecipientDomains, + ); + if (refused.length > 0) { + return failure( + `This deployment only sends mail to ${[...access.config.allowedRecipientDomains].sort().join(", ")}, and ${refused.join(", ")} is not among them. Nothing was sent. An administrator sets MAILBOX_ALLOWED_RECIPIENT_DOMAINS.`, + ); + } + + const inReplyTo = integerArg(args, "in_reply_to"); + if (inReplyTo.error) return failure(inReplyTo.error); + // The same answer a uid that is not there gets, for the same reason as `read_message`, and + // with the same promise that nothing was sent. + if (inReplyTo.value !== undefined && inReplyTo.value > MAX_UID) { + return failure( + `There is no message with uid ${inReplyTo.value} in ${place}, so there is nothing to reply to. Nothing was sent.`, + ); + } + + let outgoing: OutgoingMessage = { to, subject, body }; + if (inReplyTo.value !== undefined) { + const original = await clients.withSession((session) => + session.message(folder, inReplyTo.value as number), + ); + /* + * Refused rather than sent as a new message. A model that asked for a reply and got an + * unthreaded mail to the same person has been told the wrong thing about what it did, and + * the recipient sees an answer that appears to be about nothing. + */ + if (!original) { + return failure( + `There is no message with uid ${inReplyTo.value} in ${place}, so there is nothing to reply to. Nothing was sent.`, + ); + } + const threaded = replyFrom(original, subject); + outgoing = { + to, + subject: threaded.subject, + body, + ...(threaded.reply ? { reply: threaded.reply } : {}), + }; + } + + const sent = await clients.send(outgoing); + return asResult( + [ + `Sent from ${account} to ${to}, subject "${outgoing.subject}".`, + outgoing.reply + ? "It threads as a reply to that message." + : "It starts a new thread.", + /* + * WHERE THE COPY WENT, or why there is none, and never as a failure. + * + * Naming the folder is what lets a Bot check its own work: it can list that folder and + * find the message it just sent. The other branch is the one that matters more. Filing is + * a separate operation against a separate server and can fail after the mail has gone, so + * it says plainly that the mail WAS sent. A model told only that something failed sends + * the message again, and mail cannot be recalled. + */ + sent.filedTo + ? `A copy is in folder ${sent.filedTo}.` + : `The mail was sent, but no copy could be filed in the Sent folder (${redacted(sent.fileError ?? "no reason given", password, account).slice(0, 200)}). It was delivered, so do not send it again.`, + sent.messageId ? `Message id ${sent.messageId}.` : null, + ] + .filter((one): one is string => one !== null) + .join(" "), + ); + } + + return failure( + `${toolName} is not a tool Mailbox implements. The stored tool list is out of date; refresh it on the Plugins page.`, + ); + } catch (error) { + /* + * The mail server's own sentence, scrubbed of the password and nothing else. + * + * It is the most useful thing available, since "Invalid credentials", "Mailbox does not exist" + * and "Relay access denied" each name a different fix, and rewording it here would turn a + * specific failure into a vague one. Capped, because a failure is not a promise about length. + */ + const message = + error instanceof MailboxError || error instanceof Error + ? error.message + : String(error); + return failure(redacted(message, password, account).slice(0, 400)); + } +} diff --git a/server/src/plugins/catalogue.ts b/server/src/plugins/catalogue.ts index b8445bfc..4ce2008d 100644 --- a/server/src/plugins/catalogue.ts +++ b/server/src/plugins/catalogue.ts @@ -38,10 +38,16 @@ export type CatalogueAuth = /** One token, held by the deployment, used for everybody. */ | { kind: "deployment-bearer" } /** - * First-party and in-process. There is no credential, because there is nothing to authenticate - * to: the call runs against this deployment's own tables, as the person whose turn it is. + * First-party and in-process. There is no vendor to authenticate to, because the call runs inside + * this deployment rather than against somebody else's server. + * + * `reachedAs` is not a formality. The two builtins differ on the one question the audit trail + * asks: Routines touches the asking person's own rows and is therefore reached AS them, while + * Mailbox opens one mailbox that belongs to the deployment, on a password the deployment holds, + * and is reached as the deployment however many people ask. Recording the second as the asker + * would put a person's id on a row describing access that was never theirs. */ - | { kind: "builtin" } + | { kind: "builtin"; reachedAs: "actor" | "deployment" } /** * The asker's own grant. The deployment registers an OAuth client; each person consents once and * the call runs on their token, so the vendor decides what comes back. @@ -280,7 +286,9 @@ export const CATALOGUE: readonly CatalogueEntry[] = Object.freeze([ host: "builtin://routines", path: "/", transport: "builtin-routines", - auth: Object.freeze({ kind: "builtin" }), + // Reached as the person asking: a routine is theirs, it runs with their grants, and the rows it + // touches are their own. + auth: Object.freeze({ kind: "builtin", reachedAs: "actor" }), writeTools: Object.freeze([ "create_routine", "update_routine", @@ -288,6 +296,38 @@ export const CATALOGUE: readonly CatalogueEntry[] = Object.freeze([ ]), docsUrl: "https://github.com/CopilotKit/OpenBot/blob/main/docs/routines.md", }, + { + key: "mailbox", + title: "Mailbox", + vendor: "OpenBot", + summary: + "Read and send email from the deployment's mailbox, as the Bot granted it.", + /* + * In-process, like Routines, and in the catalogue for a sharper version of the same reason. + * There is no vendor to review here, but there is a capability to grant: mail is the one thing + * a Bot can do that reaches people who never agreed to talk to it, and `send_message` is + * irrevocable the instant it runs. Which Bots may read the mail, and which of those may answer + * it, is a decision an administrator makes per tool on the Plugins page. + * + * `builtin` names the auth kind because nothing here is authenticated per person: there is one + * mailbox, it belongs to the deployment, and its password is the deployment's own. That + * password is NOT reached through `credential_id` on the server row: see + * `plugins/builtin-mailbox.ts`, which resolves it from the vault as the `mailbox` credential. + */ + host: "builtin://mailbox", + path: "/", + transport: "builtin-mailbox", + // Reached as the deployment: one mailbox, one password this deployment holds, the same mail for + // everybody who asks. Whoever asked is still on the audit row as the actor; what this settles is + // that the access was not theirs. + auth: Object.freeze({ kind: "builtin", reachedAs: "deployment" }), + /* + * One write, and it is the whole reason this entry is grantable per tool: reading mail is + * recoverable and sending it is not. + */ + writeTools: Object.freeze(["send_message"]), + docsUrl: "https://github.com/CopilotKit/OpenBot/blob/main/docs/mailbox.md", + }, ]); const BY_KEY = new Map(CATALOGUE.map((entry) => [entry.key, entry])); diff --git a/server/src/plugins/store.ts b/server/src/plugins/store.ts index 11867e4c..07d36905 100644 --- a/server/src/plugins/store.ts +++ b/server/src/plugins/store.ts @@ -315,13 +315,22 @@ const iso = (value: Date | string | null): string | null => * the row a per-person connector exists to be able to trust. * * `deployment` for a shared token; the asker's own id for a server reached as the person asking. - * `builtin` is the third case and the only one with no credential at all — the actor is not whose - * token was used, it is whose rows were touched. + * `builtin` is the third case and the only one with no credential at all, so it answers for itself: + * Routines touches the asker's own rows and says so, while Mailbox opens one mailbox belonging to + * the deployment and says that instead. The entry decides, rather than this expression guessing from + * the kind, because a builtin reached as the deployment but recorded as the asker would put + * somebody's id on a row describing access that was never theirs. */ -const reachedAsFor = (entry: CatalogueEntry | null, actorId: string): string => - entry?.auth.kind === "user-oauth" || entry?.auth.kind === "builtin" - ? actorId - : "deployment"; +const reachedAsFor = ( + entry: CatalogueEntry | null, + actorId: string, +): string => { + if (entry?.auth.kind === "user-oauth") return actorId; + if (entry?.auth.kind === "builtin") { + return entry.auth.reachedAs === "actor" ? actorId : "deployment"; + } + return "deployment"; +}; /** * Where this server actually is, when the stored row and the catalogue disagree. diff --git a/server/src/plugins/transport.ts b/server/src/plugins/transport.ts index 4fecdb32..8ae7ecf1 100644 --- a/server/src/plugins/transport.ts +++ b/server/src/plugins/transport.ts @@ -1,3 +1,4 @@ +import * as builtinMailbox from "./builtin-mailbox"; import * as builtinRoutines from "./builtin-routines"; import type { CatalogueEntry } from "./catalogue"; import * as driveRest from "./google-drive-rest"; @@ -44,9 +45,10 @@ export type VendorTransport = { * Who this call is for, and which Bot is making it. * * Ignored by every transport that dials a vendor: MCP and Drive answer to a credential, and who - * holds it is already decided by the time the connection is built. The builtin transport has no - * credential and no vendor — it acts on this deployment's own tables — so the actor is not - * context, it is the authorization, and it refuses without one. A routine is somebody's. + * holds it is already decided by the time the connection is built. A builtin transport has no + * vendor, so it answers for itself: Routines refuses a call that names no actor, because a + * routine is somebody's and the actor is the authorization, while Mailbox does not, because + * there is one mailbox belonging to the deployment and the Bot's grant is the authorization. */ actorId?: string; /** The Bot the run belongs to. A routine runs as its Bot, which is never a name a model supplies. */ @@ -60,9 +62,10 @@ export type VendorTransport = { * Who this call is for, and which Bot is making it. * * Ignored by every transport that dials a vendor: MCP and Drive answer to a credential, and who - * holds it is already decided by the time the connection is built. The builtin transport has no - * credential and no vendor — it acts on this deployment's own tables — so the actor is not - * context, it is the authorization, and it refuses without one. A routine is somebody's. + * holds it is already decided by the time the connection is built. A builtin transport has no + * vendor, so it answers for itself: Routines refuses a call that names no actor, because a + * routine is somebody's and the actor is the authorization, while Mailbox does not, because + * there is one mailbox belonging to the deployment and the Bot's grant is the authorization. */ actorId?: string; /** The Bot the run belongs to. A routine runs as its Bot, which is never a name a model supplies. */ @@ -79,12 +82,17 @@ export type VendorTransport = { * A closed union rather than a string, so adding one is a change to this file and to the registry * below together. An entry naming a transport that does not exist should not typecheck. */ -export type TransportKind = "mcp" | "google-drive-rest" | "builtin-routines"; +export type TransportKind = + | "mcp" + | "google-drive-rest" + | "builtin-routines" + | "builtin-mailbox"; const TRANSPORTS: Record = { mcp, "google-drive-rest": driveRest, "builtin-routines": builtinRoutines, + "builtin-mailbox": builtinMailbox, }; /** diff --git a/server/tests/builtin-mailbox.test.ts b/server/tests/builtin-mailbox.test.ts new file mode 100644 index 00000000..bb1653c4 --- /dev/null +++ b/server/tests/builtin-mailbox.test.ts @@ -0,0 +1,1819 @@ +import { afterEach, describe, expect, test } from "bun:test"; +import type { MailboxConfig } from "../src/config"; +import { + composeMessage, + createMailboxClients, + type FullMessage, + type ImapLike, + MAX_BODY_CHARS, + MAX_FOLDERS_LISTED, + MAX_SOURCE_BYTES, + type MailboxClients, + MailboxError, + type MailboxSession, + type MessageHeader, + type MessagePage, + mailServerSentence, + noSuchFolderSentence, + type OutgoingMessage, + readBody, + type SendReceipt, + type SmtpLike, + sentFolderFrom, + strippedHtml, + withDeadline, +} from "../src/mailbox/client"; +import { + boundedLimit, + callTool, + listTools, + redacted, + refusedRecipients, + replyFrom, + selectAccount, + selectFolder, + selectMailbox, + useMailbox, +} from "../src/plugins/builtin-mailbox"; +import { MAX_RESULT_CHARS } from "../src/plugins/mcp"; + +/** + * The builtin Mailbox transport, asserted without a mail server. + * + * What is under test is the boundary, not IMAP: which arguments reach the protocol, what a model is + * told when they are wrong, how much text can come back, and whether a reply is threaded against a + * message that actually exists. A recording stub is installed through {@link useMailbox}, which is + * the seam this module has for exactly that reason: `transportFor` resolves a kind to a MODULE, so + * there is no constructor to pass a client to. + * + * The two properties this file exists for are the ones that are expensive to be wrong about. The + * first is the password: it is resolved per call, it must never appear in an answer, and a mail + * server that echoes it back into an error must not be able to launder it into a transcript or an + * audit row. The second is bounding: every result here is somebody else's text, arriving in a + * model's context window, and an unbounded body is a sender deciding how much of it to spend. + */ + +const CONNECTION = { + url: "builtin://mailbox/", + actorId: "user_asker", + botId: "bot_helper", +}; + +/** + * Two accounts on one pair of hosts, which is the shared-hosting shape this connector serves. + * + * The first is the default, so every case that names no account is also a case about which one that + * is. The second exists so "the account was honoured" is a claim that can fail: with one configured + * account every answer would name the right address by accident. + */ +const ACCOUNTS = ["bot@example.test", "sales@example.test"] as const; +const DEFAULT_ACCOUNT = ACCOUNTS[0]; + +const CONFIG: MailboxConfig = { + imapHost: "imap.example.test", + imapPort: 993, + smtpHost: "smtp.example.test", + smtpPort: 465, + users: [...ACCOUNTS], + // Unrestricted, which is the default and what every case here but the allowlist ones wants. + allowedRecipientDomains: new Set(), +}; + +const PASSWORD = "correct-horse-battery-staple"; + +const HEADER: MessageHeader = { + uid: 42, + from: "Dana Reid ", + to: "bot@example.test", + subject: "The Friday numbers", + date: "2026-08-30T09:15:00.000Z", + seen: false, +}; + +const MESSAGE: FullMessage = { + ...HEADER, + messageId: "", + references: [""], + body: "Can you send the Friday numbers?", + bodyLength: 32, + sourceTruncated: false, + sizeBytes: 1_200, +}; + +/** One page of one message, which is what most of these cases want back. */ +const onePage = (header: MessageHeader = HEADER): MessagePage => ({ + headers: [{ ...header }], + total: 1, +}); + +type Recorded = + | { method: "recent"; mailbox: string; limit: number } + | { method: "message"; mailbox: string; uid: number } + | { method: "search"; mailbox: string; query: string; limit: number } + | { method: "send"; message: OutgoingMessage }; + +type Stubs = { + recent?: (mailbox: string, limit: number) => Promise; + message?: (mailbox: string, uid: number) => Promise; + search?: ( + mailbox: string, + query: string, + limit: number, + ) => Promise; + send?: (message: OutgoingMessage) => Promise; + password?: (account: string) => Promise; + /** A deployment configured differently from {@link CONFIG}, for the allowlist cases. */ + config?: MailboxConfig; +}; + +/** + * Installs a mailbox whose protocols are recorded rather than spoken. + * + * The config and the password handed to the factory are recorded too, because "the call went out + * with the configured host and the vault's password" is a claim worth being able to check without a + * server that would have to answer it. + */ +function recordingMailbox(stubs: Stubs = {}): { + calls: Recorded[]; + built: { config: MailboxConfig; account: string; password: string }[]; + /** Which account each password lookup was for, in order. One vault row per account. */ + unlocked: string[]; +} { + const calls: Recorded[] = []; + const built: { + config: MailboxConfig; + account: string; + password: string; + }[] = []; + const unlocked: string[] = []; + + useMailbox({ + config: stubs.config ?? CONFIG, + password: async (account) => { + unlocked.push(account); + return stubs.password ? await stubs.password(account) : PASSWORD; + }, + clients: (config, account, password): MailboxClients => { + built.push({ config, account, password }); + const session: MailboxSession = { + async recent(mailbox, limit) { + calls.push({ method: "recent", mailbox, limit }); + return stubs.recent ? await stubs.recent(mailbox, limit) : onePage(); + }, + async message(mailbox, uid) { + calls.push({ method: "message", mailbox, uid }); + return stubs.message ? await stubs.message(mailbox, uid) : MESSAGE; + }, + async search(mailbox, query, limit) { + calls.push({ method: "search", mailbox, query, limit }); + return stubs.search + ? await stubs.search(mailbox, query, limit) + : onePage(); + }, + }; + return { + withSession: (use) => use(session), + async send(message) { + calls.push({ method: "send", message }); + return stubs.send + ? await stubs.send(message) + : { + messageId: "", + filedTo: "Sent", + fileError: null, + }; + }, + }; + }, + }); + + return { calls, built, unlocked }; +} + +// The binding is module-level and the suite is one process, so a mailbox left installed here would +// be the one some other file's test unexpectedly reaches. +afterEach(() => { + useMailbox(null); +}); + +describe("the tool list", () => { + test("is the four mailbox tools, named exactly", async () => { + const tools = await listTools(); + expect(tools.map((tool) => tool.name)).toEqual([ + "list_messages", + "read_message", + "search_messages", + "send_message", + ]); + for (const tool of tools) { + expect(tool.description.length).toBeGreaterThan(0); + expect(tool.inputSchema).toBeDefined(); + } + }); + + test("requires only what a call cannot be made without", async () => { + const required = Object.fromEntries( + (await listTools()).map((tool) => [ + tool.name, + (tool.inputSchema as { required?: string[] }).required ?? [], + ]), + ); + expect(required.list_messages).toEqual([]); + expect(required.read_message).toEqual(["uid"]); + expect(required.search_messages).toEqual(["query"]); + expect(required.send_message).toEqual(["to", "subject", "body"]); + }); + + test("says the two things a model cannot see: whose mailbox, and that uids are per mailbox", async () => { + const tools = await listTools(); + const list = tools.find((tool) => tool.name === "list_messages"); + // A Bot that believes it is reading the asker's own mail will summarize somebody else's inbox + // to them, and neither party will ever be told. + expect(list?.description ?? "").toContain("shared mailbox"); + expect(list?.description ?? "").toContain("per folder"); + + const send = tools.find((tool) => tool.name === "send_message"); + expect(send?.description ?? "").toContain("in_reply_to"); + expect(send?.description ?? "").toContain("nothing to recall"); + }); + + test("names the folder argument `folder`, and says it is not an address", async () => { + /* + * The live failure this rename came from: a smaller model passed the email address as + * `mailbox`, was told "Character not allowed in mailbox name: '.'" by the IMAP server, and + * never tried `account`. Two arguments that both read as "which mailbox" is the trap, so one of + * them is called `folder` and says in its own description what it is not. + */ + recordingMailbox(); + for (const tool of await listTools()) { + const properties = ( + tool.inputSchema as { + properties?: Record; + } + ).properties; + expect(properties?.folder).toBeDefined(); + expect(properties?.mailbox).toBeUndefined(); + const description = properties?.folder?.description ?? ""; + expect(description).toContain("IMAP folder"); + expect(description).toContain( + "Leave it unset unless the person names a folder", + ); + expect(description).toContain("This is not an email address"); + // The local part is the second half of the same mistake, so the argument says so itself. + expect(description).toContain("not the part before the @"); + expect(description).toContain("use `account`"); + + /* + * A model reads a schema in order, and the argument it meets first is the one it reaches for + * when it wants to say "the support mailbox". `folder` first is how an address, and then the + * local part of one, ends up in it. + */ + const order = Object.keys(properties ?? {}); + expect(order[0]).toBe("account"); + expect(order.indexOf("account")).toBeLessThan(order.indexOf("folder")); + } + }); + + test("offers `account` and names the addresses a deployment actually has", async () => { + /* + * The choices are the deployment's, not this file's, so they are added at list time. A model + * told only that an `account` argument exists has to guess at an address, and a guessed address + * is a refusal at best. + */ + recordingMailbox(); + for (const tool of await listTools()) { + const properties = ( + tool.inputSchema as { + properties?: Record; + } + ).properties; + expect(properties?.account).toBeDefined(); + const description = properties?.account?.description ?? ""; + expect(description).toContain("as its email address"); + expect(description).toContain( + "One of: bot@example.test, sales@example.test.", + ); + expect(description).toContain("Default bot@example.test."); + expect(tool.description).toContain( + "bot@example.test, sales@example.test", + ); + expect(tool.description).toContain("`account`"); + } + // Never required: the deployment with one mailbox should not have to think about this at all. + const required = (await listTools()).flatMap( + (tool) => (tool.inputSchema as { required?: string[] }).required ?? [], + ); + expect(required).not.toContain("account"); + }); + + test("says an account can be named even with no mailbox configured", async () => { + // The list is answered either way, so the wording has to work before there are addresses to + // name. See listTools. + useMailbox(null); + for (const tool of await listTools()) { + expect(tool.description).toContain( + "the deployment's configured addresses", + ); + } + }); + + test("needs no mailbox, no credential and no actor", async () => { + // The only call site is `refreshTools`, which passes `{url, token}`. A list that emptied itself + // when a variable was unset would revoke an administrator's grants by accident. + useMailbox(null); + expect(await listTools()).toHaveLength(4); + }); +}); + +describe("a deployment with no mailbox", () => { + test("refuses every tool, naming all four things to set", async () => { + useMailbox(null); + for (const tool of [ + "list_messages", + "read_message", + "search_messages", + "send_message", + ]) { + const result = await callTool(CONNECTION, tool, { uid: 1 }); + expect(result.isError).toBe(true); + expect(result.text).toBe( + "Mailbox is not configured. Set MAILBOX_IMAP_HOST, MAILBOX_SMTP_HOST, MAILBOX_USERS and store each account's password as its mailbox credential.", + ); + } + }); + + test("a configured mailbox with no password in the vault says so separately, naming the account", async () => { + // A different job with a different fix. Telling an administrator to set three variables they can + // see are already set is how a correct instruction reads as a broken deployment. + const { calls } = recordingMailbox({ password: async () => null }); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(result.isError).toBe(true); + expect(result.text).toContain( + `holds no password for the mailbox ${DEFAULT_ACCOUNT}`, + ); + // The key id is the address, so the sentence tells an administrator exactly which row to store. + expect(result.text).toContain(`key id \`${DEFAULT_ACCOUNT}\``); + // Nothing was dialled, so nothing could have been sent to a server unauthenticated. + expect(calls).toEqual([]); + }); + + test("a password missing for one account is that account's problem, not the mailbox's", async () => { + /* + * A deployment that stored `bot@` and forgot `sales@` works for the one it has. The refusal has + * to name the account that is missing a password, or an administrator goes to look at the + * credential that is already there and finds nothing wrong with it. + */ + const { calls } = recordingMailbox({ + password: async (account) => + account === DEFAULT_ACCOUNT ? PASSWORD : null, + }); + + const working = await callTool(CONNECTION, "list_messages", {}); + expect(working.isError).toBe(false); + + const missing = await callTool(CONNECTION, "list_messages", { + account: "sales@example.test", + }); + expect(missing.isError).toBe(true); + expect(missing.text).toContain( + "holds no password for the mailbox sales@example.test", + ); + // Only the account that had one was ever dialled. + expect(calls).toEqual([{ method: "recent", mailbox: "INBOX", limit: 10 }]); + }); +}); + +/** + * Several accounts on one pair of hosts, which is what a shared host gives a deployment. + * + * The three properties worth pinning: the default is the first configured, so a deployment that + * only ever had one mailbox behaves exactly as it did; an account that was named is the one dialled, + * unlocked and answered about; and an account that is not configured is refused before anything + * leaves this process. + */ +describe("several accounts", () => { + test("works in the first configured account when none was named", async () => { + const { built, unlocked } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(unlocked).toEqual([DEFAULT_ACCOUNT]); + expect(built.map((one) => one.account)).toEqual([DEFAULT_ACCOUNT]); + expect(result.text).not.toContain("sales@example.test"); + }); + + test("unlocks and dials the account that was named, and names it back", async () => { + const { built, unlocked } = recordingMailbox({ + recent: async () => ({ headers: [{ ...HEADER }], total: 236 }), + }); + const result = await callTool(CONNECTION, "list_messages", { + account: "sales@example.test", + }); + + // The password is that account's own vault row, so the lookup is per account. + expect(unlocked).toEqual(["sales@example.test"]); + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + // A model reading two accounts in one turn cannot tell two INBOXes apart otherwise. + expect(result.text).toContain( + "[showing 1 of 236 messages in folder INBOX of sales@example.test, newest first.]", + ); + }); + + test("takes the address in whatever case it was written", async () => { + // The configured list is lower-cased and so is the credential key, so a model that shouted the + // address reaches the same mailbox rather than an account that does not exist. + const { built } = recordingMailbox(); + const result = await callTool(CONNECTION, "read_message", { + uid: 42, + account: "Sales@Example.TEST", + }); + + expect(result.isError).toBe(false); + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + expect(result.text).toContain( + "uid 42 in folder INBOX of sales@example.test", + ); + }); + + test("refuses an account this deployment does not have, before anything is dialled", async () => { + /* + * Before the vault and before the network. A model that invented an address should be corrected + * with the list of real ones rather than handed a login failure about a mailbox that does not + * exist, and `send_message` must not reach a mail server on a mistaken argument. + */ + const { calls, built, unlocked } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + account: "billing@example.test", + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("billing@example.test is not one of"); + // The configured addresses, so the next call can be right. + expect(result.text).toContain("bot@example.test, sales@example.test"); + expect(result.text).toContain("nothing was sent"); + expect(calls).toEqual([]); + expect(built).toEqual([]); + expect(unlocked).toEqual([]); + }); + + test("sends from the account it was told to, and says which", async () => { + const { calls, built } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Attached.", + account: "sales@example.test", + }); + + expect(result.isError).toBe(false); + // From is the client's, never an argument: the account decides who the mail is from. + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + expect(result.text).toContain( + "Sent from sales@example.test to dana@example.test", + ); + expect(calls.some((call) => call.method === "send")).toBe(true); + }); + + test("scrubs the account's own password out of that account's failure", async () => { + /* + * Redaction is per selected account: the AUTH=PLAIN blob a server quotes back carries the + * account and its password together, so scrubbing with the default account's name would leave + * another account's credential in the sentence. + */ + const OTHER = "a-different-secret-entirely"; + recordingMailbox({ + password: async (account) => + account === "sales@example.test" ? OTHER : PASSWORD, + recent: async () => { + const blob = Buffer.from( + `\u0000sales@example.test\u0000${OTHER}`, + "utf8", + ).toString("base64"); + throw new MailboxError(`A1 BAD failed: AUTHENTICATE PLAIN ${blob}`); + }, + }); + const result = await callTool(CONNECTION, "list_messages", { + account: "sales@example.test", + }); + + expect(result.isError).toBe(true); + expect(result.text).not.toContain(OTHER); + expect(result.text).not.toContain( + Buffer.from(`\u0000sales@example.test\u0000${OTHER}`, "utf8").toString( + "base64", + ), + ); + expect(result.text).toContain("[redacted]"); + expect(result.text).toContain("BAD failed"); + }); + + test("refuses an address in `folder`, before anything is dialled, pointing at `account`", async () => { + /* + * The mistake as it actually happened: the address in the folder argument. Left to the mail + * server it comes back as "Character not allowed in mailbox name" and then "Mailbox doesn't + * exist: support", which are sentences about IMAP folder naming that no model turns into "use + * the other argument". The refusal has to carry the fix. + */ + const { calls, built, unlocked } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + folder: "support@example.test", + }); + + expect(result.isError).toBe(true); + expect(result.text).toBe( + "support@example.test looks like an email address, and `folder` names an IMAP folder such as INBOX. Pass the address as `account` instead. Nothing was read and nothing was sent.", + ); + expect(calls).toEqual([]); + expect(built).toEqual([]); + expect(unlocked).toEqual([]); + }); + + test("refuses an address in `folder` on the write tool too, sending nothing", async () => { + // An address this deployment does not have, so there is nothing to adopt. A configured one is + // taken as the account instead: see the adoption cases below. + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + folder: "billing@example.test", + in_reply_to: 42, + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("Pass the address as `account` instead"); + expect(result.text).toContain("nothing was sent"); + expect(calls).toEqual([]); + }); + + test("still reads the old `mailbox` key, so a stale tool list is not a silent wrong folder", async () => { + // A Bot holding a tool list from before the rename would otherwise pass a folder that is + // ignored, and read INBOX while believing it read Archive. + const { calls } = recordingMailbox(); + await callTool(CONNECTION, "list_messages", { mailbox: "Archive" }); + expect(calls).toEqual([ + { method: "recent", mailbox: "Archive", limit: 10 }, + ]); + + // And the guard covers it under the old name as well, so it cannot bring the trap back. + const address = await callTool(CONNECTION, "list_messages", { + mailbox: "support@example.test", + }); + expect(address.isError).toBe(true); + expect(address.text).toContain("Pass the address as `account` instead"); + }); + + test("takes a configured address out of `folder` and reads that account's INBOX", async () => { + /* + * The mistake, adopted rather than refused. A model that put a configured address in `folder` + * has said something unambiguous: there is one mailbox it can mean and it named it. Live runs + * show this on the FIRST mailbox call of a turn, so refusing it spends the turn on a vocabulary + * lesson before any work happens. + */ + const { calls, built } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + folder: "Sales@Example.TEST", + }); + + expect(result.isError).toBe(false); + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + // The folder falls back to INBOX, because the argument that named one was spent on the account. + expect(calls).toEqual([{ method: "recent", mailbox: "INBOX", limit: 10 }]); + // The lesson is taught on the way past rather than instead of the work. + expect(result.text.split("\n")[0]).toBe( + "[folder took the address sales@example.test; it was used as account, reading INBOX.]", + ); + expect(result.text).toContain("uid 42"); + }); + + test("adopts the address on the write path too, threading and sending from that account", async () => { + const { calls, built } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Friday it is.", + in_reply_to: 42, + folder: "sales@example.test", + }); + + expect(result.isError).toBe(false); + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + // The original is fetched from the adopted account's INBOX, and the reply goes out from it. + expect(calls[0]).toEqual({ method: "message", mailbox: "INBOX", uid: 42 }); + expect(calls[1]?.method).toBe("send"); + expect(result.text).toContain( + "[folder took the address sales@example.test; it was used as account, reading INBOX.]", + ); + expect(result.text).toContain("Sent from sales@example.test"); + }); + + test("adopting is not overriding: the same address in both arguments is fine", async () => { + const { built } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + folder: "sales@example.test", + account: "sales@example.test", + }); + + expect(result.isError).toBe(false); + expect(built.map((one) => one.account)).toEqual(["sales@example.test"]); + }); + + test("refuses two arguments naming two different mailboxes, naming both", async () => { + /* + * Nothing to adopt: a model that named one mailbox in `folder` and another in `account` has + * lost track of which it is reading, and picking either would be picking for it. + */ + const { calls, built, unlocked } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + folder: "sales@example.test", + account: "bot@example.test", + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("folder is sales@example.test"); + expect(result.text).toContain("account is bot@example.test"); + expect(result.text).toContain("Nothing was read and nothing was sent."); + expect(calls).toEqual([]); + expect(built).toEqual([]); + expect(unlocked).toEqual([]); + }); + + test("refuses an account's local part in `folder`, before anything is dialled", async () => { + /* + * The retry after the @ guard fired, as it actually happened: refused `support@example.test`, + * the model tried `support`, then `webmaster`, which are the parts before the @ of configured + * accounts. The mail server answers "Mailbox doesn't exist: support", which reads as a folder + * that happens to be missing rather than as an argument that is wrong. + */ + const { calls, built, unlocked } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + folder: "Sales", + }); + + expect(result.isError).toBe(true); + expect(result.text).toBe( + "Sales is the account sales@example.test, not a folder. Pass account=sales@example.test and leave folder unset to read INBOX. Nothing was read and nothing was sent.", + ); + expect(calls).toEqual([]); + expect(built).toEqual([]); + expect(unlocked).toEqual([]); + }); + + test("resolves the two arguments together, on its own", () => { + const users = ["bot@example.test", "sales@example.test"]; + expect(selectMailbox({}, users)).toEqual({ + account: "bot@example.test", + folder: "INBOX", + }); + expect(selectMailbox({ folder: "Archive" }, users)).toEqual({ + account: "bot@example.test", + folder: "Archive", + }); + expect(selectMailbox({ folder: " SALES@example.test " }, users)).toEqual({ + account: "sales@example.test", + folder: "INBOX", + note: "[folder took the address sales@example.test; it was used as account, reading INBOX.]", + }); + // An address this deployment does not have has nothing to adopt, so the refusal stands. + expect( + selectMailbox({ folder: "billing@example.test" }, users).error, + ).toContain("looks like an email address"); + // A folder named like an account's local part is still refused: `support` is not an address, + // and a folder genuinely called that can exist. + expect(selectMailbox({ folder: "sales" }, users).error).toContain( + "is the account sales@example.test, not a folder", + ); + }); + + test("picks the folder out of the arguments, on its own", () => { + const users = ["bot@example.test", "sales@example.test"]; + expect(selectFolder({}, users)).toEqual({ folder: "INBOX" }); + expect(selectFolder({ folder: " " }, users)).toEqual({ folder: "INBOX" }); + expect(selectFolder({ folder: "Archive" }, users)).toEqual({ + folder: "Archive", + }); + expect(selectFolder({ mailbox: "Sent" }, users)).toEqual({ + folder: "Sent", + }); + expect(selectFolder({ folder: "a@b.test" }, users).error).toContain( + "looks like an email address", + ); + // Case-insensitively and after trimming, which is how a model writes it. + expect(selectFolder({ folder: " BOT " }, users).error).toContain( + "is the account bot@example.test, not a folder", + ); + // A folder that merely starts the same way is still a folder. + expect(selectFolder({ folder: "bot-archive" }, users)).toEqual({ + folder: "bot-archive", + }); + }); + + test("picks the account out of the arguments, on its own", () => { + const users = ["bot@example.test", "sales@example.test"]; + expect(selectAccount({}, users)).toEqual({ account: "bot@example.test" }); + expect(selectAccount({ account: " " }, users)).toEqual({ + account: "bot@example.test", + }); + expect(selectAccount({ account: "SALES@example.test" }, users)).toEqual({ + account: "sales@example.test", + }); + expect( + selectAccount({ account: "nobody@example.test" }, users).error, + ).toContain("not one of this deployment's mailbox accounts"); + }); +}); + +describe("listing", () => { + test("dials the configured mailbox with the vault's password", async () => { + const { built } = recordingMailbox(); + await callTool(CONNECTION, "list_messages", {}); + expect(built).toEqual([ + { config: CONFIG, account: DEFAULT_ACCOUNT, password: PASSWORD }, + ]); + }); + + test("defaults to ten of INBOX", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(result.isError).toBe(false); + expect(calls).toEqual([{ method: "recent", mailbox: "INBOX", limit: 10 }]); + expect(result.text).toContain("uid 42"); + expect(result.text).toContain("Dana Reid"); + expect(result.text).toContain("The Friday numbers"); + expect(result.text).toContain("unread"); + }); + + test("honours a mailbox and a limit that were asked for", async () => { + const { calls } = recordingMailbox(); + await callTool(CONNECTION, "list_messages", { + folder: "Archive", + limit: 3, + }); + expect(calls).toEqual([{ method: "recent", mailbox: "Archive", limit: 3 }]); + }); + + test("caps a huge ask at fifty and says it did", async () => { + // Capped rather than refused: "give me 200" is a clear intention this tool does not serve, and a + // refusal would cost a round trip to be told a smaller number. Said, so a model that asked for + // 200 and got 50 does not conclude the mailbox holds fifty messages. + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { limit: 200 }); + + expect(calls).toEqual([{ method: "recent", mailbox: "INBOX", limit: 50 }]); + expect(result.text).toContain("50 is the most this tool lists at once"); + }); + + test("says how many there were when the mailbox holds more than the page", async () => { + /* + * A page of ten with nothing else on it reads to a model as the whole mailbox, and it will + * answer "you have ten messages" about a mailbox holding four thousand. The count is free (it + * is the mailbox's own `exists`) and it is the difference between a listing and a claim. + */ + recordingMailbox({ + recent: async (_mailbox, limit) => ({ + headers: Array.from({ length: limit }, (_, index) => ({ + ...HEADER, + uid: index + 1, + })), + total: 4_321, + }), + }); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(result.text).toContain( + `showing 10 of 4321 messages in folder INBOX of ${DEFAULT_ACCOUNT}, newest first.`, + ); + // Nothing was capped, so the tool's own limit is not mentioned as well. + expect(result.text).not.toContain("most this tool lists"); + }); + + test("refuses a limit that is not a whole number, rather than rounding it", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "list_messages", { + limit: "ten", + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("`limit` has to be a whole number"); + expect(calls).toEqual([]); + }); + + test("an empty mailbox is said in words, not answered with nothing", async () => { + // An empty string reads to a model as "the tool had nothing to say" and gets filled in from + // memory, which for a mailbox means inventing mail. + recordingMailbox({ recent: async () => ({ headers: [], total: 0 }) }); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(result.isError).toBe(false); + expect(result.text).toBe( + `There are no messages in folder INBOX of ${DEFAULT_ACCOUNT}.`, + ); + }); + + test("bounds the whole answer the way every other connector's is", async () => { + const many: MessageHeader[] = Array.from({ length: 50 }, (_, index) => ({ + ...HEADER, + uid: index + 1, + subject: "x".repeat(1_000), + })); + recordingMailbox({ + recent: async () => ({ headers: many, total: many.length }), + }); + const result = await callTool(CONNECTION, "list_messages", { limit: 50 }); + + expect(result.truncated).toBe(true); + expect(result.text).toContain("[truncated:"); + expect(result.text.length).toBeLessThan(MAX_RESULT_CHARS + 200); + }); +}); + +describe("reading one message", () => { + test("reaches the mailbox the uid came from", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "read_message", { + uid: 42, + folder: "Archive", + }); + + expect(result.isError).toBe(false); + expect(calls).toEqual([{ method: "message", mailbox: "Archive", uid: 42 }]); + expect(result.text).toContain("From: Dana Reid "); + expect(result.text).toContain("Can you send the Friday numbers?"); + }); + + test("accepts a uid a model wrote as a string", async () => { + const { calls } = recordingMailbox(); + await callTool(CONNECTION, "read_message", { uid: "42" }); + expect(calls).toEqual([{ method: "message", mailbox: "INBOX", uid: 42 }]); + }); + + test("refuses a missing uid and a fractional one without dialling", async () => { + const { calls } = recordingMailbox(); + const missing = await callTool(CONNECTION, "read_message", {}); + expect(missing.isError).toBe(true); + expect(missing.text).toContain("Say which message to read"); + + const fractional = await callTool(CONNECTION, "read_message", { + uid: 12.7, + }); + expect(fractional.isError).toBe(true); + expect(fractional.text).toContain("whole number"); + expect(calls).toEqual([]); + }); + + test("a uid that is not there is named as such, with why", async () => { + recordingMailbox({ message: async () => null }); + const result = await callTool(CONNECTION, "read_message", { uid: 7 }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("no message with uid 7 in folder INBOX"); + expect(result.text).toContain("per folder"); + }); + + test("a long body is cut and says how long it was", async () => { + recordingMailbox({ + message: async () => ({ + ...MESSAGE, + body: "y".repeat(MAX_BODY_CHARS), + bodyLength: 41_000, + }), + }); + const result = await callTool(CONNECTION, "read_message", { uid: 42 }); + + expect(result.isError).toBe(false); + // The length is stated, because a message cut at 8000 of 9000 has almost certainly said what it + // came to say and one cut at 8000 of 41000 has not. + expect(result.text).toContain("[truncated: the message body is 41000"); + expect(result.text).toContain(String(MAX_BODY_CHARS)); + }); + + test("a message read only in part says the rest was never seen", async () => { + /* + * A different fact from a long body. A cut body means the deployment holds the whole message + * and is showing part of it; a cut source means it never read the rest off the wire, so an + * answer offering to go back for the attachment would be offering something impossible. + */ + recordingMailbox({ + message: async () => ({ + ...MESSAGE, + sourceTruncated: true, + sizeBytes: 4_000_000, + }), + }); + const result = await callTool(CONNECTION, "read_message", { uid: 42 }); + + expect(result.text).toContain(`only the first ${MAX_SOURCE_BYTES} bytes`); + expect(result.text).toContain("4000000 bytes"); + expect(result.text).toContain("attachments"); + }); + + test("a uid too large for IMAP is answered as a message that is not there", async () => { + // Uids are 32-bit (RFC 3501). Passed through, `1e21` reaches imapflow and comes back as + // "Invalid sequence set value", which is a sentence about a data structure rather than about + // the message a model thinks it asked for. + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "read_message", { + uid: 4_294_967_296, + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain( + "no message with uid 4294967296 in folder INBOX", + ); + expect(calls).toEqual([]); + }); + + test("a message with no readable text says that rather than nothing", async () => { + recordingMailbox({ + message: async () => ({ ...MESSAGE, body: "", bodyLength: 0 }), + }); + const result = await callTool(CONNECTION, "read_message", { uid: 42 }); + expect(result.text).toContain("no readable text"); + }); +}); + +describe("searching", () => { + test("passes the query through and defaults to twenty", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "search_messages", { + query: "invoice", + }); + + expect(result.isError).toBe(false); + expect(calls).toEqual([ + { method: "search", mailbox: "INBOX", query: "invoice", limit: 20 }, + ]); + }); + + test("refuses an empty query without dialling", async () => { + const { calls } = recordingMailbox(); + const blank = await callTool(CONNECTION, "search_messages", { query: " " }); + expect(blank.isError).toBe(true); + expect(blank.text).toContain("Say what to search"); + expect(calls).toEqual([]); + }); + + test("says how many matched when it is showing fewer", async () => { + recordingMailbox({ + search: async (_mailbox, _query, limit) => ({ + headers: Array.from({ length: limit }, (_, index) => ({ + ...HEADER, + uid: index + 1, + })), + total: 300, + }), + }); + const result = await callTool(CONNECTION, "search_messages", { + query: "invoice", + }); + + // A search that found 300 and is answering with 20 has to say so, or the model reports 20. + expect(result.text).toContain("showing 20 of 300 matches"); + }); + + test("nothing found is stated, with nothing to answer from", async () => { + recordingMailbox({ search: async () => ({ headers: [], total: 0 }) }); + const result = await callTool(CONNECTION, "search_messages", { + query: "invoice", + }); + + expect(result.isError).toBe(false); + expect(result.text).toContain( + `Nothing in folder INBOX of ${DEFAULT_ACCOUNT} matches "invoice"`, + ); + expect(result.text).toContain("nothing here to answer from"); + }); +}); + +describe("sending", () => { + test("sends what it was given and confirms it", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Attached.", + }); + + expect(result.isError).toBe(false); + expect(calls).toEqual([ + { + method: "send", + message: { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Attached.", + }, + }, + ]); + expect(result.text).toContain( + `Sent from ${DEFAULT_ACCOUNT} to dana@example.test`, + ); + expect(result.text).toContain("starts a new thread"); + // Where to verify it. A Bot that cannot find its own sent mail concludes it was never sent. + expect(result.text).toContain("A copy is in folder Sent."); + }); + + test("refuses a message with no recipient, no subject or no body, sending nothing", async () => { + const { calls } = recordingMailbox(); + for (const args of [ + { subject: "s", body: "b" }, + { to: "dana@example.test", body: "b" }, + { to: "dana@example.test", subject: "s" }, + ]) { + const result = await callTool(CONNECTION, "send_message", args); + expect(result.isError).toBe(true); + } + expect(calls).toEqual([]); + }); + + test("a reply threads on the original's own ids and gains a Re:", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Friday it is.", + in_reply_to: 42, + }); + + expect(result.isError).toBe(false); + // Fetched first, then sent: the ids come from a message that exists rather than from a model. + expect(calls[0]).toEqual({ method: "message", mailbox: "INBOX", uid: 42 }); + expect(calls[1]).toEqual({ + method: "send", + message: { + to: "dana@example.test", + subject: "Re: The Friday numbers", + body: "Friday it is.", + reply: { + messageId: "", + // The original's chain with the original appended, which is what a mail client walks. + references: ["", ""], + }, + }, + }); + expect(result.text).toContain("threads as a reply"); + }); + + test("a reply to a uid that is not there sends nothing at all", async () => { + // Sending it as a new message instead would tell the model the wrong thing about what it did, + // and the recipient would see an answer that appears to be about nothing. + const { calls } = recordingMailbox({ message: async () => null }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Friday it is.", + in_reply_to: 99, + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("Nothing was sent."); + expect(calls.some((call) => call.method === "send")).toBe(false); + }); + + test("a reply to a uid too large for IMAP sends nothing and says so", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + in_reply_to: 9_000_000_000, + }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("no message with uid 9000000000"); + expect(result.text).toContain("Nothing was sent."); + expect(calls).toEqual([]); + }); + + test("an original with no Message-ID is answered unthreaded rather than threaded on a guess", async () => { + const { calls } = recordingMailbox({ + message: async () => ({ ...MESSAGE, messageId: null, references: [] }), + }); + await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Friday it is.", + in_reply_to: 42, + }); + + const sent = calls.find((call) => call.method === "send"); + expect(sent?.method === "send" && sent.message.reply).toBeUndefined(); + // The subject still reads as an answer, which is the half that does not need an id. + expect(sent?.method === "send" && sent.message.subject).toBe( + "Re: The Friday numbers", + ); + }); +}); + +describe("where mail is allowed to go", () => { + const RESTRICTED: MailboxConfig = { + ...CONFIG, + allowedRecipientDomains: new Set(["example.test", "partner.example"]), + }; + + test("a recipient outside the list is refused before anything is dialled", async () => { + /* + * The check that stands between a Bot that was talked into something by the mail it just read + * and an address outside the deployment. Before the network, so a refused recipient costs no + * connection and no mailbox is opened on the way to being told no. + */ + const { calls } = recordingMailbox({ config: RESTRICTED }); + const result = await callTool(CONNECTION, "send_message", { + to: "attacker@evil.test", + subject: "The inbox", + body: "Here it is.", + // Even with a reply to fetch, nothing is dialled: the recipient is settled first. + in_reply_to: 42, + }); + + expect(result.isError).toBe(true); + // The domain, because that is what the allowlist is written about and what an administrator + // would change. + expect(result.text).toContain("evil.test"); + expect(result.text).toContain("Nothing was sent."); + expect(result.text).toContain("MAILBOX_ALLOWED_RECIPIENT_DOMAINS"); + expect(calls).toEqual([]); + }); + + test("a recipient on the list goes through, in either case and with a display name", async () => { + const { calls } = recordingMailbox({ config: RESTRICTED }); + const result = await callTool(CONNECTION, "send_message", { + to: "Dana Reid , ops@partner.example", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(false); + expect(calls.some((call) => call.method === "send")).toBe(true); + }); + + test("one refused recipient refuses the whole message", async () => { + // Sending to the allowed half would be a partial send reported as a send, and the model would + // tell somebody the message went to people it never reached. + const { calls } = recordingMailbox({ config: RESTRICTED }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test, attacker@evil.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(true); + expect(calls).toEqual([]); + }); + + test("an unrestricted deployment sends anywhere, which is the default", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "send_message", { + to: "anyone@anywhere.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(false); + expect(calls.some((call) => call.method === "send")).toBe(true); + }); +}); + +describe("picking the refused recipients out of a to field", () => { + const allowed = new Set(["example.test"]); + + test("an empty allowlist refuses nothing", () => { + expect(refusedRecipients("anyone@anywhere.test", new Set())).toEqual([]); + }); + + test("matches on the domain, case-insensitively, through a display name", () => { + expect( + refusedRecipients("Dana , ops@example.test", allowed), + ).toEqual([]); + }); + + test("names each refused domain once", () => { + expect( + refusedRecipients("a@evil.test, b@evil.test, c@other.test", allowed), + ).toEqual(["evil.test", "other.test"]); + }); + + test("something that is not an address at all is refused rather than passed on", () => { + // An unparseable recipient is not one this list has cleared, and letting it through to be + // somebody else's validation error would be a hole in a safety check. + expect(refusedRecipients("not-an-address", allowed)).toEqual([ + "not-an-address", + ]); + expect(refusedRecipients("trailing@", allowed)).toEqual(["trailing@"]); + }); +}); + +describe("a folder that is not there", () => { + test("answers with the folders that are, and says the account is not chosen this way", () => { + /* + * "Mailbox doesn't exist: support" reads as a folder that happens to be missing, so a model + * retries with another folder name. The folders that do exist, plus one sentence about which + * argument picks the account, turn a loop into a correction. + */ + expect( + noSuchFolderSentence("support", "bot@example.test", [ + "INBOX", + "Sent", + "Archive", + ]), + ).toBe( + "No folder named support in bot@example.test. Folders here: INBOX, Sent, Archive. The account is chosen by `account`, not by folder; leave folder unset for INBOX.", + ); + }); + + test("says nothing about folders when the server would not list them", () => { + // A LIST that failed leaves the correction without its examples. Still better than the vendor's + // own sentence, and never a reason to lose the refusal. + const sentence = noSuchFolderSentence("support", "bot@example.test", []); + expect(sentence).not.toContain("Folders here"); + expect(sentence).toContain("The account is chosen by `account`"); + }); + + test("keeps the instruction when there are more folders than fit", () => { + /* + * Trimmed from the end of the LIST rather than the end of the sentence: the closing instruction + * is the half that changes what the model does next, and a failure is capped at 400 characters + * by the transport above this. + */ + const many = Array.from( + { length: MAX_FOLDERS_LISTED + 20 }, + (_, index) => `Folder-Number-${index}`, + ); + const sentence = noSuchFolderSentence("support", "bot@example.test", many); + + expect(sentence.length).toBeLessThanOrEqual(400); + expect(sentence).toContain("Folders here: Folder-Number-0,"); + expect(sentence).toContain(", and more."); + expect(sentence).toContain( + "The account is chosen by `account`, not by folder; leave folder unset for INBOX.", + ); + }); + + test("reaches the model whole, rather than cut by the failure cap", async () => { + const sentence = noSuchFolderSentence("support", "bot@example.test", [ + "INBOX", + "Sent", + "Archive", + "Drafts", + "Junk", + ]); + recordingMailbox({ + recent: async () => { + throw new MailboxError(sentence); + }, + }); + const result = await callTool(CONNECTION, "list_messages", { + folder: "Newsletters", + }); + + expect(result.isError).toBe(true); + expect(result.text).toBe(sentence); + }); +}); + +describe("the reply headers themselves", () => { + test("append the original to its own chain", () => { + expect( + replyFrom( + { messageId: "", references: [""] }, + "The Friday numbers", + ), + ).toEqual({ + subject: "Re: The Friday numbers", + reply: { messageId: "", references: ["", ""] }, + }); + }); + + test("do not stack Re: on a subject that already has one, in any case", () => { + for (const subject of ["Re: numbers", "RE: numbers", "re: numbers"]) { + expect( + replyFrom({ messageId: "", references: [] }, subject).subject, + ).toBe(subject); + } + }); + + test("recognise the reply marker other mail clients write", () => { + // A client that only knew "Re:" is how a thread ends up titled "Re: AW: Re: AW: the numbers". + for (const subject of [ + "AW: numbers", + "SV:numbers", + "Antw: numbers", + "Ref : numbers", + ]) { + expect( + replyFrom({ messageId: "", references: [] }, subject).subject, + ).toBe(subject); + } + }); + + test("still prefix a subject that merely starts with those letters", () => { + // "Reference pricing" is not a reply, and neither is "Software renewal". + expect( + replyFrom({ messageId: "", references: [] }, "Reference pricing") + .subject, + ).toBe("Re: Reference pricing"); + }); + + test("are absent entirely when the original carries no Message-ID", () => { + // An In-Reply-To pointing at nothing is not a reply, and an invented id threads the answer into + // a conversation that does not exist. + expect(replyFrom({ messageId: null, references: [""] }, "n")).toEqual({ + subject: "Re: n", + }); + }); +}); + +describe("what a model is told when the mail server fails", () => { + test("the server's own sentence survives, so the fix is nameable", async () => { + recordingMailbox({ + recent: async () => { + throw new MailboxError("Mailbox does not exist"); + }, + }); + const result = await callTool(CONNECTION, "list_messages", {}); + + expect(result.isError).toBe(true); + expect(result.text).toBe("Mailbox does not exist"); + }); + + test("the password never survives, even when the server echoes it back", async () => { + /* + * Some IMAP and SMTP servers quote the offending command on a failed login, and that command is + * `LOGIN user password`. That sentence goes into an audit row and in front of a model, and + * neither is a place for this deployment's mailbox password. + */ + recordingMailbox({ + send: async () => { + throw new MailboxError( + `Invalid credentials for LOGIN bot@example.test ${PASSWORD}`, + ); + }, + }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(true); + expect(result.text).not.toContain(PASSWORD); + expect(result.text).toContain("[redacted]"); + expect(result.text).toContain("Invalid credentials"); + }); + + test("a failure is capped, because a failure is not a promise about length", async () => { + recordingMailbox({ + recent: async () => { + throw new Error("z".repeat(5_000)); + }, + }); + const result = await callTool(CONNECTION, "list_messages", {}); + expect(result.text.length).toBe(400); + }); + + test("a tool this transport does not implement is refused by name", async () => { + const { calls } = recordingMailbox(); + const result = await callTool(CONNECTION, "delete_message", { uid: 1 }); + + expect(result.isError).toBe(true); + expect(result.text).toContain("delete_message is not a tool Mailbox"); + expect(calls).toEqual([]); + }); +}); + +describe("redaction, on its own", () => { + test("replaces every occurrence", () => { + expect(redacted(`a ${PASSWORD} b ${PASSWORD}`, PASSWORD)).toBe( + "a [redacted] b [redacted]", + ); + }); + + test("catches the base64 forms, which are what the wire actually carries", () => { + /* + * Neither client prefers plaintext LOGIN: imapflow authenticates with AUTH=PLAIN when the + * server offers it and falls back to AUTH=LOGIN, and both put the credential on the wire + * base64-encoded. A redaction that only knew the plaintext would pass the secret straight + * through while looking like it had worked. + */ + const login = Buffer.from(PASSWORD, "utf8").toString("base64"); + const plain = Buffer.from( + `\u0000${DEFAULT_ACCOUNT}\u0000${PASSWORD}`, + "utf8", + ).toString("base64"); + + const message = `A1 BAD failed: A1 AUTHENTICATE PLAIN ${plain} / LOGIN ${login}`; + const scrubbed = redacted(message, PASSWORD, DEFAULT_ACCOUNT); + + expect(scrubbed).not.toContain(login); + expect(scrubbed).not.toContain(plain); + expect(scrubbed).not.toContain(PASSWORD); + expect(scrubbed).toContain("[redacted]"); + // The rest of the server's sentence survives, which is the whole reason it is kept at all. + expect(scrubbed).toContain("BAD failed"); + }); + + test("leaves a message alone when there is no password, or a trivial one", () => { + // Replacing a one-character secret would redact half the alphabet out of an unrelated sentence. + expect(redacted("no such mailbox", null)).toBe("no such mailbox"); + expect(redacted("no such mailbox", "x")).toBe("no such mailbox"); + }); +}); + +describe("the limit bound, on its own", () => { + test("falls back, passes through, and caps", () => { + const bounds = { fallback: 10, max: 50 }; + expect(boundedLimit(undefined, bounds)).toEqual({ + limit: 10, + capped: false, + }); + expect(boundedLimit(3, bounds)).toEqual({ limit: 3, capped: false }); + expect(boundedLimit(50, bounds)).toEqual({ limit: 50, capped: false }); + expect(boundedLimit(200, bounds)).toEqual({ limit: 50, capped: true }); + }); +}); + +describe("reading a body out of a parsed message", () => { + test("prefers the text part, whatever length it is", () => { + // A plain-text alternative saying "this message needs an HTML viewer" is still what the sender + // wrote. Preferring the HTML on length would swap the sender's own words out for markup. + expect( + readBody({ text: "short", html: "

much longer body here

" }), + ).toEqual({ body: "short", bodyLength: 5 }); + }); + + test("falls back to stripped HTML only when there is no text part", () => { + expect( + readBody({ html: "

Hello

there

" }), + ).toEqual({ body: "Hello\nthere", bodyLength: 11 }); + }); + + test("caps the body and reports the length it had", () => { + const long = "y".repeat(MAX_BODY_CHARS + 500); + const { body, bodyLength } = readBody({ text: long }); + expect(body).toHaveLength(MAX_BODY_CHARS); + expect(bodyLength).toBe(MAX_BODY_CHARS + 500); + }); + + test("a message with neither part is empty rather than undefined", () => { + expect(readBody({})).toEqual({ body: "", bodyLength: 0 }); + expect(readBody({ html: false })).toEqual({ body: "", bodyLength: 0 }); + }); +}); + +describe("stripping HTML", () => { + test("drops scripts and styles rather than reading them as prose", () => { + expect(strippedHtml("

Hi

")).toBe("Hi"); + }); + + test("keeps line structure and decodes the entities that matter", () => { + expect(strippedHtml("

a & b

c
d

")).toBe("a & b\nc\nd"); + }); +}); + +describe("the sentence a mail server actually wrote", () => { + test("prefers responseText, which is where imapflow puts it", () => { + /* + * imapflow answers every IMAP NO and BAD with an Error whose message is the fixed string + * "Command failed" and puts the server's own words on `responseText`. Reading `message` turns + * "Invalid credentials", "Mailbox does not exist" and "Over quota" into one useless sentence + * that names none of them, which is the opposite of why the vendor's wording is kept at all. + */ + const error = Object.assign(new Error("Command failed"), { + responseText: "Invalid credentials (Failure)", + responseStatus: "NO", + response: { command: "NO" }, + }); + expect(mailServerSentence(error)).toBe("Invalid credentials (Failure)"); + }); + + test("falls back to a string response, which is where nodemailer puts the SMTP reply", () => { + const error = Object.assign(new Error("Invalid login"), { + response: "535 5.7.8 Authentication credentials invalid", + }); + expect(mailServerSentence(error)).toBe( + "535 5.7.8 Authentication credentials invalid", + ); + }); + + test("never renders a response object, and falls back to the message", () => { + // `String({})` is "[object Object]", which is worse than the generic message it replaced. + const error = Object.assign(new Error("Command failed"), { + response: { command: "BAD" }, + }); + expect(mailServerSentence(error)).toBe("Command failed"); + expect(mailServerSentence(new Error("socket hang up"))).toBe( + "socket hang up", + ); + expect(mailServerSentence("plain string")).toBe("plain string"); + }); +}); + +describe("the wall clock every network operation runs against", () => { + test("ends a call that never finishes, and closes the socket on the way out", async () => { + /* + * The three timeouts imapflow is built with are INACTIVITY timeouts, so a server sending one + * byte every twenty seconds resets all of them forever and the turn never ends. The cleanup is + * the point rather than the rejection: without it the losing work keeps an authenticated + * connection this deployment has stopped waiting for. + */ + let closed = 0; + const expired = withDeadline( + () => new Promise(() => {}), + () => { + closed += 1; + }, + "reading the mailbox", + 5, + ); + + await expect(expired).rejects.toBeInstanceOf(MailboxError); + await expect(expired).rejects.toThrow("did not finish reading the mailbox"); + expect(closed).toBe(1); + }); + + test("leaves a call that finishes in time alone", async () => { + let closed = 0; + const answer = await withDeadline( + async () => "done", + () => { + closed += 1; + }, + "sending the message", + 50, + ); + + expect(answer).toBe("done"); + // Nothing to close, and the timer is cleared rather than left holding the process open. + expect(closed).toBe(0); + }); +}); + +/** + * The copy in Sent, which is the half SMTP does not do. + * + * SMTP delivers; it does not file. Without the append the account's Sent folder stays empty, + * webmail shows nothing sent, and a person checking concludes the mail never went. That happened + * here, to three messages that had all been delivered, which is why the bytes are built once and + * used twice and why a failure to file is never reported as a failure to send. + * + * These cases reach the real client rather than the stub above, through the one seam that exists + * for it: the property worth being sure about is that the bytes handed to SMTP and the bytes + * appended to IMAP are the same bytes, and no stub one level up can see both. + */ +describe("filing a copy in Sent", () => { + type Delivered = { envelope: unknown; raw: unknown }; + type Appended = { path: string; flags: unknown; raw: unknown; date: unknown }; + + /** + * The real clients, wired to a recording IMAP and a recording SMTP. + * + * `folders` is what the server would answer LIST with, and `appendFails` is the server that + * refuses the append after the mail has already gone out. + */ + function wired( + options: { + folders?: { path: string; name?: string; specialUse?: string }[]; + appendFails?: string; + } = {}, + ) { + const delivered: Delivered[] = []; + const appended: Appended[] = []; + const listed = options.folders ?? [ + { path: "INBOX", name: "INBOX" }, + { path: "Sent", name: "Sent" }, + ]; + + const clients = createMailboxClients(CONFIG, DEFAULT_ACCOUNT, PASSWORD, { + imap: () => + ({ + connect: async () => {}, + logout: async () => {}, + close: () => {}, + list: async () => listed, + append: async ( + path: string, + raw: unknown, + flags: unknown, + date: unknown, + ) => { + if (options.appendFails) throw new Error(options.appendFails); + appended.push({ path, raw, flags, date }); + return { destination: path } as never; + }, + }) as unknown as ImapLike, + smtp: () => + ({ + sendMail: async (mail: { envelope?: unknown; raw?: unknown }) => { + delivered.push({ envelope: mail.envelope, raw: mail.raw }); + return {} as never; + }, + close: () => {}, + }) as unknown as SmtpLike, + }); + + return { clients, delivered, appended }; + } + + const MAIL = { + to: "dana@example.test", + subject: "The Friday numbers", + body: "Attached.", + }; + + test("delivers and files the very same bytes", async () => { + /* + * Composed twice would be two messages: different Message-ID, different Date, different + * boundaries. A person reading Sent would be looking at a near-copy of what the recipient got, + * and a Bot verifying its own send by Message-ID would find nothing. + */ + const { clients, delivered, appended } = wired(); + const receipt = await clients.send(MAIL); + + expect(delivered).toHaveLength(1); + expect(appended).toHaveLength(1); + expect(Buffer.isBuffer(delivered[0]?.raw)).toBe(true); + expect(appended[0]?.raw).toBe(delivered[0]?.raw); + + const raw = String(delivered[0]?.raw); + expect(raw).toContain(`From: ${DEFAULT_ACCOUNT}`); + expect(raw).toContain("To: dana@example.test"); + expect(raw).toContain("Subject: The Friday numbers"); + expect(raw).toContain("Attached."); + // The id in the receipt is the id in the bytes, so it can be looked up in Sent afterwards. + expect(receipt.messageId).not.toBeNull(); + expect(raw).toContain(String(receipt.messageId)); + + // The envelope goes with it, so delivery uses the addresses the headers name. + expect(delivered[0]?.envelope).toMatchObject({ + from: DEFAULT_ACCOUNT, + to: ["dana@example.test"], + }); + + expect(appended[0]?.path).toBe("Sent"); + // Seen, because the account wrote this message rather than receiving it. + expect(appended[0]?.flags).toEqual(["\\Seen"]); + expect(appended[0]?.date).toBeInstanceOf(Date); + expect(receipt.filedTo).toBe("Sent"); + expect(receipt.fileError).toBeNull(); + }); + + test("finds Sent by its special-use flag, whatever it is called", async () => { + // A French account's Sent folder is "Éléments envoyés", and no list of English names finds it. + const { clients, appended } = wired({ + folders: [ + { path: "INBOX", name: "INBOX" }, + { + path: "Éléments envoyés", + name: "Éléments envoyés", + specialUse: "\\Sent", + }, + ], + }); + const receipt = await clients.send(MAIL); + + expect(appended[0]?.path).toBe("Éléments envoyés"); + expect(receipt.filedTo).toBe("Éléments envoyés"); + }); + + test("falls back to the name when the server marks nothing", async () => { + const { clients, appended } = wired({ + folders: [ + { path: "INBOX", name: "INBOX" }, + { path: "Sent Items", name: "Sent Items" }, + ], + }); + const receipt = await clients.send(MAIL); + + expect(appended[0]?.path).toBe("Sent Items"); + expect(receipt.filedTo).toBe("Sent Items"); + }); + + test("an account with no Sent folder still sends, and says why there is no copy", async () => { + const { clients, delivered, appended } = wired({ + folders: [{ path: "INBOX", name: "INBOX" }], + }); + const receipt = await clients.send(MAIL); + + expect(delivered).toHaveLength(1); + expect(appended).toEqual([]); + expect(receipt.filedTo).toBeNull(); + expect(receipt.fileError).toBe("this account has no Sent folder"); + }); + + test("an append that fails is not a send that failed", async () => { + /* + * The expensive mistake this prevents: reported as a failure, a Bot sends the message again, + * and mail cannot be recalled. Delivery already happened by the time the append is attempted. + */ + const { clients, delivered } = wired({ + appendFails: "Over quota", + }); + const receipt = await clients.send(MAIL); + + expect(delivered).toHaveLength(1); + expect(receipt.messageId).not.toBeNull(); + expect(receipt.filedTo).toBeNull(); + expect(receipt.fileError).toContain("Over quota"); + }); + + test("picks the folder out of a LIST, on its own", () => { + expect( + sentFolderFrom([ + { path: "INBOX", name: "INBOX" }, + { path: "Archive", name: "Archive" }, + { path: "S", name: "S", specialUse: "\\Sent" }, + ]), + ).toBe("S"); + expect( + sentFolderFrom([ + { path: "INBOX", name: "INBOX" }, + { path: "INBOX.Sent Messages", name: "Sent Messages" }, + ]), + ).toBe("INBOX.Sent Messages"); + // Null rather than a guess: appending to the wrong folder puts outgoing mail where a person + // reads it as incoming. + expect(sentFolderFrom([{ path: "INBOX", name: "INBOX" }])).toBeNull(); + }); + + test("writes both threading headers into the bytes, once", async () => { + const composed = await composeMessage(DEFAULT_ACCOUNT, { + ...MAIL, + reply: { + messageId: "", + references: ["", ""], + }, + }); + const raw = String(composed.raw); + + expect(raw).toContain("In-Reply-To: "); + expect(raw).toContain(""); + expect(raw).toContain(`Message-ID: ${composed.messageId}`); + }); +}); + +describe("what a model is told about the copy", () => { + test("the confirmation names the folder, so a Bot can verify its own send", async () => { + recordingMailbox({ + send: async () => ({ + messageId: "", + filedTo: "Sent Items", + fileError: null, + }), + }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(false); + expect(result.text).toContain("A copy is in folder Sent Items."); + }); + + test("a copy that could not be filed is a note on a successful send, not an error", async () => { + // A model told the send failed sends it again. Mail cannot be recalled. + recordingMailbox({ + send: async () => ({ + messageId: "", + filedTo: null, + fileError: "Over quota", + }), + }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(false); + expect(result.text).toContain("Sent from"); + expect(result.text).toContain("no copy could be filed in the Sent folder"); + expect(result.text).toContain("Over quota"); + expect(result.text).toContain("do not send it again"); + }); + + test("a mail server that quoted the password into the filing failure is scrubbed too", async () => { + // The filing failure is a sentence a server wrote, on an authenticated connection, and it + // reaches a model and an audit row like any other. + recordingMailbox({ + send: async () => ({ + messageId: null, + filedTo: null, + fileError: `NO [AUTHENTICATIONFAILED] LOGIN ${PASSWORD}`, + }), + }); + const result = await callTool(CONNECTION, "send_message", { + to: "dana@example.test", + subject: "s", + body: "b", + }); + + expect(result.isError).toBe(false); + expect(result.text).not.toContain(PASSWORD); + expect(result.text).toContain("[redacted]"); + }); +}); diff --git a/server/tests/config.test.ts b/server/tests/config.test.ts index 5b1d9ce2..4d95fa6a 100644 --- a/server/tests/config.test.ts +++ b/server/tests/config.test.ts @@ -726,3 +726,178 @@ describe("how far a Bot may hand work on", () => { ).toThrow("BOT_HANDOFF_MAX_PER_RUN"); }); }); + +/** + * The mailbox, which is three variables that mean nothing apart and secrets that are not here. + * + * The absent case is the one worth pinning: it is the ordinary state of every deployment that does + * not want this, and it must not be a start-up failure: the catalogue entry stays admissible and + * grantable either way, and a tool call is where a deployment finds out. + */ +describe("the deployment's mailbox", () => { + const MAILBOX = { + MAILBOX_IMAP_HOST: "imap.example.test", + MAILBOX_SMTP_HOST: "smtp.example.test", + MAILBOX_USERS: "bot@example.test", + }; + + test("is absent when none of it is set, without refusing to start", () => { + expect(loadConfig(baseEnvironment).mailbox).toBeUndefined(); + }); + + test("defaults both ports to the implicit-TLS ones", () => { + // 993 and 465 rather than the STARTTLS ports, so the connection is encrypted before the + // password is sent rather than negotiating for it in the clear. + expect(loadConfig({ ...baseEnvironment, ...MAILBOX }).mailbox).toEqual({ + imapHost: "imap.example.test", + imapPort: 993, + smtpHost: "smtp.example.test", + smtpPort: 465, + users: ["bot@example.test"], + // Unset means anywhere, which is the behaviour every deployment had before the list existed. + allowedRecipientDomains: new Set(), + }); + }); + + test("reads several accounts, in order, with the first as the default", () => { + // One pair of hosts, an account each: the shared-hosting shape. The order is a decision, since + // the first is what a tool call that named no account works in. + expect( + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_USERS: + "Support@Example.test, sales@example.test ,billing@example.test", + }).mailbox?.users, + ).toEqual([ + "support@example.test", + "sales@example.test", + "billing@example.test", + ]); + }); + + test("deduplicates accounts that differ only in case", () => { + // The address is also the key of the vault credential holding that account's password, so two + // spellings of one mailbox would be a second account nothing can ever unlock. + expect( + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_USERS: "bot@example.test,BOT@example.test", + }).mailbox?.users, + ).toEqual(["bot@example.test"]); + }); + + test("still reads the singular MAILBOX_USER, as a list of one", () => { + // The variable this feature shipped with. A deployment that already has one should not have to + // be edited to keep working. + expect( + loadConfig({ + ...baseEnvironment, + MAILBOX_IMAP_HOST: "imap.example.test", + MAILBOX_SMTP_HOST: "smtp.example.test", + MAILBOX_USER: "Bot@Example.test", + }).mailbox?.users, + ).toEqual(["bot@example.test"]); + }); + + test("refuses both spellings at once, naming the conflict", () => { + // Two answers to the same question. Merging them or preferring one silently would be guessing + // at an intention nothing here can read. + expect(() => + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_USER: "other@example.test", + }), + ).toThrow("MAILBOX_USERS and MAILBOX_USER are both set"); + }); + + test("refuses an account that is not an address, naming it", () => { + // It would otherwise become an account a model can select, a credential key an administrator + // cannot guess, and a login failure at run time in front of somebody. + expect(() => + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_USERS: "bot@example.test,not-an-address", + }), + ).toThrow('MAILBOX_USERS entry "not-an-address"'); + expect(() => + loadConfig({ + ...baseEnvironment, + MAILBOX_IMAP_HOST: "imap.example.test", + MAILBOX_SMTP_HOST: "smtp.example.test", + MAILBOX_USER: "bot at example.test", + }), + ).toThrow('MAILBOX_USER entry "bot at example.test"'); + }); + + test("reads the recipient allowlist, lower-cased and without the @ people write", () => { + expect( + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_ALLOWED_RECIPIENT_DOMAINS: "Example.com, @partner.example", + }).mailbox?.allowedRecipientDomains, + ).toEqual(new Set(["example.com", "partner.example"])); + }); + + test("an emptied allowlist means anywhere, not nowhere", () => { + // A blanked line in a .env has switched the restriction off. The opposite reading would be a + // mailbox that silently refuses everybody. + expect( + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_ALLOWED_RECIPIENT_DOMAINS: " ", + }).mailbox?.allowedRecipientDomains.size, + ).toBe(0); + }); + + test("refuses an allowlist entry that is not a domain", () => { + // A safety list written as an address would match nothing while looking like a restriction. + for (const entry of ["sales@example.com", "https://example.com", "a b"]) { + expect(() => + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_ALLOWED_RECIPIENT_DOMAINS: entry, + }), + ).toThrow("MAILBOX_ALLOWED_RECIPIENT_DOMAINS"); + } + }); + + test("takes ports a deployment names", () => { + expect( + loadConfig({ + ...baseEnvironment, + ...MAILBOX, + MAILBOX_IMAP_PORT: "1993", + MAILBOX_SMTP_PORT: "2465", + }).mailbox, + ).toMatchObject({ imapPort: 1993, smtpPort: 2465 }); + }); + + test("refuses half a mailbox, naming what is missing", () => { + // Booting with a host and no user is a deployment where every mail tool fails at the first + // login, at run time, in front of somebody, with nothing but an auth error to go on. + expect(() => + loadConfig({ + ...baseEnvironment, + MAILBOX_IMAP_HOST: "imap.example.test", + }), + ).toThrow("MAILBOX_SMTP_HOST, MAILBOX_USERS"); + expect(() => + loadConfig({ ...baseEnvironment, MAILBOX_USERS: "bot@example.test" }), + ).toThrow("MAILBOX_IMAP_HOST, MAILBOX_SMTP_HOST"); + }); + + test("refuses a port that is not a port, rather than falling back to the default", () => { + for (const port of ["nine-nine-three", "0", "70000", "993.5"]) { + expect(() => + loadConfig({ ...baseEnvironment, ...MAILBOX, MAILBOX_IMAP_PORT: port }), + ).toThrow("MAILBOX_IMAP_PORT must be a port number between 1 and 65535"); + } + }); +}); diff --git a/server/tests/plugin-catalogue.test.ts b/server/tests/plugin-catalogue.test.ts index d6b533a4..79fc22f6 100644 --- a/server/tests/plugin-catalogue.test.ts +++ b/server/tests/plugin-catalogue.test.ts @@ -93,10 +93,15 @@ describe("which servers this deployment will talk to", () => { expect(entry.hostPattern?.startsWith("^")).toBe(true); expect(entry.hostPattern?.endsWith("$")).toBe(true); } else if (entry.auth.kind === "builtin") { - // First-party and in-process: there is no host outside this process to reach, so the - // https requirement below does not apply. Asserted positively instead, so this branch - // cannot quietly become a loophole for a future entry that DOES dial a real host. - expect(entry.host).toBe("builtin://routines"); + /* + * First-party and in-process: there is no host outside this process to reach, so the + * https requirement below does not apply. Asserted against a list rather than against the + * `builtin://` scheme, so this branch cannot quietly become a loophole for a future entry + * that DOES dial a real host: adding one means adding it here, deliberately. + */ + expect(["builtin://routines", "builtin://mailbox"]).toContain( + entry.host, + ); } else { expect(entry.host.startsWith("https://")).toBe(true); } @@ -105,6 +110,19 @@ describe("which servers this deployment will talk to", () => { }); describe("whose credential a server uses", () => { + test("a builtin entry says whether it is reached as the asker or as the deployment", () => { + /* + * The one question the audit trail asks about a call with no vendor. Routines touches the + * asker's own rows; Mailbox opens one mailbox the deployment owns, on a password the deployment + * holds. Recording the second as the asker would put a person's id on a row describing access + * that was never theirs, so every builtin has to state it rather than inherit a guess. + */ + for (const entry of CATALOGUE) { + if (entry.auth.kind !== "builtin") continue; + expect(["actor", "deployment"]).toContain(entry.auth.reachedAs); + } + }); + test("every entry says which, rather than leaving it to be inferred", () => { // The whole point of replacing a `needsCredential` boolean. "Needs a credential" did not say // whose, and a reader who guessed would guess the deployment's, which for a user-oauth vendor @@ -252,7 +270,9 @@ describe("Routines", () => { }); test("has no credential, because there is nothing to authenticate to", () => { - expect(entry?.auth.kind).toBe("builtin"); + // Reached as the person asking, unlike the other builtin: a routine is theirs, and the audit + // trail should say whose rows a call touched. + expect(entry?.auth).toEqual({ kind: "builtin", reachedAs: "actor" }); }); test("is reached through the builtin transport, not a vendor", () => { @@ -285,6 +305,50 @@ describe("Routines", () => { }); }); +describe("Mailbox", () => { + const entry = catalogueEntry("mailbox"); + + test("is in the catalogue and resolves to its own builtin address", () => { + expect(entry).not.toBeNull(); + expect(resolveServerUrl("mailbox")?.url).toBe("builtin://mailbox"); + }); + + test("has no per-person credential, because there is one mailbox and it is the deployment's", () => { + expect(entry?.auth).toEqual({ kind: "builtin", reachedAs: "deployment" }); + // `builtin` takes no credential from the server row, so nothing an administrator points at this + // entry is ever spent. The mailbox password is resolved from the vault by its own key instead. + expect(serverCredentialKind(entry as CatalogueEntry)).toBeNull(); + }); + + test("is reached through its own builtin transport, not a vendor and not Routines", () => { + expect(entry?.transport).toBe("builtin-mailbox"); + }); + + test("pins the exact write list, so a dropped or renamed entry fails here", () => { + // One write, and the reason this entry is grantable per tool: reading mail is recoverable and + // sending it is not. + expect(entry?.writeTools).toEqual(["send_message"]); + }); + + test("classifies its tools the same way every other vendor's are classified", () => { + expect(classifyTool(entry, "send_message", true)).toBe("write"); + expect(classifyTool(entry, "list_messages", true)).toBe("read"); + expect(classifyTool(entry, "read_message", true)).toBe("read"); + expect(classifyTool(entry, "search_messages", true)).toBe("read"); + // A name nothing here has vouched for is a write, the same as for any other vendor. + expect(classifyTool(entry, "delete_message", false)).toBe("write"); + // Every tool, advertised or not, is a write when the server never said it was advertised. + for (const name of [ + "send_message", + "list_messages", + "read_message", + "search_messages", + ]) { + expect(classifyTool(entry, name, false)).toBe("write"); + } + }); +}); + describe("what a tool does", () => { const drive = catalogueEntry("google-drive")!; diff --git a/server/tests/plugin-reached-as.integration.test.ts b/server/tests/plugin-reached-as.integration.test.ts new file mode 100644 index 00000000..dda20c01 --- /dev/null +++ b/server/tests/plugin-reached-as.integration.test.ts @@ -0,0 +1,171 @@ +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { randomUUID } from "node:crypto"; +import { and, eq, inArray } from "drizzle-orm"; +import { createAuditStore } from "../src/audit"; +import type { ActionPolicy } from "../src/computer/policy"; +import { createDatabase } from "../src/db/client"; +import { + agents, + auditEvents, + mcpServers, + pluginGrants, +} from "../src/db/schema"; +import { catalogueEntry } from "../src/plugins/catalogue"; +import { createPluginStore } from "../src/plugins/store"; +import { TEST_POOL } from "./support/database"; + +/** + * Whose access a call ran on, as the audit trail records it, for the two connectors with no vendor. + * + * The catalogue says which each builtin is (`reachedAs` on the entry) and that is asserted where it + * is written. This file asserts the half that actually matters: that the value reaches the ROW. The + * question an investigation asks of a per-person connector is "who did this run reach as", and the + * two builtins answer it differently for a real reason. A routine touches the asking person's own + * rows, so the row names them. The mailbox is one mailbox belonging to the deployment, opened on a + * password the deployment holds, so a row naming the asker would attribute access to somebody who + * never had it. + * + * Nothing is dialled: the vendor is injected, so neither builtin transport runs and no mailbox or + * routine store is needed. What is under test is the row, not the tool. + */ + +const database = createDatabase( + process.env.DATABASE_URL ?? + "postgres://openbot:openbot@localhost:5432/openbot", + TEST_POOL, +); + +const suite = randomUUID().slice(0, 8); +const botId = `agent_reached_as_${suite}`; +const actorId = `person_${suite}@openbot.local`; + +const CALLS = [ + { serverId: "routines", toolName: "list_routines", reachedAs: actorId }, + { serverId: "mailbox", toolName: "list_messages", reachedAs: "deployment" }, +] as const; + +const policy: ActionPolicy = { mode: "enforce", deny: [], allow: ["true"] }; + +/** Which of the two server rows this suite created, so it removes only those. */ +const created: string[] = []; + +const store = createPluginStore({ + database, + auditStore: createAuditStore(database), + credentials: { + // A builtin server row holds no credential, so nothing here is ever read. Loud rather than + // absent: a read would mean this file had started exercising something it does not claim to. + readSecret: async () => { + throw new Error("a builtin server has no credential to read"); + }, + create: async () => { + throw new Error("this suite does not write credentials"); + }, + updateSecret: async () => { + throw new Error("this suite does not write credentials"); + }, + revoke: async () => new Date(), + }, + encryptionKey: "x".repeat(44), + policy: () => policy, + // No transport runs. The row is written from the entry and the actor, and both are known before + // anything would have been dialled. + callVendor: async () => ({ text: "ok", isError: false }), +}); + +beforeAll(async () => { + await database + .insert(agents) + .values({ id: botId, name: botId, type: "remote_ag_ui", configuration: {} }) + .onConflictDoNothing(); + + for (const { serverId, toolName } of CALLS) { + const entry = catalogueEntry(serverId); + if (!entry) throw new Error(`${serverId} is not in the catalogue`); + + const existing = await database + .select({ id: mcpServers.id }) + .from(mcpServers) + .where(eq(mcpServers.id, serverId)); + if (existing.length === 0) created.push(serverId); + + // Written directly rather than through addServer, so the test needs nothing to be reachable. + await database + .insert(mcpServers) + .values({ + id: serverId, + title: entry.title, + vendor: entry.vendor, + url: entry.host as string, + provenance: "first-party", + }) + .onConflictDoNothing(); + + await store.grant( + "mcp", + `${serverId}/${toolName}`, + botId, + "admin@openbot.local", + ); + } +}); + +afterAll(async () => { + // Scoped to this suite's own Bot. The refs name real connectors, so a delete by ref alone would + // take an administrator's grants for a Bot people use. + await database.delete(pluginGrants).where( + and( + inArray( + pluginGrants.ref, + CALLS.map(({ serverId, toolName }) => `${serverId}/${toolName}`), + ), + eq(pluginGrants.agentId, botId), + ), + ); + await database.delete(agents).where(eq(agents.id, botId)); + // A server row is deployment configuration. Removed only where this suite is what added it. + if (created.length > 0) { + await database.delete(mcpServers).where(inArray(mcpServers.id, created)); + } +}); + +describe("whose access a builtin call is recorded as", () => { + for (const { serverId, toolName, reachedAs } of CALLS) { + test(`${serverId} is reached as ${reachedAs === "deployment" ? "the deployment" : "the asker"}`, async () => { + const ref = `${serverId}/${toolName}`; + const result = await store.callTool({ + ref, + args: {}, + botId, + actorId, + }); + expect(result.isError).toBe(false); + + const rows = await database + .select({ + eventType: auditEvents.eventType, + payload: auditEvents.payload, + }) + .from(auditEvents) + .where( + and( + eq(auditEvents.targetType, "mcp_tool"), + eq(auditEvents.targetId, ref), + ), + ); + + const mine = rows.filter( + (row) => + row.eventType === "mcp.call_succeeded" && + (row.payload as { bot?: string }).bot === botId, + ); + expect(mine.length).toBeGreaterThan(0); + expect((mine[0].payload as { reachedAs?: string }).reachedAs).toBe( + reachedAs, + ); + // The actor is on the row either way. What `reachedAs` settles is whether the access was + // theirs, which is a different question from who asked. + expect((mine[0].payload as { actor?: string }).actor).toBe(actorId); + }); + } +});