OAuth 2.0 support for http MCP servers

[bellmast]

Problem

Connecting JupyterAI to an http MCP server that requires auth currently requires the user to copy/paste a bearer token (e.g., PAT) into mcp_settings.json. Tokens expire with no refresh, and there's no 401 handling, so connections silently break and users have to manually rotate their tokens.

Proposed Solution

Following the MCP authorization spec, we propose to add an OAuth flow to the http transport -- when the client sees a 401 error from an MCP server, it will try to discover the auth server, send the user to sign-in and consent to the requested scopes, complete the authorization-code flow with PKCE, store the token, and do retry/refresh silently. In short, this feature moves the burden of token management from the user to the client agent.

Additional context

• This feature may be redundant for frontier agents (e.g., Claude) that do MCP OAuth themselves. A Jupyter-owned runner would unify OAuth across any persona that the user loads, including agents whose own OAuth is missing or immature (e.g., Goose re-runs the full sign-in every session, without token persistence/refresh).
• We can surface the sign-in URL as a link that the user clicks on rather than auto-opening a browser. Auto-open is normal for local clients (e.g., Claude Code in VS Code), but for a remote deployment like JupyterHub a user-initiated link is probably the more reliable design.
• Redirect handling differs by deployment: loopback for local (RFC 8252), a Jupyter Server route for remote. To make this work for arbitrary JupyterHub deployments, supporting a configured client_id (+ optional dynamic registration, RFC 7591) avoids pre-registering every hub URL. Recommend that we start with the local case first, then a configured client_id for known hubs, then dynamic registration to scale to arbitrary hubs in later builds.
• Relates to MCP servers manager extension #1567