Merge pull request #5810 from sysown/docs/passthrough-auth-spec

feat(passthrough-auth): pass-through authentication (spec + Phase 1, WIP)

Author: renecannao

CHANGELOG.md

@@ -2,6 +2,73 @@
 
 ## Version 3.0.x Series - PostgreSQL Support Introduction
 
+### Unreleased
+
+#### Pass-Through Authentication (Phase 1 — opt-in, MySQL only)
+
+New authentication mode that lets a MySQL client authenticate through
+ProxySQL without a pre-provisioned password in `mysql_users`. ProxySQL
+intercepts the cleartext password during the `caching_sha2_password`
+*full-auth* exchange, probes the configured backend with it, and on
+success caches the credential for subsequent connects. See PR #5810 and
+`doc/internal/passthrough_authentication.md` for the design.
+
+Feature is **off by default**. Existing deployments see no behavioral
+change unless an operator explicitly enables it.
+
+To opt in (one of):
+
+- Provision a `mysql_users` row with `password=''` (empty) and set
+  `mysql-passthrough_auth_enabled=true`. The empty-password row now
+  semantically means "learn the password from the first successful
+  caching_sha2_password connect" (it no longer permits passwordless
+  login when the master gate is on — documented behavior change).
+- Set `mysql-passthrough_auth_unknown_users=true` to also pass-through
+  usernames that don't exist in `mysql_users` at all. Routing comes
+  from `mysql-passthrough_default_hg`. **SECURITY**: this combination
+  without `mysql-passthrough_auth_username_pattern` set is the most
+  dangerous pass-through configuration — ProxySQL will emit a WARNING
+  at `LOAD MYSQL VARIABLES TO RUNTIME` flagging the operator.
+
+New globals (13):
+- `mysql-passthrough_auth_enabled` (bool, default `false`)
+- `mysql-passthrough_auth_empty_password` (bool, default `true`)
+- `mysql-passthrough_auth_unknown_users` (bool, default `false`)
+- `mysql-passthrough_auth_require_tls` (bool, default `true`)
+- `mysql-passthrough_default_hg` (int, default `0`)
+- `mysql-passthrough_default_schema` (str, default `""`)
+- `mysql-passthrough_auth_cache_ttl_s` (int, default `0` = no expiry)
+- `mysql-passthrough_auth_max_inflight_probes` (int, default `100`)
+- `mysql-passthrough_auth_username_pattern` (str, default `""` = allow all; re2 regex allowlist)
+- `mysql-passthrough_auth_max_failures_per_user` (int, default `3`)
+- `mysql-passthrough_auth_max_failures_per_ip` (int, default `10`)
+- `mysql-passthrough_auth_failure_window_s` (int, default `60`)
+- `mysql-passthrough_auth_failure_map_cap` (int, default `100000`)
+
+New admin commands:
+- `PROXYSQL FLUSH PASSTHROUGH_AUTH_CACHE [FOR USER '<name>']`
+
+New stats virtual tables:
+- `stats_mysql_passthrough_auth_cache` (entries, no passwords)
+- `stats_mysql_passthrough_auth_metrics` (9 counters + 2 gauges)
+
+New audit events:
+- `MySQL_Client_Connect_Passthrough_OK`
+- `MySQL_Client_Connect_Passthrough_FAIL`
+
+Phase 1 limitations (documented in code + spec):
+- `caching_sha2_password` only (`mysql_clear_password` → Phase 2;
+  `mysql_native_password` → Phase 3)
+- Synchronous backend probe (matches MySQL_Monitor's pattern;
+  pinning a worker thread for the duration of one backend handshake)
+- One-shot probe connections do not go through the connection pool
+  (don't count against `mysql_servers.max_connections`; the effective
+  ceiling is `mysql-passthrough_auth_max_inflight_probes`)
+- No persistence; restart = cold cache
+- `COM_CHANGE_USER` targeting a pass-through user is rejected
+- Cluster sync of variables works via the standard `mysql_variables`
+  path; metric counters are per-node
+
 ### v3.0.2 (2025-08-06)
 #### PostgreSQL Enhancements
 - Improved error processing and reporting format for backend connections using libpq (#4947)

doc/internal/passthrough_authentication.md

@@ -82,6 +82,35 @@ mysql_variables=
 	commands_stats=true
 	sessions_sort=true
 	connect_retries_on_failure=10
+
+#####################################################################
+# Pass-through authentication (opt-in, off by default).
+#
+# When enabled, ProxySQL can authenticate a client to the backend
+# without a pre-provisioned password in mysql_users: it intercepts
+# the cleartext password sent during the caching_sha2_password
+# full-auth exchange, probes a backend with it, and on success
+# caches the credential in memory for the next connect. See
+# doc/internal/passthrough_authentication.md for the design.
+#
+# IMPORTANT: enabling passthrough_auth_unknown_users WITHOUT setting
+# passthrough_auth_username_pattern lets ANY username be probed
+# against the backend. ProxySQL emits a startup warning when that
+# combination is published; treat the warning seriously.
+#####################################################################
+#	passthrough_auth_enabled=false                  # master gate; nothing below applies until true
+#	passthrough_auth_empty_password=true            # honor mysql_users rows with password='' as pass-through opt-in
+#	passthrough_auth_unknown_users=false            # also allow usernames NOT in mysql_users (gated by passthrough_default_hg)
+#	passthrough_auth_require_tls=true               # refuse to send AuthMoreData{0x04} over non-TLS client connections
+#	passthrough_default_hg=0                        # hostgroup used for unknown-user probes
+#	passthrough_default_schema=""                   # default schema for unknown-user sessions (falls back to default_schema if empty)
+#	passthrough_auth_cache_ttl_s=0                  # 0 = entries never expire; nonzero = lazy eviction window in seconds
+#	passthrough_auth_max_inflight_probes=100        # concurrent probes cap (DoS / thundering-herd protection)
+#	passthrough_auth_username_pattern=""            # re2 regex; empty = allow all (HIGHLY recommended to set when unknown_users=true)
+#	passthrough_auth_max_failures_per_user=3        # per-user sliding-window threshold
+#	passthrough_auth_max_failures_per_ip=10         # per-source-IP sliding-window threshold
+#	passthrough_auth_failure_window_s=60            # sliding-window size for both counters
+#	passthrough_auth_failure_map_cap=100000         # max distinct keys retained in failure deques (defense-in-depth)
 }
 
 

etc/proxysql.cnf

@@ -170,6 +170,16 @@ class MySQL_Data_Stream
 	// Updated **only** when an 'auth_switch' has been sent to client
 	enum proxysql_auth_plugins switching_auth_sent;
 	int auth_in_progress; // if 0 , no authentication is in progress. Any value greater than 0 depends from the implementation
+	// Pass-through authentication: cleartext password received from the
+	// client during the caching_sha2_password full-auth exchange (spec
+	// §4.1). Captured by PPHR_passthrough_init at stage 5 and consumed by
+	// handler_again___status_AUTHENTICATING_BACKEND_FOR_CLIENT when it
+	// drives the non-blocking backend connect. Lives on a dedicated field
+	// (NOT userinfo->password) because process_pkt_handshake_response's
+	// epilogue overwrites userinfo->password with "" when auth is still in
+	// progress -- which would clobber the borrowed cleartext before the
+	// session handler runs. NULL when no pass-through probe is in flight.
+	char *passthrough_cleartext;
 	unsigned int tmp_charset;
 
 	short revents;

include/MySQL_Data_Stream.h

@@ -553,6 +553,33 @@ class MySQL_Logger {
 	 */
 	void log_audit_entry(log_event_type _et, MySQL_Session* sess, MySQL_Data_Stream* myds, char* xi = NULL);
 
+	/**
+	 * @brief Emit an audit entry tagged with a hostgroup id.
+	 *
+	 * Spec §7.4 for pass-through auth requires the audit entry to include
+	 * "username, source IP, hostgroup probed, outcome". The username and
+	 * source IP fields are already populated automatically from
+	 * @p sess->client_myds. This overload threads the hostgroup that was
+	 * probed (or that drove a denied attempt) onto the underlying
+	 * MySQL_Event::hid field, which @ref MySQL_Event::write_auth then
+	 * emits as the JSON @c hostgroup field. The existing
+	 * single-parameter form continues to be used by every non-pass-through
+	 * caller and leaves @c hid as @c UINT64_MAX (omitted from the JSON).
+	 *
+	 * @param _et       Event type (PROXYSQL_MYSQL_AUTH_PASSTHROUGH_OK or _FAIL).
+	 * @param sess      Originating session (provides user / IP).
+	 * @param myds      Backend data stream, if any. NULL for the pass-through
+	 *                  probe path because the probe is a one-shot
+	 *                  libmariadbclient connection that never becomes a
+	 *                  MySQL_Data_Stream.
+	 * @param xi        Optional extra-info string, written verbatim into the
+	 *                  JSON @c extra_info field.
+	 * @param hostgroup Hostgroup id that was probed (>= 0). Use the helper
+	 *                  via this overload only when the value is meaningful;
+	 *                  pass @c -1 to fall back to the non-hostgroup form.
+	 */
+	void log_audit_entry(log_event_type _et, MySQL_Session* sess, MySQL_Data_Stream* myds, char* xi, int hostgroup);
+
 	/**
 	 * @brief Flushes the log files.
 	 */

include/MySQL_Logger.hpp

@@ -0,0 +1,208 @@
+#ifndef PROXYSQL_MYSQL_PASSTHROUGH_AUTH_CACHE_H
+#define PROXYSQL_MYSQL_PASSTHROUGH_AUTH_CACHE_H
+
+#include <pthread.h>
+#include <atomic>
+#include <cstddef>
+#include <cstdint>
+#include <deque>
+#include <string>
+#include <unordered_map>
+#include <vector>
+
+/**
+ * @brief Forward declaration of re2::RE2 to keep this header light.
+ *
+ * The full re2/re2.h pulls in a large set of headers and dependencies; we
+ * only need a pointer to RE2 in the class state, so a forward declaration
+ * suffices. The translation unit that owns the compiled regex
+ * (lib/MySQL_Passthrough_Auth_Cache.cpp) includes the full header.
+ */
+namespace re2 { class RE2; }
+
+#ifdef DEBUG
+#define MYSQL_PASSTHROUGH_AUTH_CACHE_DEB "_DEBUG"
+#else
+#define MYSQL_PASSTHROUGH_AUTH_CACHE_DEB ""
+#endif
+#define MYSQL_PASSTHROUGH_AUTH_CACHE_VERSION "0.1.0000" MYSQL_PASSTHROUGH_AUTH_CACHE_DEB
+
+struct passthrough_entry_view {
+	std::string username;
+	uint64_t learned_at_us;
+	int hostgroup_probed;
+};
+
+class MySQL_Passthrough_Auth_Cache {
+	private:
+		struct entry_t {
+			std::string cleartext_password;
+			uint64_t learned_at_us;
+			int hostgroup_probed;
+		};
+		mutable pthread_rwlock_t lock;
+		std::unordered_map<std::string, entry_t> entries;
+		std::atomic<int> inflight_probes;
+
+		/**
+		 * @brief Atomic counters for operational observability (spec §7.4 follow-up).
+		 *
+		 * Exposed via @c stats_mysql_passthrough_auth_metrics. Each
+		 * counter increments at exactly one well-defined point in
+		 * @c handler_again___status_AUTHENTICATING_BACKEND_FOR_CLIENT
+		 * (see the corresponding @c bump_* methods below). Monotonic
+		 * since process start; reset only by process restart.
+		 *
+		 * Naming mirrors the existing @c stats_mysql_global pattern of
+		 * "what happened" snake-case-counters; no special suffixes.
+		 */
+		std::atomic<uint64_t> stat_probes_attempted;
+		std::atomic<uint64_t> stat_probes_ok;
+		std::atomic<uint64_t> stat_probes_failed_credentials;
+		std::atomic<uint64_t> stat_probes_failed_transport;
+		std::atomic<uint64_t> stat_lockouts_user;
+		std::atomic<uint64_t> stat_lockouts_ip;
+		std::atomic<uint64_t> stat_inflight_cap_rejects;
+		std::atomic<uint64_t> stat_cache_hits;
+		std::atomic<uint64_t> stat_cache_invalidations;
+		// Sliding-window failure counters (spec §7.2). Per-username and
+		// per-source-IP. Mutated only behind failure_lock — a separate
+		// mutex from `lock` since these are write-mostly and accessed on
+		// every probe.
+		mutable pthread_mutex_t failure_lock;
+		mutable std::unordered_map<std::string, std::deque<uint64_t>> failures_by_user;
+		mutable std::unordered_map<std::string, std::deque<uint64_t>> failures_by_ip;
+
+		/**
+		 * @brief Compiled-regex cache for the username allowlist (spec §7.1).
+		 *
+		 * Compiling an re2::RE2 is cheap (single-digit microseconds) but
+		 * is paid every time a candidate connect is checked. Cache the
+		 * last-seen pattern string alongside its compiled form so that as
+		 * long as @c mysql-passthrough_auth_username_pattern is unchanged
+		 * we hit the compiled form. A pattern change (admin SET, reload)
+		 * triggers a re-compile under the write lock.
+		 *
+		 * @c pattern_lock is a pthread_rwlock so the COMMON case
+		 * (steady-state pattern, every probe takes the read lock for
+		 * FullMatch) doesn't serialize through a single mutex. The write
+		 * lock is taken only when the pattern STRING changes, which
+		 * happens on admin SET / LOAD MYSQL VARIABLES TO RUNTIME and is
+		 * effectively rare. re2::RE2::FullMatch is documented as
+		 * thread-safe on a const RE2 instance, so concurrent readers
+		 * are fine.
+		 *
+		 * Holds a raw pointer (forward-declared above) rather than
+		 * unique_ptr so we don't need to drag re2/re2.h into this header.
+		 */
+		mutable pthread_rwlock_t pattern_lock;
+		mutable std::string compiled_pattern_str;
+		mutable re2::RE2 *compiled_pattern;
+
+	public:
+		MySQL_Passthrough_Auth_Cache();
+		~MySQL_Passthrough_Auth_Cache();
+
+		// Look up a cached credential. Returns true on hit (and populates
+		// out_cleartext); false on miss. If ttl_s > 0 and the entry is older
+		// than ttl_s, the entry is evicted and a miss is returned.
+		bool lookup(const std::string& username, std::string& out_cleartext, uint32_t ttl_s);
+
+		// Insert or replace a cached credential.
+		void insert(const std::string& username, const std::string& cleartext, int hostgroup_probed);
+
+		// Evict a single entry. Returns true if the entry was present.
+		bool evict(const std::string& username);
+
+		// Remove every entry.
+		void clear();
+
+		// Number of entries currently held.
+		size_t size() const;
+
+		// Snapshot of entries (without password) for stats / observability.
+		std::vector<passthrough_entry_view> snapshot() const;
+
+		// Global in-flight probe counter (spec §7.3). Sessions wishing to
+		// start a backend probe call try_acquire_inflight with the current
+		// configured cap; on true they MUST pair with release_inflight when
+		// the probe completes (success or failure). On false the session
+		// must reject the auth with a generic ERR.
+		bool try_acquire_inflight(int max_inflight);
+		void release_inflight();
+		int  inflight() const;
+
+		// Sliding-window failure counters (spec §7.2). Sessions check
+		// would_lockout before probing; on probe failure, record a
+		// failure. window_s defines the sliding window in seconds; older
+		// timestamps are dropped lazily on check.
+		bool would_lockout_user(const std::string& username, int max_failures, uint32_t window_s) const;
+		bool would_lockout_ip(const std::string& ip, int max_failures, uint32_t window_s) const;
+		/**
+		 * @brief Record a probe failure against (username, ip) deques.
+		 *
+		 * @param max_keys Operator-tunable cap on the size of each
+		 *                 failure map (failures_by_user, failures_by_ip).
+		 *                 Driven by @c mysql-passthrough_auth_failure_map_cap.
+		 *                 When the map exceeds the cap, evict_oldest
+		 *                 reclaims an entry (defense-in-depth against
+		 *                 username/IP churn that would otherwise grow
+		 *                 the maps unbounded).
+		 */
+		void record_failure(const std::string& username, const std::string& ip, int max_keys);
+
+		/**
+		 * @brief Observability counters (B7 follow-up).
+		 *
+		 * Each @c bump_* method increments the corresponding atomic at
+		 * the single call site documented in @c MySQL_Session.cpp. The
+		 * @c metrics_snapshot helper returns the current values for the
+		 * @c stats_mysql_passthrough_auth_metrics virtual table.
+		 */
+		void bump_probes_attempted();
+		void bump_probes_ok();
+		void bump_probes_failed_credentials();
+		void bump_probes_failed_transport();
+		void bump_lockouts_user();
+		void bump_lockouts_ip();
+		void bump_inflight_cap_rejects();
+		void bump_cache_hits();
+		void bump_cache_invalidations();
+
+		/**
+		 * @brief Snapshot of metric counters + current-state gauges.
+		 *
+		 * Returns a vector of (name, value) pairs ordered for stable JSON /
+		 * stats-table output. Values are read with relaxed memory ordering
+		 * since stats are advisory, not synchronizing.
+		 */
+		struct metric_kv {
+			std::string name;
+			uint64_t value;
+		};
+		std::vector<metric_kv> metrics_snapshot() const;
+
+		/**
+		 * @brief Check whether @p username matches the configured allowlist
+		 * regex (spec §7.1, mysql-passthrough_auth_username_pattern).
+		 *
+		 * @param username Frontend user attempting pass-through.
+		 * @param pattern  Regex string from the global variable. Empty means
+		 *                 "allow every username" (back-compat default).
+		 * @return @c true when the pattern is empty or @p username FullMatches
+		 *         the compiled regex; @c false when the regex is set and
+		 *         either fails to compile or the username doesn't match.
+		 *
+		 * The compiled regex is cached on the class behind @c pattern_lock;
+		 * a pattern-string change triggers a re-compile on the next call.
+		 * Match semantics are RE2 FullMatch (the entire username must
+		 * match), matching how query rules use re2 elsewhere in ProxySQL.
+		 * A regex that fails to compile is treated as a deny-all -- the
+		 * fail-safe direction for a security gate.
+		 */
+		bool username_allowed(const std::string& username, const std::string& pattern);
+
+		void print_version();
+};
+
+#endif // PROXYSQL_MYSQL_PASSTHROUGH_AUTH_CACHE_H

include/MySQL_Passthrough_Auth_Cache.h

@@ -196,6 +196,16 @@ class MySQL_Protocol {
 	void PPHR_6auth2(bool& ret, MyProt_tmp_auth_vars& vars1);
 	bool PPHR_verify_sha2(MyProt_tmp_auth_vars& vars1, enum proxysql_auth_plugins passformat, PASSWORD_TYPE::E passtype);
 	void PPHR_sha2full(bool& ret, MyProt_tmp_auth_vars& vars1, enum proxysql_auth_plugins passformat, PASSWORD_TYPE::E passtype);
+	// Pass-through authentication (see doc/internal/passthrough_authentication.md).
+	// PPHR_passthrough_init runs the protocol-side state machine for the
+	// caching_sha2_password full-auth exchange when ProxySQL doesn't yet
+	// have a password for the user. At switching_auth_stage==0 it sends
+	// AuthMoreData{0x04} so the client emits its cleartext; at stage 5 it
+	// stashes the captured cleartext on the data stream and transitions
+	// the session to AUTHENTICATING_BACKEND_FOR_CLIENT so the backend
+	// probe (handler_again___status_AUTHENTICATING_BACKEND_FOR_CLIENT)
+	// can validate the credential.
+	void PPHR_passthrough_init(MyProt_tmp_auth_vars& vars1);
 	void PPHR_7auth1(bool& ret, MyProt_tmp_auth_vars& vars1, char * reply, account_details_t& attr1);
 	void PPHR_7auth2(bool& ret, MyProt_tmp_auth_vars& vars1, char * reply, account_details_t& attr1);
 	void PPHR_next_auth_stage(MyProt_tmp_auth_vars& vars1, PASSWORD_TYPE::E passtype);