Skip to content

Add cluster interserver-secret authentication#1855

Open
BorisTyshkevich wants to merge 1 commit into
ClickHouse:mainfrom
Altinity:feature/interserver-secret-upstream
Open

Add cluster interserver-secret authentication#1855
BorisTyshkevich wants to merge 1 commit into
ClickHouse:mainfrom
Altinity:feature/interserver-secret-upstream

Conversation

@BorisTyshkevich
Copy link
Copy Markdown

@BorisTyshkevich BorisTyshkevich commented May 5, 2026

Add cluster interserver-secret authentication

A trusted internal service can authenticate as a ClickHouse cluster peer by
sharing the cluster secret instead of a user password, and to execute
each query as an arbitrary initial_user which the server accepts via
AlwaysAllowCredentials. Mirrors the existing distributed-query
protocol documented in src/Server/TCPHandler.cpp.

API

  • Options.Cluster{Name, Secret} configures interserver mode.
  • WithInitialUser sets the impersonated user per query (falls back to
    Auth.Username).
  • ClusterCredentials.String/GoString redact Secret so accidental
    logging of the Options struct cannot leak it.
  • Hello sends " INTERSERVER SECRET ", empty password, cluster name,
    and a 32-byte random salt when Cluster.Secret is set.
  • Each query encodes SHA256(salt + secret + body + id + initial_user)
    into the interserver-secret slot and flips ClientQueryInitial to
    ClientQuerySecondary. When Cluster.Secret is empty the slot stays
    the legacy empty string, so existing callers are unaffected.
  • Driver advertises protocol revision 54460, so nonce and externally-
    granted-roles fields from DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET_V2
    (54462) and later are intentionally omitted.

Validation at Open

Fail-closed defaults to keep accidental misconfiguration loud:

  • ErrClusterSecretRequiresName — Cluster.Secret without Cluster.Name.
  • ErrClusterSecretNeedsNative — Cluster.Secret with Protocol=HTTP.
  • ErrClusterSecretRequiresUsername — Cluster.Secret with empty
    Auth.Username (otherwise setDefaults rewrites it to "default" and a
    forgotten WithInitialUser would silently run as the cluster superuser).
    A Warn line is emitted when interserver mode is active, and a second
    Warn when it is active without TLS — the V1 protocol the driver speaks
    has no nonce, so an on-path attacker on a plaintext link could replay a
    captured signed query frame on the same connection.

DSN: deliberately not supported

Cluster.Secret is sensitive cluster-wide credential material. DSNs
end up in startup logs, error messages, config files, and stack traces;
the secret must not. The driver intentionally has no DSN parameter for
the cluster secret — keep it in memory (env var → Options{}).

Tests

  • lib/proto/query_interserver_test.go covers the hash layout, asserts
    the empty-secret branch preserves the legacy empty slot byte-for-byte,
    and verifies query_kind flips Initial→Secondary in actual encoded
    output.
  • tests/interserver_test.go covers the end-to-end flow against a real
    server: creates a passwordless user, opens a connection with
    Cluster.Secret, runs a query under WithInitialUser, and asserts
    system.query_log shows is_initial_query=0 with user=initial_user.
    Negative paths cover wrong-secret rejection by the server, the three
    Open() validation errors, and the redaction guard.
  • tests/resources/custom.xml adds a <test_cluster_secret> entry so
    the single-node test container accepts incoming
    " INTERSERVER SECRET " connections naming that cluster.

Docs

  • README gains "Cluster interserver-secret authentication" with
    Security model, Why-not-DSN rationale, and a comparison with the new
    EXECUTE AS SQL statement (server version coverage, connection-pool
    friction, identity-in-query-text surface, audit-trail differences).
  • examples/clickhouse_api/cluster_secret.go runnable example sources
    the secret from CLICKHOUSE_CLUSTER_SECRET.

Allows a client to authenticate as a trusted ClickHouse cluster peer by
sharing the cluster secret instead of a user password, and to execute
each query as an arbitrary `initial_user` which the server accepts via
`AlwaysAllowCredentials`. Mirrors the existing distributed-query
protocol documented in `src/Server/TCPHandler.cpp`.

API
---
- `Options.Cluster{Name, Secret}` configures interserver mode.
- `WithInitialUser` sets the impersonated user per query (falls back to
  `Auth.Username`).
- `ClusterCredentials.String/GoString` redact `Secret` so accidental
  logging of the Options struct cannot leak it.
- Hello sends `"  INTERSERVER SECRET  "`, empty password, cluster name,
  and a 32-byte random salt when `Cluster.Secret` is set.
- Each query encodes `SHA256(salt + secret + body + id + initial_user)`
  into the interserver-secret slot and flips `ClientQueryInitial` to
  `ClientQuerySecondary`. When `Cluster.Secret` is empty the slot stays
  the legacy empty string, so existing callers are unaffected.
- Driver advertises protocol revision 54460, so nonce and externally-
  granted-roles fields from `DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET_V2`
  (54462) and later are intentionally omitted.

Validation at Open
------------------
Fail-closed defaults to keep accidental misconfiguration loud:
- `ErrClusterSecretRequiresName`     — Cluster.Secret without Cluster.Name.
- `ErrClusterSecretNeedsNative`      — Cluster.Secret with Protocol=HTTP.
- `ErrClusterSecretRequiresUsername` — Cluster.Secret with empty
  Auth.Username (otherwise setDefaults rewrites it to "default" and a
  forgotten WithInitialUser would silently run as the cluster superuser).
A Warn line is emitted when interserver mode is active, and a second
Warn when it is active without TLS — the V1 protocol the driver speaks
has no nonce, so an on-path attacker on a plaintext link could replay a
captured signed query frame on the same connection.

DSN: deliberately not supported
-------------------------------
`Cluster.Secret` is sensitive cluster-wide credential material. DSNs
end up in startup logs, error messages, config files, and stack traces;
the secret must not. The driver intentionally has no DSN parameter for
the cluster secret — keep it in memory (env var → `Options{}`).

Tests
-----
- `lib/proto/query_interserver_test.go` covers the hash layout, asserts
  the empty-secret branch preserves the legacy empty slot byte-for-byte,
  and verifies `query_kind` flips Initial→Secondary in actual encoded
  output.
- `tests/interserver_test.go` covers the end-to-end flow against a real
  server: creates a passwordless user, opens a connection with
  Cluster.Secret, runs a query under WithInitialUser, and asserts
  `system.query_log` shows `is_initial_query=0` with `user=initial_user`.
  Negative paths cover wrong-secret rejection by the server, the three
  Open() validation errors, and the redaction guard.
- `tests/resources/custom.xml` adds a `<test_cluster_secret>` entry so
  the single-node test container accepts incoming
  `"  INTERSERVER SECRET  "` connections naming that cluster.

Docs
----
- README gains "Cluster interserver-secret authentication" with
  Security model, Why-not-DSN rationale, and a comparison with the new
  `EXECUTE AS` SQL statement (server version coverage, connection-pool
  friction, identity-in-query-text surface, audit-trail differences).
- `examples/clickhouse_api/cluster_secret.go` runnable example sources
  the secret from `CLICKHOUSE_CLUSTER_SECRET`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented May 5, 2026

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants