Skip to content

Commit 714448d

Browse files
authored
SNOW-3351450: emit client_connection_identifier_shape in-band telemetry (#2877)
## Summary Capture the shape of connection-identifier inputs (which of `account` / `region` / `host` the caller supplied, and whether the raw account string carries dot/dash markers) at the earliest parse point in `SnowflakeConnection.__config`, then emit a single in-band `client_connection_identifier_shape` telemetry record per successful login alongside the existing `_log_minicore_import` / `_log_nanoarrow_import` hooks. Mirrors the gosnowflake reference implementation in [snowflakedb/gosnowflake#1797](snowflakedb/gosnowflake#1797). Time-boxed data collection ahead of the Northstar migration. The emission honors the post-login `CLIENT_TELEMETRY_ENABLED` parameter (automatic via ``self.telemetry_enabled`` inside ``_log_telemetry``) and a local `SF_TELEMETRY_DISABLE_CONNECTION_SHAPE=true` env-var kill switch (case-insensitive). No hostname or account value is included in the payload — only five stringified booleans describing intent. Removal is tracked in SNOW-3548350 (target: 2026-11-30); all new code is marked with a `TODO(SNOW-3548350)` so future grep finds it. ## Changes - **New** `src/snowflake/connector/connection_identifier_shape.py`: ``ConnectionIdentifierShape`` dataclass + pure ``record_input_shape(kwargs)`` helper. Captures the dash signal only from the *account portion* of the raw account string (the part before any first dot), so region-tail dashes (the `-east-` in `"myacct.us-east-1"`) aren't miscounted as `account_org_provided`. - **Capture site** at the very top of ``SnowflakeConnection.__config``, idempotent so a re-config with normalized values cannot overwrite the original user intent. Runs before the ``setattr`` loop, before ``construct_hostname`` for host inference, and before ``parse_account`` strips the ``.global`` suffix. - **Emission site** mirrors ``_log_minicore_import`` exactly: ``_log_connection_identifier_shape()`` is invoked from ``__init__`` right after the existing minicore/nanoarrow hooks (sync path) and from ``aio._connection.SnowflakeConnection.connect`` right after ``_log_telemetry_imported_packages()`` (async path). The async class has its own ``async def _log_connection_identifier_shape`` override because ``_log_telemetry`` is overridden as an awaitable there; calling the sync emit from the async path would return an unawaited coroutine. - **TelemetryField** picks up one new event type (``CONNECTION_IDENTIFIER_SHAPE = "client_connection_identifier_shape"``) and five payload keys (``KEY_ACCOUNT_PROVIDED`` etc.) so the wire format matches the Go reference byte-for-byte. - **DESCRIPTION.md** entry under ``Upcoming Release`` mirroring the gosnowflake CHANGELOG voice. ## Test plan - New `test/unit/test_connection_identifier_shape.py` — 13 table-driven cases exercising ``record_input_shape`` over the truth table (locator, dotted account, org-prefixed account, region-dash isolation, host-only, all-three, empty/None/non-string defensive paths, irrelevant-kwargs isolation). - New `test/unit/test_connection_identifier_shape_telemetry.py` — 16 cases exercising the emission method: payload field check, skip when shape unset / attribute missing, env-switch kill (case-insensitive ``true`` only), env-switch off-by-default for every other value including former truthy aliases (``1`` / ``yes``) so the new contract is unambiguous. - 643 existing unit tests still pass locally; the 3 failures + 27 errors observed are pre-existing arrow-stream-extension / wiremock requirements unrelated to this change.
1 parent c1b03e1 commit 714448d

10 files changed

Lines changed: 803 additions & 7 deletions

DESCRIPTION.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Source code is also available at: https://github.com/snowflakedb/snowflake-conne
99
# Release Notes
1010
- Upcoming Release
1111
- Fixed sdist to only install the minicore binary matching the current platform (SNOW-3526469). Previous 4.x releases copied every platform's minicore `.so`/`.dylib`/`.dll` into the install prefix, breaking downstream packagers (e.g. Homebrew) whose audits reject foreign-arch binaries.
12+
- Added one in-band telemetry record per successful login describing which connection-identifier fields the user supplied (`account_provided`, `account_with_region`, `account_org_provided`, `region_provided`, `host_provided`). No hostname or account value is included. This is gated by the existing server-side `CLIENT_TELEMETRY_ENABLED` parameter and can additionally be disabled locally by setting `SF_TELEMETRY_DISABLE_CONNECTION_SHAPE=true`. The telemetry collection is time-boxed and will be removed in a future release.
1213

1314
- v4.5.0(May 12,2026)
1415
- Fixed `write_pandas` temp stage name collisions (SNOW-3481510). The old PRNG could produce identical name sequences in forked processes (e.g. Notebook kernels), causing `CREATE TEMPORARY STAGE` to fail with "Object already exists".
Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
#!/usr/bin/env python
2+
"""Capture user-supplied connection-identifier provenance for in-band telemetry.
3+
4+
The shape captured here is consumed by ``SnowflakeConnection._log_connection_identifier_shape``
5+
and emitted as a single ``client_connection_identifier_shape`` telemetry event
6+
per successful login. The capture function inspects the raw kwargs passed to
7+
``SnowflakeConnection.__config`` before any normalization (host inference,
8+
account stripping of ``.global``, region extraction from dotted account) runs,
9+
so the shape reflects user intent rather than the final post-normalization
10+
state of the connection.
11+
12+
Removal of this module and the emission it backs is tracked in SNOW-3548350
13+
(target: 2026-11-30).
14+
"""
15+
from __future__ import annotations
16+
17+
from dataclasses import dataclass
18+
from typing import Any, Mapping
19+
20+
from .telemetry import TelemetryField
21+
22+
23+
@dataclass(frozen=True)
24+
class ConnectionIdentifierShape:
25+
"""Provenance of connection-identifier fields the user supplied.
26+
27+
All fields describe what the user supplied at the moment of input — they
28+
reflect intent, not the final post-normalization state of the connection.
29+
30+
- ``account_provided``: the user explicitly set the ``account`` parameter
31+
(via ``connect(account=...)``, kwargs, or ``connections.toml`` merged
32+
into kwargs before ``__config`` runs).
33+
- ``account_with_region``: the raw account string the user typed contained
34+
a dot (e.g. ``"myacct.us-east-1"``), signaling the deprecated
35+
``account.region`` embedded form. Set only on the raw input.
36+
- ``account_org_provided``: the raw account string carried a dash in its
37+
account portion (e.g. ``"myorg-myacct"``), signaling the org-prefixed
38+
form. Region-portion dashes (e.g. the ``-east-`` in
39+
``"myacct.us-east-1"``) are intentionally not counted; only the portion
40+
before the first ``.`` is examined.
41+
- ``region_provided``: the user explicitly set the ``region`` parameter as
42+
a distinct kwarg. A region embedded inside a dotted account string is
43+
NOT ``region_provided``; that's ``account_with_region``.
44+
- ``host_provided``: the user explicitly set the ``host`` parameter.
45+
"""
46+
47+
account_provided: bool = False
48+
account_with_region: bool = False
49+
account_org_provided: bool = False
50+
region_provided: bool = False
51+
host_provided: bool = False
52+
53+
54+
def _is_user_supplied_string(value: Any) -> bool:
55+
"""A kwarg counts as user-supplied iff it's a non-empty string. Non-string
56+
truthy values (e.g. an accidentally-passed ``True`` / ``int``) are not
57+
treated as provided here — the regular ``__config`` validation will warn
58+
or error on them later, and shape capture is best kept conservative."""
59+
return isinstance(value, str) and value != ""
60+
61+
62+
def record_input_shape(kwargs: Mapping[str, Any]) -> ConnectionIdentifierShape:
63+
"""Capture the connection-identifier shape from the raw kwargs that
64+
``SnowflakeConnection.__config`` receives.
65+
66+
Must be invoked before any normalization (the ``setattr`` loop in
67+
``__config``, the ``construct_hostname`` call for host inference, or any
68+
``parse_account`` stripping) — otherwise inferred values are
69+
indistinguishable from user-supplied ones and ``host_provided`` /
70+
``account_provided`` are no longer trustworthy.
71+
"""
72+
account = kwargs.get("account")
73+
region = kwargs.get("region")
74+
host = kwargs.get("host")
75+
76+
account_provided = _is_user_supplied_string(account)
77+
account_with_region = False
78+
account_org_provided = False
79+
if account_provided:
80+
# Only a dot at position > 0 splits the string into account / region;
81+
# a leading dot (pathological input like ``.us-east-1``) leaves the
82+
# whole string as the account portion. Mirrors gosnowflake's
83+
# ``recordAccountShape`` (internal/config/dsn.go), which gates on
84+
# ``i > 0`` so the dash search runs against the full raw value when
85+
# there is no real account/region split.
86+
dot_index = account.find(".")
87+
if dot_index > 0:
88+
account_with_region = True
89+
account_portion = account[:dot_index]
90+
else:
91+
account_portion = account
92+
# ``Contains(accountPortion, "-")`` in Go — any dash anywhere in the
93+
# account portion (including position 0) flips the flag. The
94+
# region-tail dashes are excluded by virtue of being outside
95+
# ``account_portion``, not by a position check.
96+
account_org_provided = "-" in account_portion
97+
98+
return ConnectionIdentifierShape(
99+
account_provided=account_provided,
100+
account_with_region=account_with_region,
101+
account_org_provided=account_org_provided,
102+
region_provided=_is_user_supplied_string(region),
103+
host_provided=_is_user_supplied_string(host),
104+
)
105+
106+
107+
def build_shape_telemetry_message(shape: ConnectionIdentifierShape) -> dict[str, str]:
108+
"""Build the wire-format payload for the ``client_connection_identifier_shape``
109+
in-band telemetry event from a captured ``ConnectionIdentifierShape``.
110+
111+
Hoisted out of the sync / async ``_log_connection_identifier_shape``
112+
emitters so both branches stay in lockstep — the five payload keys
113+
(``account_provided``, ``account_with_region``, ``account_org_provided``,
114+
``region_provided``, ``host_provided``) and their stringified-lowercase
115+
boolean values are byte-identical across sibling drivers and must remain
116+
so. Changing this builder is the only place that affects the wire format.
117+
118+
Booleans are stringified as lowercase ``"true"`` / ``"false"`` (matching
119+
JSON-style boolean text) for cross-driver parity with gosnowflake's
120+
``strconv.FormatBool`` and the JDBC / Node.js siblings.
121+
122+
TODO(SNOW-3548350): remove with the telemetry emission
123+
(target: 2026-11-30).
124+
"""
125+
return {
126+
TelemetryField.KEY_TYPE.value: TelemetryField.CONNECTION_IDENTIFIER_SHAPE.value,
127+
TelemetryField.KEY_ACCOUNT_PROVIDED.value: str(shape.account_provided).lower(),
128+
TelemetryField.KEY_ACCOUNT_WITH_REGION.value: str(
129+
shape.account_with_region
130+
).lower(),
131+
TelemetryField.KEY_ACCOUNT_ORG_PROVIDED.value: str(
132+
shape.account_org_provided
133+
).lower(),
134+
TelemetryField.KEY_REGION_PROVIDED.value: str(shape.region_provided).lower(),
135+
TelemetryField.KEY_HOST_PROVIDED.value: str(shape.host_provided).lower(),
136+
}

src/snowflake/connector/aio/_connection.py

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,14 @@
2424
ProgrammingError,
2525
)
2626

27+
from .._connection_identifier_shape import (
28+
ConnectionIdentifierShape,
29+
build_shape_telemetry_message,
30+
)
2731
from .._query_context_cache import QueryContextCache
2832
from ..compat import IS_LINUX, quote, urlencode
2933
from ..config_manager import CONFIG_MANAGER, _get_default_connection_params
34+
from ..connection import _DISABLE_CONNECTION_SHAPE_ENV
3035
from ..connection import DEFAULT_CONFIGURATION as DEFAULT_CONFIGURATION_SYNC
3136
from ..connection import SnowflakeConnection as SnowflakeConnectionSync
3237
from ..connection import _get_private_bytes_from_file
@@ -783,6 +788,42 @@ async def _log_telemetry_imported_packages(self) -> None:
783788
)
784789
)
785790

791+
async def _log_connection_identifier_shape(self) -> None:
792+
"""Async counterpart to ``SnowflakeConnectionSync._log_connection_identifier_shape``.
793+
794+
The sync implementation reaches for ``self._log_telemetry`` directly,
795+
which is overridden as an awaitable on the async class — so the sync
796+
method's call to ``self._log_telemetry(...)`` would return an
797+
unawaited coroutine here. This override mirrors the sync logic and
798+
awaits the emission instead.
799+
800+
TODO(SNOW-3548350): remove with the telemetry emission
801+
(target: 2026-11-30).
802+
"""
803+
# See sync sibling for the strict (no ``strip``) semantics.
804+
if os.environ.get(_DISABLE_CONNECTION_SHAPE_ENV, "").lower() == "true":
805+
logger.debug(
806+
"connection-identifier-shape telemetry disabled via %s",
807+
_DISABLE_CONNECTION_SHAPE_ENV,
808+
)
809+
return
810+
shape: ConnectionIdentifierShape | None = getattr(
811+
self, "_connection_identifier_shape", None
812+
)
813+
if shape is None:
814+
logger.debug(
815+
"connection-identifier-shape telemetry skipped: shape not captured"
816+
)
817+
return
818+
ts = get_time_millis()
819+
await self._log_telemetry(
820+
TelemetryData.from_telemetry_data_dict(
821+
from_dict=build_shape_telemetry_message(shape),
822+
timestamp=ts,
823+
connection=self,
824+
)
825+
)
826+
786827
async def _next_sequence_counter(self) -> int:
787828
"""Gets next sequence counter. Used internally."""
788829
async with self._lock_sequence_counter:
@@ -1099,6 +1140,7 @@ async def connect(self, **kwargs) -> None:
10991140
await self.__open_connection()
11001141
self._telemetry = TelemetryClient(self._rest)
11011142
await self._log_telemetry_imported_packages()
1143+
await self._log_connection_identifier_shape()
11021144

11031145
def cursor(self, cursor_class: type[CursorCls] = SnowflakeCursor) -> CursorCls:
11041146
logger.debug("cursor")

src/snowflake/connector/connection.py

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,11 @@
3838
from cryptography.hazmat.primitives.asymmetric.rsa import RSAPrivateKey
3939

4040
from . import errors
41+
from ._connection_identifier_shape import (
42+
ConnectionIdentifierShape,
43+
build_shape_telemetry_message,
44+
record_input_shape,
45+
)
4146
from ._query_context_cache import QueryContextCache
4247
from ._utils import (
4348
_DEFAULT_VALUE_SERVER_DOP_CAP_FOR_FILE_TRANSFER,
@@ -161,6 +166,13 @@
161166
MAX_CLIENT_FETCH_THREADS = 1024
162167
DEFAULT_BACKOFF_POLICY = exponential_backoff()
163168

169+
# Local kill switch for the client_connection_identifier_shape in-band
170+
# telemetry event (case-insensitive "true" disables emission). Sibling
171+
# drivers use the same env-var name for the same purpose.
172+
# TODO(SNOW-3548350): remove together with the telemetry emission
173+
# (target: 2026-11-30).
174+
_DISABLE_CONNECTION_SHAPE_ENV = "SF_TELEMETRY_DISABLE_CONNECTION_SHAPE"
175+
164176

165177
def DefaultConverterClass() -> type:
166178
if IS_WINDOWS:
@@ -699,6 +711,7 @@ def __init__(
699711
self._log_telemetry_imported_packages()
700712
self._log_nanoarrow_import()
701713
self._log_minicore_import()
714+
self._log_connection_identifier_shape()
702715
# check SNOW-1218851 for long term improvement plan to refactor ocsp code
703716
atexit.register(self._close_at_exit)
704717

@@ -1655,6 +1668,18 @@ def __open_connection(self):
16551668
def __config(self, **kwargs):
16561669
"""Sets up parameters in the connection object."""
16571670
logger.debug("__config")
1671+
# Capture connection-identifier shape from the raw user-supplied kwargs
1672+
# before any normalization (the setattr loop below, construct_hostname
1673+
# at "account" handling, or parse_account further down) runs.
1674+
# Idempotent: only set on the first __config call so the original
1675+
# user intent isn't overwritten by a later reconfigure with derived
1676+
# values. Consumed by _log_connection_identifier_shape.
1677+
# TODO(SNOW-3548350): remove with the telemetry emission
1678+
# (target: 2026-11-30).
1679+
if getattr(self, "_connection_identifier_shape", None) is None:
1680+
self._connection_identifier_shape: ConnectionIdentifierShape = (
1681+
record_input_shape(kwargs)
1682+
)
16581683
# Handle special cases first
16591684
if "sequence_counter" in kwargs:
16601685
self.sequence_counter = kwargs["sequence_counter"]
@@ -2588,6 +2613,47 @@ def _log_nanoarrow_import(self):
25882613
)
25892614
)
25902615

2616+
def _log_connection_identifier_shape(self):
2617+
"""Emit a single client_connection_identifier_shape in-band telemetry
2618+
record describing which connection-identifier fields the user supplied.
2619+
2620+
Honors a local environment kill switch
2621+
``SF_TELEMETRY_DISABLE_CONNECTION_SHAPE`` (case-insensitive ``"true"``)
2622+
and the post-login ``CLIENT_TELEMETRY_ENABLED`` server parameter (via
2623+
``self.telemetry_enabled`` consulted inside ``_log_telemetry``).
2624+
2625+
TODO(SNOW-3548350): remove together with the supporting
2626+
``ConnectionIdentifierShape`` capture (target: 2026-11-30).
2627+
"""
2628+
# Mirrors gosnowflake's ``strings.EqualFold(os.Getenv(...), "true")``:
2629+
# case-insensitive ``"true"`` only, with NO surrounding-whitespace
2630+
# tolerance. Keeping the comparison strict here means a shell
2631+
# accidentally exporting ``" true "`` (with stray whitespace) leaves
2632+
# the emission enabled instead of silently disabling it, matching the
2633+
# Go / Node.js / JDBC siblings byte-for-byte.
2634+
if os.environ.get(_DISABLE_CONNECTION_SHAPE_ENV, "").lower() == "true":
2635+
logger.debug(
2636+
"connection-identifier-shape telemetry disabled via %s",
2637+
_DISABLE_CONNECTION_SHAPE_ENV,
2638+
)
2639+
return
2640+
shape: ConnectionIdentifierShape | None = getattr(
2641+
self, "_connection_identifier_shape", None
2642+
)
2643+
if shape is None:
2644+
logger.debug(
2645+
"connection-identifier-shape telemetry skipped: shape not captured"
2646+
)
2647+
return
2648+
ts = get_time_millis()
2649+
self._log_telemetry(
2650+
TelemetryData.from_telemetry_data_dict(
2651+
from_dict=build_shape_telemetry_message(shape),
2652+
timestamp=ts,
2653+
connection=self,
2654+
)
2655+
)
2656+
25912657
def _log_telemetry_imported_packages(self) -> None:
25922658
if self._log_imported_packages_in_telemetry:
25932659
# filter out duplicates caused by submodules

src/snowflake/connector/telemetry.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,11 @@ class TelemetryField(Enum):
4444
NANOARROW_IMPORT = "nanoarrow_import"
4545
# multi-statement usage
4646
MULTI_STATEMENT = "client_multi_statement_query"
47+
# Connection-identifier shape (see connection_identifier_shape.py).
48+
# TODO(SNOW-3548350): remove this event type and the five KEY_*_PROVIDED /
49+
# KEY_ACCOUNT_WITH_REGION keys below together with the emission
50+
# (target: 2026-11-30).
51+
CONNECTION_IDENTIFIER_SHAPE = "client_connection_identifier_shape"
4752
# Keys for telemetry data sent through either in-band or out-of-band telemetry
4853
KEY_TYPE = "type"
4954
KEY_SOURCE = "source"
@@ -54,6 +59,17 @@ class TelemetryField(Enum):
5459
KEY_REASON = "reason"
5560
KEY_VALUE = "value"
5661
KEY_EXCEPTION = "exception"
62+
# Payload keys for the client_connection_identifier_shape event. Values
63+
# are stringified booleans ("true" / "false") to match the existing
64+
# in-band telemetry style used by sibling drivers (Go / JDBC). See the
65+
# docstring on ConnectionIdentifierShape for the semantic meaning.
66+
# TODO(SNOW-3548350): remove together with CONNECTION_IDENTIFIER_SHAPE
67+
# above (target: 2026-11-30).
68+
KEY_ACCOUNT_PROVIDED = "account_provided"
69+
KEY_ACCOUNT_WITH_REGION = "account_with_region"
70+
KEY_ACCOUNT_ORG_PROVIDED = "account_org_provided"
71+
KEY_REGION_PROVIDED = "region_provided"
72+
KEY_HOST_PROVIDED = "host_provided"
5773
# Reserved UpperCamelName keys
5874
KEY_ERROR_NUMBER = "ErrorNumber"
5975
KEY_ERROR_MESSAGE = "ErrorMessage"

test/integ/aio_it/test_connection_async.py

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1646,9 +1646,9 @@ async def test_disable_telemetry(conn_cnx, caplog):
16461646
async with conn_cnx() as conn:
16471647
async with conn.cursor() as cur:
16481648
await (await cur.execute("select 1")).fetchall()
1649-
assert (
1650-
len(conn._telemetry._log_batch) == 3
1651-
) # 3 events are `import package`, `fetch first`, `fetch last`
1649+
assert len(conn._telemetry._log_batch) == 4 # 4 events: import package,
1650+
# client_connection_identifier_shape (SNOW-3548350; remove with
1651+
# the emission, target 2026-11-30), fetch first, fetch last
16521652

16531653
assert "POST /telemetry/send" in caplog.text
16541654
caplog.clear()
@@ -1672,7 +1672,10 @@ async def test_disable_telemetry(conn_cnx, caplog):
16721672
# test disable telemetry in the client
16731673
with caplog.at_level(logging.DEBUG):
16741674
async with conn_cnx() as conn:
1675-
assert conn.telemetry_enabled and len(conn._telemetry._log_batch) == 1
1675+
# Bumped from 1 to 2 by SNOW-3351450 (one extra event:
1676+
# client_connection_identifier_shape). Revert to 1 with the
1677+
# emission removal (SNOW-3548350, target 2026-11-30).
1678+
assert conn.telemetry_enabled and len(conn._telemetry._log_batch) == 2
16761679
conn.telemetry_enabled = False
16771680
async with conn.cursor() as cur:
16781681
await (await cur.execute("select 1")).fetchall()

test/integ/test_connection.py

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1869,8 +1869,10 @@ def test_disable_telemetry(conn_cnx, caplog):
18691869
with conn.cursor() as cur:
18701870
cur.execute("select 1").fetchall()
18711871
assert (
1872-
len(conn._telemetry._log_batch) == 5
1873-
) # 4 events are import package, minicore import, fetch first, fetch last
1872+
len(conn._telemetry._log_batch) == 6
1873+
) # 6 events: import package, nanoarrow import, minicore import,
1874+
# client_connection_identifier_shape (SNOW-3548350; remove with
1875+
# the emission, target 2026-11-30), fetch first, fetch last
18741876
assert "POST /telemetry/send" in caplog.text
18751877
caplog.clear()
18761878

@@ -1893,7 +1895,10 @@ def test_disable_telemetry(conn_cnx, caplog):
18931895
# test disable telemetry in the client
18941896
with caplog.at_level(logging.DEBUG):
18951897
with conn_cnx() as conn:
1896-
assert conn.telemetry_enabled and len(conn._telemetry._log_batch) == 3
1898+
# Bumped from 3 to 4 by SNOW-3351450 (one extra event:
1899+
# client_connection_identifier_shape). Revert to 3 with the
1900+
# emission removal (SNOW-3548350, target 2026-11-30).
1901+
assert conn.telemetry_enabled and len(conn._telemetry._log_batch) == 4
18971902
conn.telemetry_enabled = False
18981903
with conn.cursor() as cur:
18991904
cur.execute("select 1").fetchall()

0 commit comments

Comments
 (0)