UI SQL autocomplete can issue unbounded getMapKeys scans without a date filter

[lance-elestyle]

What happened?

With field metadata enabled, a UI SQL autocomplete path can send an unbounded
Map-key discovery query to ClickHouse when either dateRange or
timestampValueExpression is missing.

The observed query was:

WITH sampledKeys AS
(
    SELECT getSubcolumn(ResourceAttributes, 'keys') AS keysArr
    FROM otel.otel_logs
    LIMIT 3000000
)
SELECT DISTINCT lowCardinalityKeys(arrayJoin(keysArr)) AS key
FROM sampledKeys
LIMIT 1000
FORMAT JSON

There is no timestamp predicate. On a large, date-partitioned Distributed table
whose older parts are on an S3-backed cold tier, the inner LIMIT does not
provide partition pruning. This query selected approximately 15,085 parts and
1.7 million marks. The coordinator timed out after about 300 seconds, while the
remote queries continued for approximately 760-778 seconds before failing with
broken-pipe errors.

During the incident hour, S3 GET operations increased from a normal baseline of
roughly 10-200 per hour to 58,708.

Expected behavior: UI metadata discovery should not fall back to a raw-table
Map-key scan unless it has a usable timestamp expression and bounded date
range. If those inputs are unavailable, it should skip Map-key expansion, use a
safe default lookback, use configured metadata MVs/text indexes, or require an
explicit opt-in for an unbounded scan.

The team-level Disable Field Metadata setting prevents the query and is an
effective workaround, but it also disables autocomplete and filter metadata.

This appears closely related to #1036. That issue contains the earlier form of
the same no-date-filter lowCardinalityKeys query and was closed as not planned
without a linked PR. #1201 added the sampledKeys pre-limit shown above, but it
does not prevent an all-part scan. #2426 fixed the same class of missing
metadataMVs/dateRange propagation for the MCP describe_source path; the UI
path still appears able to reach the raw fallback without those inputs.

Steps to reproduce

1. Configure a log source backed by a large, date-partitioned ClickHouse table
   containing Map columns such as ResourceAttributes or LogAttributes.
2. Enable field metadata and use a UI SQL editor/autocomplete call site that
   supplies a tableConnection without both a date range and source timestamp
   expression (for example, a dashboard filter or raw SQL chart editor).
3. Open/focus the editor so that field autocomplete loads.
4. Inspect system.query_log for a query containing sampledKeys,
   getSubcolumn(..., 'keys'), and lowCardinalityKeys.
5. Observe that the generated query has no timestamp WHERE clause and selects
   parts across the full retention period.

The behavior is data-size dependent: it may appear harmless on small or
filesystem-cached tables, but becomes expensive on large cold-tier tables.

How are you running HyperDX?

• HyperDX Docker image: hyperdx/hyperdx:2.33.0
• Bring-your-own ClickHouse: 26.7.3.19
• Three-replica ClickHouse cluster on Kubernetes
• Distributed OTel logs table with older parts on an S3-backed cold tier
• metadataMaterializedViews were not used by the observed fallback query

The relevant behavior is also present on the current main branch as of
commit 808b345313485977b1a0c32629b418e6389adba2.

Where does it show up?

The HyperDX web UI. The ClickHouse query was issued through the browser-facing
/clickhouse-proxy endpoint with user agent hyperdx 2.33.0. No /mcp request
was present around the query start time.

The source path appears to be:

SQLInlineEditor / SQLEditor autocomplete
  -> useMultipleAllFields()
  -> metadata.getAllFields()
  -> metadata.getMapKeys()
  -> raw sampledKeys query

Relevant code:

• SQLInlineEditor passes optional dateRange and
  source?.timestampValueExpression to useMultipleAllFields:
  

  hyperdx/packages/app/src/components/SQLEditor/SQLInlineEditor.tsx

  Lines 96 to 109 in e73af38

  	dateRange,
  	sourceId,
  	intersectFields,
  	}: SQLInlineEditorProps & TableConnectionChoice) {
  	const { colorScheme } = useMantineColorScheme();
  	const _tableConnections = tableConnection
  	? [tableConnection]
  	: tableConnections;
  	const { data: source } = useSource({ id: sourceId });
  	const { data: fields } = useMultipleAllFields(_tableConnections ?? [], {
  	dateRange,
  	timestampValueExpression: source?.timestampValueExpression,
  	intersect: intersectFields,
  	});

• useMultipleAllFields accepts both values as optional and calls
  getAllFields without guarding against a missing metadata scope:
  

  hyperdx/packages/app/src/hooks/useMetadata.tsx

  Lines 226 to 264 in e73af38

  	export function useMultipleAllFields(
  	tableConnections: TableConnection[],
  	options?: Partial<UseQueryOptions<Field[]>> & {
  	dateRange?: [Date, Date];
  	timestampValueExpression?: string;
  	// Return only fields present in EVERY table connection instead of the
  	// union. Use for a shared expression (e.g. a chart-level Group By over
  	// multiple series) that must be valid against all of them — the union
  	// would offer fields that exist in one table but not another.
  	intersect?: boolean;
  	},
  	) {
  	const metadata = useMetadataWithSettings();
  	const { data: me, isFetched } = api.useMe();
  	const {
  	dateRange,
  	timestampValueExpression,
  	intersect,
  	enabled: enabledOption = true,
  	...queryOptions
  	} = options ?? {};
  	return useQuery<Field[]>({
  	queryKey: [
  	'useMetadata.useMultipleAllFields',
  	...tableConnections.map(tc => ({ ...tc })),
  	dateRange ? [dateRange[0].getTime(), dateRange[1].getTime()] : undefined,
  	timestampValueExpression,
  	intersect ?? false,
  	],
  	queryFn: async () => {
  	const team = me?.team;
  	if (team?.fieldMetadataDisabled) {
  	return [];
  	}
  	const promiseResults = await Promise.allSettled(
  	tableConnections.map(tc =>
  	metadata.getAllFields({ ...tc, dateRange, timestampValueExpression }),
  	),

• getAllFields forwards the optional values into getMapKeys:
  

  hyperdx/packages/common-utils/src/core/metadata.ts

  Lines 1422 to 1486 in e73af38

  	async getAllFields({
  	databaseName,
  	tableName,
  	connectionId,
  	metricName,
  	metadataMVs,
  	dateRange,
  	timestampValueExpression,
  	}: TableConnection & {
  	dateRange?: [Date, Date];
  	timestampValueExpression?: string;
  	}) {
  	const fields: Field[] = [];
  	const columns = await this.getColumns({
  	databaseName,
  	tableName,
  	connectionId,
  	});
  	for (const c of columns) {
  	// HDX-2480 delete condition below to reenable json filters
  	if (c.type === 'JSON') continue;
  	fields.push({
  	path: [c.name],
  	type: c.type,
  	jsType: convertCHDataTypeToJSType(c.type),
  	});
  	}
  	const mapColumns =
  	filterColumnMetaByType(columns, [JSDataType.Map, JSDataType.JSON]) ?? [];
  	await Promise.all(
  	mapColumns.map(async column => {
  	if (convertCHDataTypeToJSType(column.type) === JSDataType.JSON) {
  	const paths = await this.getJSONKeys({
  	databaseName,
  	tableName,
  	column: column.name,
  	connectionId,
  	metricName,
  	dateRange,
  	timestampValueExpression,
  	});
  	for (const path of paths) {
  	fields.push({
  	path: [column.name, path.key],
  	type: path.chType,
  	jsType: convertCHDataTypeToJSType(path.chType),
  	});
  	}
  	return;
  	}
  	const keys = await this.getMapKeys({
  	databaseName,
  	tableName,
  	column: column.name,
  	connectionId,
  	metricName,
  	metadataMVs,
  	dateRange,
  	timestampValueExpression,
  	});

• getMapKeys creates a time condition only when both values are present;
  otherwise the raw query is generated without WHERE:
  

  hyperdx/packages/common-utils/src/core/metadata.ts

  Lines 854 to 873 in e73af38

  	const timeFilterCondition =
  	dateRange && timestampValueExpression
  	? await timeFilterExpr({
  	connectionId,
  	databaseName,
  	tableName,
  	dateRange,
  	dateRangeStartInclusive: true,
  	dateRangeEndInclusive: true,
  	timestampValueExpression,
  	metadata: this,
  	})
  	: null;
  	const whereConditions: ChSql[] = [
  	...(metricName ? [chSql`MetricName=${{ String: metricName }}`] : []),
  	...(timeFilterCondition ? [timeFilterCondition] : []),
  	];
  	const where = whereConditions.length
  	? chSql`WHERE ${concatChSql(' AND ', ...whereConditions)}`
  	: '';

The filters sidebar path itself passes both values, so the issue is not that all
field-metadata calls are unbounded. It is specifically the fail-open behavior
when a UI caller does not provide the complete metadata scope.

Logs

Sanitized system.query_log details for one occurrence:

event_time:           2026-08-31 01:37:08 UTC
interface:            HTTP
http_user_agent:      hyperdx 2.33.0
is_initial_query:     1
selected_parts:       15085
selected_marks:       1699997
coordinator_duration: 300376 ms
coordinator_result:   socket receive timeout from a replica

remote query 1:       760019 ms, broken pipe
remote query 2:       778333 ms, broken pipe

The application access log shows multiple /clickhouse-proxy requests in the
same second as the ClickHouse query start and no /mcp request in the
surrounding nine-minute window.

Suggested regression coverage:

1. A UI field-autocomplete query must not call raw getMapKeys until both a
   valid dateRange and timestampValueExpression are available.
2. getMapKeys should not silently generate a raw query without a time filter
   unless the caller explicitly opts into an unbounded scan.
3. Propagate React Query's AbortSignal through getAllFields to getMapKeys
   so abandoned metadata requests are cancelled.