Skip to content

Commit 07bb29e

Browse files
wrn14897wrn14897
authored
[HDX-3963] Optimize event detail and trace waterfall queries (#2104)
## Summary Optimizes the queries powering the event details panel and trace waterfall chart in the TUI, and improves the trace waterfall UX. Fixes HDX-3963 Linear: https://linear.app/clickhouse/issue/HDX-3963 ### Query optimizations **Full row fetch (`SELECT *`)** — Removed the 1-year `dateRange` from `buildFullRowQuery`. The WHERE clause already uniquely identifies the row, so ClickHouse can use the filter directly without scanning time partitions. Matches the web frontend's `useRowData` pattern in `DBRowDataPanel.tsx`. **Trace waterfall queries** — Replaced raw SQL builders (`buildTraceSpansSql`/`buildTraceLogsSql`) with `renderChartConfig`-based async builders. This enables: - Time partition pruning via a tight ±1h `dateRange` derived from the event timestamp - Materialized field optimization - Query parameterization **Trace span detail fetch** — Replaced raw SQL `SELECT * FROM ... WHERE ... LIMIT 1` with `renderChartConfig` via `buildTraceRowDetailConfig`, omitting `dateRange`/`timestampValueExpression` so ClickHouse uses the WHERE clause directly. **Shared logic** — Extracted all trace config construction into `packages/cli/src/shared/traceConfig.ts`. ### UX improvements **Trace Event Details → dedicated page** — Previously Event Details was rendered inline below the waterfall, consuming screen space. Now: - Waterfall view gets the full terminal height - Press `l`/`Enter` to drill into a span's full Event Details - Press `h`/`Esc` to return to the waterfall - `Ctrl+D/U` scrolls the detail view using full terminal height **Trace waterfall scrolling** — Previously the waterfall truncated at `maxRows` with no way to see remaining spans. Now `j`/`k` scrolls the viewport when the cursor reaches the edge, with a scroll position indicator showing spans above/below. ### Files changed | File | Change | |---|---| | `packages/cli/src/shared/traceConfig.ts` | NEW — shared trace config builders | | `packages/cli/src/api/eventQuery.ts` | Replace raw SQL with `renderChartConfig`; remove date range from full row fetch | | `packages/cli/src/components/TraceWaterfall/TraceWaterfall.tsx` | Split into waterfall + detail views; add scroll support | | `packages/cli/src/components/TraceWaterfall/useTraceData.ts` | Use async builders + parameterized queries | | `packages/cli/src/components/TraceWaterfall/types.ts` | Add `metadata`, `eventTimestamp`, `detailExpanded` props | | `packages/cli/src/components/EventViewer/useKeybindings.ts` | Add `l`/`h` keybindings for trace detail navigation | | `packages/cli/src/components/EventViewer/DetailPanel.tsx` | Thread `metadata`, `eventTimestamp`, `traceDetailExpanded` | | `packages/cli/src/components/EventViewer/EventViewer.tsx` | Add `traceDetailExpanded` state; pass new props |
1 parent a5869f0 commit 07bb29e

9 files changed

Lines changed: 465 additions & 213 deletions

File tree

‎.changeset/optimize-trace-waterfall.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@hyperdx/cli": patch
3+
---
4+
5+
Optimize event detail and trace waterfall queries; add trace detail page and waterfall scrolling

‎packages/cli/src/api/eventQuery.ts‎

Lines changed: 51 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -11,15 +11,19 @@ import { chSqlToAliasMap } from '@hyperdx/common-utils/dist/clickhouse';
1111
import { renderChartConfig } from '@hyperdx/common-utils/dist/core/renderChartConfig';
1212
import type { Metadata } from '@hyperdx/common-utils/dist/core/metadata';
1313
import { DisplayType } from '@hyperdx/common-utils/dist/types';
14-
import type { BuilderChartConfigWithDateRange } from '@hyperdx/common-utils/dist/types';
15-
import SqlString from 'sqlstring';
14+
import type {
15+
BuilderChartConfigWithDateRange,
16+
BuilderChartConfigWithOptDateRange,
17+
} from '@hyperdx/common-utils/dist/types';
1618

1719
import type { SourceResponse } from './client';
18-
import {
19-
getFirstTimestampValueExpression,
20-
getDisplayedTimestampValueExpression,
21-
} from '@/shared/source';
20+
import { getFirstTimestampValueExpression } from '@/shared/source';
2221
import { buildRowDataSelectList } from '@/shared/rowDataPanel';
22+
import {
23+
buildTraceSpansConfig,
24+
buildTraceLogsConfig,
25+
buildTraceRowDetailConfig,
26+
} from '@/shared/traceConfig';
2327
import { buildColumnMap, getRowWhere } from '@/shared/useRowWhere';
2428

2529
export interface SearchQueryOptions {
@@ -104,94 +108,55 @@ export async function buildEventSearchQuery(
104108

105109
// ---- Full row fetch (SELECT *) -------------------------------------
106110

107-
// ---- Trace waterfall query (all spans for a traceId) ----------------
111+
// ---- Trace waterfall queries ----------------------------------------
108112

109113
export interface TraceSpansQueryOptions {
110114
source: SourceResponse;
111115
traceId: string;
116+
dateRange?: [Date, Date];
112117
}
113118

114119
/**
115-
* Build a raw SQL query to fetch all spans for a given traceId.
116-
* Returns columns needed for the waterfall chart.
120+
* Build a query to fetch all spans for a given traceId using
121+
* renderChartConfig. Enables time partition pruning when dateRange
122+
* is provided and materialized field optimisation.
117123
*/
118-
export function buildTraceSpansSql(opts: TraceSpansQueryOptions): {
119-
sql: string;
120-
connectionId: string;
121-
} {
122-
const { source, traceId } = opts;
123-
124-
const db = source.from.databaseName;
125-
const table = source.from.tableName;
126-
const traceIdExpr = source.traceIdExpression ?? 'TraceId';
127-
const spanIdExpr = source.spanIdExpression ?? 'SpanId';
128-
const parentSpanIdExpr = source.parentSpanIdExpression ?? 'ParentSpanId';
129-
const spanNameExpr = source.spanNameExpression ?? 'SpanName';
130-
const serviceNameExpr = source.serviceNameExpression ?? 'ServiceName';
131-
const durationExpr = source.durationExpression ?? 'Duration';
132-
const statusCodeExpr = source.statusCodeExpression ?? 'StatusCode';
133-
134-
const tsExpr = getDisplayedTimestampValueExpression(source);
135-
136-
const cols = [
137-
`${tsExpr} AS Timestamp`,
138-
`${traceIdExpr} AS TraceId`,
139-
`${spanIdExpr} AS SpanId`,
140-
`${parentSpanIdExpr} AS ParentSpanId`,
141-
`${spanNameExpr} AS SpanName`,
142-
`${serviceNameExpr} AS ServiceName`,
143-
`${durationExpr} AS Duration`,
144-
`${statusCodeExpr} AS StatusCode`,
145-
];
146-
147-
const escapedTraceId = SqlString.escape(traceId);
148-
const sql = `SELECT ${cols.join(', ')} FROM ${db}.${table} WHERE ${traceIdExpr} = ${escapedTraceId} ORDER BY ${tsExpr} ASC LIMIT 10000`;
149-
150-
return {
151-
sql,
152-
connectionId: source.connection,
153-
};
124+
export async function buildTraceSpansQuery(
125+
opts: TraceSpansQueryOptions,
126+
metadata: Metadata,
127+
): Promise<ChSql> {
128+
const config = buildTraceSpansConfig(opts);
129+
return renderChartConfig(config, metadata, opts.source.querySettings);
154130
}
155131

156132
/**
157-
* Build a raw SQL query to fetch correlated log events for a given traceId.
158-
* Returns columns matching the SpanRow shape used by the waterfall chart.
159-
* Logs are linked to spans via their SpanId.
133+
* Build a query to fetch correlated log events for a given traceId
134+
* using renderChartConfig.
160135
*/
161-
export function buildTraceLogsSql(opts: TraceSpansQueryOptions): {
162-
sql: string;
163-
connectionId: string;
164-
} {
165-
const { source, traceId } = opts;
166-
167-
const db = source.from.databaseName;
168-
const table = source.from.tableName;
169-
const traceIdExpr = source.traceIdExpression ?? 'TraceId';
170-
const spanIdExpr = source.spanIdExpression ?? 'SpanId';
171-
const bodyExpr = source.bodyExpression ?? 'Body';
172-
const serviceNameExpr = source.serviceNameExpression ?? 'ServiceName';
173-
const sevExpr = source.severityTextExpression ?? 'SeverityText';
174-
175-
const tsExpr = getDisplayedTimestampValueExpression(source);
176-
177-
const cols = [
178-
`${tsExpr} AS Timestamp`,
179-
`${traceIdExpr} AS TraceId`,
180-
`${spanIdExpr} AS SpanId`,
181-
`'' AS ParentSpanId`,
182-
`${bodyExpr} AS SpanName`,
183-
`${serviceNameExpr} AS ServiceName`,
184-
`0 AS Duration`,
185-
`${sevExpr} AS StatusCode`,
186-
];
187-
188-
const escapedTraceId = SqlString.escape(traceId);
189-
const sql = `SELECT ${cols.join(', ')} FROM ${db}.${table} WHERE ${traceIdExpr} = ${escapedTraceId} ORDER BY ${tsExpr} ASC LIMIT 10000`;
136+
export async function buildTraceLogsQuery(
137+
opts: TraceSpansQueryOptions,
138+
metadata: Metadata,
139+
): Promise<ChSql> {
140+
const config = buildTraceLogsConfig(opts);
141+
return renderChartConfig(config, metadata, opts.source.querySettings);
142+
}
190143

191-
return {
192-
sql,
193-
connectionId: source.connection,
194-
};
144+
/**
145+
* Build a query to fetch a single span/log row (SELECT *) from the
146+
* trace waterfall detail panel. Omits dateRange so ClickHouse uses
147+
* the WHERE clause directly.
148+
*/
149+
export async function buildTraceRowDetailQuery(
150+
opts: {
151+
source: SourceResponse;
152+
traceId: string;
153+
spanId?: string;
154+
timestamp: string;
155+
},
156+
metadata: Metadata,
157+
): Promise<ChSql> {
158+
const config = buildTraceRowDetailConfig(opts);
159+
return renderChartConfig(config, metadata, opts.source.querySettings);
195160
}
196161

197162
export interface FullRowQueryOptions {
@@ -242,17 +207,13 @@ export async function buildFullRowQuery(
242207

243208
const selectList = buildRowDataSelectList(source);
244209

245-
// Use a very wide date range — the WHERE clause already uniquely
246-
// identifies the row, so the time range is just a safety net
247-
const now = new Date();
248-
const yearAgo = new Date(now.getTime() - 365 * 24 * 60 * 60 * 1000);
249-
250-
const config: BuilderChartConfigWithDateRange = {
210+
// Omit dateRange and timestampValueExpression — the WHERE clause
211+
// already uniquely identifies the row so ClickHouse can use the
212+
// filter directly without scanning time partitions.
213+
// This matches the web frontend's useRowData in DBRowDataPanel.tsx.
214+
const config: BuilderChartConfigWithOptDateRange = {
251215
connection: source.connection,
252216
from: source.from,
253-
timestampValueExpression:
254-
source.timestampValueExpression ?? 'TimestampTime',
255-
dateRange: [yearAgo, now],
256217
select: selectList,
257218
where: rowWhereResult.where,
258219
limit: { limit: 1 },

‎packages/cli/src/components/EventViewer/DetailPanel.tsx‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,11 @@ import React from 'react';
22
import { Box, Text } from 'ink';
33
import Spinner from 'ink-spinner';
44

5+
import type { Metadata } from '@hyperdx/common-utils/dist/core/metadata';
6+
57
import type { SourceResponse, ProxyClickhouseClient } from '@/api/client';
8+
import { ROW_DATA_ALIASES } from '@/shared/rowDataPanel';
9+
import { getDisplayedTimestampValueExpression } from '@/shared/source';
610
import ColumnValues from '@/components/ColumnValues';
711
import ErrorDisplay from '@/components/ErrorDisplay';
812
import RowOverview from '@/components/RowOverview';
@@ -18,13 +22,15 @@ type DetailPanelProps = {
1822
source: SourceResponse;
1923
sources: SourceResponse[];
2024
clickhouseClient: ProxyClickhouseClient;
25+
metadata: Metadata;
2126
detailTab: DetailTab;
2227
expandedRowData: Record<string, unknown> | null;
2328
expandedRowLoading: boolean;
2429
expandedRowError: Error | null;
2530
expandedTraceId: string | null;
2631
expandedSpanId: string | null;
2732
traceSelectedIndex: number | null;
33+
traceDetailExpanded: boolean;
2834
onTraceSelectedIndexChange: (index: number | null) => void;
2935
detailSearchQuery: string;
3036
focusDetailSearch: boolean;
@@ -52,13 +58,15 @@ export function DetailPanel({
5258
source,
5359
sources,
5460
clickhouseClient,
61+
metadata,
5562
detailTab,
5663
expandedRowData,
5764
expandedRowLoading,
5865
expandedRowError,
5966
expandedTraceId,
6067
expandedSpanId,
6168
traceSelectedIndex,
69+
traceDetailExpanded,
6270
onTraceSelectedIndexChange,
6371
detailSearchQuery,
6472
focusDetailSearch,
@@ -75,6 +83,17 @@ export function DetailPanel({
7583
expandedRow,
7684
onTraceChSqlChange,
7785
}: DetailPanelProps) {
86+
// Extract the event timestamp from the full row data (or the raw
87+
// table row) so we can scope the trace waterfall date range tightly
88+
// for partition pruning.
89+
const eventTimestamp = (() => {
90+
const tsExpr = getDisplayedTimestampValueExpression(source);
91+
const data = expandedRowData ?? expandedFormattedRow?.raw;
92+
if (!data) return undefined;
93+
const val = data[ROW_DATA_ALIASES.TIMESTAMP] ?? data[tsExpr] ?? undefined;
94+
return val != null ? String(val) : undefined;
95+
})();
96+
7897
const hasTrace =
7998
source.kind === 'trace' || (source.kind === 'log' && source.traceSourceId);
8099

@@ -202,12 +221,15 @@ export function DetailPanel({
202221
return (
203222
<TraceWaterfall
204223
clickhouseClient={clickhouseClient}
224+
metadata={metadata}
205225
source={traceSource}
206226
logSource={logSource}
207227
traceId={expandedTraceId}
228+
eventTimestamp={eventTimestamp}
208229
searchQuery={detailSearchQuery}
209230
selectedIndex={traceSelectedIndex}
210231
onSelectedIndexChange={onTraceSelectedIndexChange}
232+
detailExpanded={traceDetailExpanded}
211233
maxRows={waterfallMaxRows}
212234
highlightHint={
213235
expandedSpanId

‎packages/cli/src/components/EventViewer/EventViewer.tsx‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ export default function EventViewer({
7070
const [traceSelectedIndex, setTraceSelectedIndex] = useState<number | null>(
7171
null,
7272
);
73+
const [traceDetailExpanded, setTraceDetailExpanded] = useState(false);
7374
const [timeRange, setTimeRange] = useState<TimeRange>(() => {
7475
const now = new Date();
7576
return { start: new Date(now.getTime() - 60 * 60 * 1000), end: now };
@@ -166,6 +167,7 @@ export default function EventViewer({
166167
showSql,
167168
expandedRow,
168169
detailTab,
170+
traceDetailExpanded,
169171
selectedRow,
170172
scrollOffset,
171173
isFollowing,
@@ -176,7 +178,6 @@ export default function EventViewer({
176178
source,
177179
timeRange,
178180
customSelect,
179-
detailMaxRows,
180181
fullDetailMaxRows,
181182
switchItems,
182183
findActiveIndex,
@@ -191,6 +192,7 @@ export default function EventViewer({
191192
setScrollOffset,
192193
setExpandedRow,
193194
setDetailTab,
195+
setTraceDetailExpanded,
194196
setIsFollowing,
195197
setWrapLines,
196198
setDetailSearchQuery,
@@ -272,13 +274,15 @@ export default function EventViewer({
272274
source={source}
273275
sources={sources}
274276
clickhouseClient={clickhouseClient}
277+
metadata={metadata}
275278
detailTab={detailTab}
276279
expandedRowData={expandedRowData}
277280
expandedRowLoading={expandedRowLoading}
278281
expandedRowError={expandedRowError}
279282
expandedTraceId={expandedTraceId}
280283
expandedSpanId={expandedSpanId}
281284
traceSelectedIndex={traceSelectedIndex}
285+
traceDetailExpanded={traceDetailExpanded}
282286
onTraceSelectedIndexChange={setTraceSelectedIndex}
283287
detailSearchQuery={detailSearchQuery}
284288
focusDetailSearch={focusDetailSearch}

0 commit comments

Comments
 (0)