Logs work good, but for traces, when I'm trying to open any trace, I'm getting an error:
error querying the database: sendQuery: [HTTP 500] response body: "Code: 43. DB::Exception: Function mapKeys requires at least one argument of type Map: In scope WITH '796d4425a821b7a4274751176519003e' AS trace_id, (SELECT min(Start) FROM o11y.otel_traces_trace_id_ts WHERE TraceId = trace_id) AS trace_start, (SELECT max(End) + 1 FROM o11y.otel_traces_trace_id_ts WHERE TraceId = trace_id) AS trace_end SELECT TraceId AS traceID, SpanId AS spanID, ParentSpanId AS parentSpanID, ServiceName AS serviceName, SpanName AS operationName, toUnixTimestamp64Nano(Timestamp) * 0.000001 AS startTime, Duration * 0.000001 AS duration, arrayMap(key -> map('key', key, 'value', SpanAttributes[key]), mapKeys(SpanAttributes)) AS tags, arrayMap(key -> map('key', key, 'value', ResourceAttributes[key]), mapKeys(ResourceAttributes)) AS serviceTags, if(StatusCode IN ('Error', 'STATUS_CODE_ERROR'), 2, 0) AS statusCode, arrayMap((name, timestamp, attributes) -> CAST((name, toString(toUnixTimestamp64Milli(timestamp)), arrayMap(key -> map('key', key, 'value', attributes[key]), mapKeys(attributes))), 'Tuple(name String, timestamp String, fields Array(Map(String, String)))'), Events.Name, Events.Timestamp, Events.Attributes) AS logs, arrayMap((traceID, spanID, attributes) -> CAST((traceID, spanID, arrayMap(key -> map('key', key, 'value', attributes[key]), mapKeys(attributes))), 'Tuple(traceID String, spanID String, tags Array(Map(String, String)))'), Links.TraceId, Links.SpanId, Links.Attributes) AS references, SpanKind AS kind, StatusMessage AS statusMessage, TraceState AS traceState FROM o11y.otel_traces WHERE (traceID = trace_id) AND (Timestamp >= trace_start) AND (Timestamp <= trace_end) LIMIT 1000. (ILLEGAL_TYPE_OF_ARGUMENT) (version 25.6.4.12 (official build)) "
(Trace ID: 4bf2ee5e23dd796218f84e54bb6689e6)
What happened:
o11y data is collected and pushed to the Clickhouse instance through otel collector with JSON type support:
enable_json_type=1. Clickhouse plugin in Grafana is configured with custom settings:Logs work good, but for traces, when I'm trying to open any trace, I'm getting an error:
I'm using Grafana 12.1.1, and plugin version is 4.11.2
What you expected to happen:
Both Logs and Traces work with JSON schemas
How to reproduce it (as minimally and precisely as possible):
Screenshots
Anything else we need to know?:
Environment: