feat(tools): add VictoriaLogs datasource support#634
feat(tools): add VictoriaLogs datasource support#634kylemclaren wants to merge 3 commits intografana:mainfrom
Conversation
Add 5 new tools for querying VictoriaLogs datasources: - query_victorialogs: Execute LogsQL queries via POST /select/logsql/query - list_victorialogs_field_names: List available fields via /select/logsql/field_names - list_victorialogs_field_values: Get field values with hit counts via /select/logsql/field_values - query_victorialogs_hits: Log volume over time via /select/logsql/hits - query_victorialogs_streams: List matching streams via /select/logsql/streams Key implementation details: - Handles VictoriaLogs NDJSON (newline-delimited JSON) response format - Uses POST with application/x-www-form-urlencoded for query endpoint - Applies limits via LogsQL pipe syntax (| limit N) - Follows existing Loki tool patterns for client construction and transport stack - Enabled by default; can be disabled with --disable-victorialogs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Address review feedback from cursor bot:
- Fix hits endpoint: parse as JSON {"hits":[...]} not NDJSON
- Fix streams endpoint: parse as JSON {"values":[...]} not NDJSON
- Fix field_names endpoint: parse as JSON {"values":[...]} not NDJSON
- Fix field_values endpoint: parse as JSON {"values":[...]} not NDJSON
- Remove unused VictoriaLogsLogEntry type
- Reuse getDefaultTimeRange from loki.go instead of duplicate
Only /select/logsql/query returns NDJSON; all other endpoints return
a single JSON object with wrapped arrays.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use VictoriaLogsHit directly in victoriaLogsHitsResponse since the types are structurally identical, eliminating unnecessary indirection. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
sd2k
left a comment
sd2k
left a comment
There was a problem hiding this comment.
Hi! thanks for the contribution.
I'm endeavouring to avoid adding new tools if possible, since this MCP server's tools already adds 14-15000 tokens. Rather than adding a whole new suite of tools for VL, I think it makes sense to merge this with query_loki and the other Loki related tools. We can do so by abstracting the 'backend' away and switching based on datasource type, and renaming the tools to query_logs etc instead.
Happy to accept a PR that makes that change, if you're up for it.
|
good call @sd2k -- will look into it |
| } | ||
| } | ||
|
|
||
| return &QueryVictoriaLogsHitsResult{Data: resp.Hits}, nil |
There was a problem hiding this comment.
Hits result Data field can be nil unlike other endpoints
Medium Severity
queryVictoriaLogsHits returns resp.Hits directly without nil-checking, so Data can be nil when the response body is empty or the JSON lacks a hits key. This causes JSON serialization to produce {"data":null} instead of {"data":[]}. Every other endpoint in this file ensures a non-nil slice — queryVictoriaLogs has an explicit nil guard, queryVictoriaLogsStreams uses make(), listVictoriaLogsFieldValues uses make(), and listVictoriaLogsFieldNames uses make().
| type VictoriaLogsStream struct { | ||
| Value string `json:"value"` | ||
| Hits int64 `json:"hits"` | ||
| } |
There was a problem hiding this comment.
Duplicate struct types with identical fields
Low Severity
VictoriaLogsStream and VictoriaLogsFieldValue are structurally identical types — both have Value string and Hits int64 with the same JSON tags. One type could be reused for both the streams and field values endpoints, reducing unnecessary duplication. The comment on VictoriaLogsFieldValue even says it's "Used by field_names, field_values, and streams endpoints," yet streams uses its own separate copy.
Additional Locations (1)
|
i have my own fork where i added victorialogs support for my own needs. it was incredibly useful to be able to expose the raw underlying victorialogs endpoint for queries. query_loki is not syntax compatible with victoria logs docs, and be able to do actual queries with matching docs was very useful. it would at least be i think useful to expose query_victorialogs tool, even if other tools are hidden behind abstraction. |
3 participants
Summary
victoriametrics-logs-datasourcetype)query_victorialogs— Execute LogsQL queries (POST/select/logsql/query)list_victorialogs_field_names— List available field nameslist_victorialogs_field_values— Get values for a specific field with hit countsquery_victorialogs_hits— Log volume over time bucketsquery_victorialogs_streams— List matching log streams with hit countsWhy
The existing Loki tools fail against VictoriaLogs datasources because VictoriaLogs uses different API endpoints (
/select/logsql/*vs/loki/api/v1/*), a different query language (LogsQL vs LogQL), and returns NDJSON instead of Loki's wrapped JSON format.Implementation notes
application/x-www-form-urlencodedfor the query endpoint, GET for metadata| limit N) rather than query params--disable-victorialogsTest plan
make lint-jsonschemapasses (no unescaped commas)go build ./...compiles successfullymake test-unitpassesmake test-integrationwith docker-compose (requires VictoriaLogs container)🤖 Generated with Claude Code
Note
Medium Risk
Adds a new datasource integration with multiple HTTP query paths and NDJSON parsing, and enables the tool category by default, which could affect runtime behavior and integration test/dev environments.
Overview
Adds a new VictoriaLogs tool category with five read-only tools (
query_victorialogs, field name/value discovery, hits, and streams) that query Grafana’s datasource proxy using VictoriaLogs/select/logsql/*endpoints and handle NDJSON responses.Wires the category into server startup and CLI (
victorialogsenabled by default,--disable-victorialogsadded), and updates docs/tool table accordingly. Extends the local dev/integration setup by provisioning a VictoriaLogs datasource and adding avictorialogscontainer plus integration tests for the new tools.Written by Cursor Bugbot for commit 2ac6074. This will update automatically on new commits. Configure here.