feat: add CloudWatch Logs Insights query support#706
Open
m-q-t wants to merge 1 commit intografana:mainfrom
Open
feat: add CloudWatch Logs Insights query support#706m-q-t wants to merge 1 commit intografana:mainfrom
m-q-t wants to merge 1 commit intografana:mainfrom
Conversation
Add three new MCP tools for querying CloudWatch Logs via Grafana: - list_cloudwatch_log_groups: discover available log groups - list_cloudwatch_log_group_fields: discover queryable fields - query_cloudwatch_logs: execute Logs Insights queries with async StartQuery/GetQueryResults polling handled internally Key implementation details: - Async polling with exponential backoff (200ms->2s, 30s timeout) - Strips Grafana-internal metadata fields from query results - Custom response parsers for log-groups and log-group-fields APIs which return nested object values, not plain strings - Custom JSON unmarshaler for frame metadata to handle polymorphic "custom" field (object for Logs, string for Metrics) - Registered under existing "cloudwatch" tool category Also improves existing CloudWatch code: - Migrate newCloudWatchClient to use BuildTransport (adds ExtraHeaders support, fixes latent panic on type assertion) - Extract shared postDsQuery and fetchCloudWatchResource helpers to eliminate ~200 lines of duplicated HTTP boilerplate - Add LocalStack test data seeding for log groups/events
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds three new MCP tools for querying CloudWatch Logs via Grafana's datasource proxy API:
list_cloudwatch_log_groups— Discover available log groups (with optional prefix filter and cross-account support)list_cloudwatch_log_group_fields— Discover queryable fields for a log groupquery_cloudwatch_logs— Execute CloudWatch Logs Insights queries with async StartQuery/GetQueryResults polling handled internallyAlso refactors the existing CloudWatch metrics code to share infrastructure with the new Logs tools.
Key implementation details
cloudWatchCustomMetato handle the polymorphicschema.meta.customfield — it's an object ({"Status":"Complete"}) for Logs but a string ("timeSeriesQuery") for Metrics@ptr,*__grafana_internal__) from query results{"value": {"name": "..."}}objects rather than plain stringscloudwatchtool category viaAddCloudWatchToolsRefactors to existing CloudWatch code
newCloudWatchClientto useBuildTransport— replaces manual TLS transport setup with the shared helper, which also addsExtraHeaderssupport and fixes a latent panic from a baretransport.(*http.Transport)type assertion on non-default transportspostDsQueryhelper — shared by metrics queries,startLogsQuery, andgetLogsQueryResults, eliminating duplicated/api/ds/queryPOST boilerplatefetchCloudWatchResourcehelper — shared bylistCloudWatchNamespaces,listCloudWatchMetrics,listCloudWatchDimensions,listCloudWatchLogGroups, andlistCloudWatchLogGroupFields, eliminating ~200 lines of repeated resource API GET boilerplatecloudwatch.gogoes from 108 deletions / 109 additions with fewer total lines despite adding new shared infrastructureVerification against Grafana Cloud
Tested all 9 CloudWatch tools against a production Grafana Cloud instance with real AWS data:
list_cloudwatch_log_groupslist_cloudwatch_log_group_fields/ecs/core-prodlog groupquery_cloudwatch_logsstats/pct/latestaggregations, multi-log-group queriesquery_cloudwatch_logsResourceNotFoundExceptionor empty results with hintsquery_cloudwatchlist_cloudwatch_namespacesfetchCloudWatchResourcerefactor)list_cloudwatch_metricslist_cloudwatch_dimensionsMemoryUtilized, RDSDatabaseConnectionsPlease let me know if you have any questions/feedback.
Best regards,
Maxim
Note
Medium Risk
Adds new CloudWatch Logs Insights query functionality and refactors shared HTTP/transport code used by existing CloudWatch metric tools, so regressions could affect both logs and metrics queries. Risk is mitigated by new unit/integration tests and conservative parsing/backoff behavior.
Overview
Adds CloudWatch Logs Insights support via three new MCP tools:
list_cloudwatch_log_groups,list_cloudwatch_log_group_fields, andquery_cloudwatch_logs, including async StartQuery/GetQueryResults polling with backoff, result limiting, and filtering out Grafana-internal fields.Refactors CloudWatch’s Grafana client to centralize
/api/ds/queryPOSTs (postDsQuery), resource API GETs (fetchCloudWatchResource), and transport creation (switch tomcpgrafana.BuildTransport), plus extends frame schema parsing with a polymorphicschema.meta.customunmarshaller to support both metrics and logs responses.Expands test coverage with new Go unit/integration tests for logs parsing/status handling and adds LocalStack log seeding + Python LLM-loop tests to validate log group discovery and basic log querying.
Written by Cursor Bugbot for commit a37a4ac. This will update automatically on new commits. Configure here.