feat(dbi): Add SQL Server Database Insights support - #2234
Merged
Conversation
- Enable 26 additional metrics for RDS parity (43 counter + 7 DB Load) - Configure query plan caching (1000 plans, 1hr TTL, 900KB max) - Enable query sample and top query collection - Add sample configuration files for Linux - Platform coverage: Linux 94%, Windows 100% Metrics include: - Performance & Resource (4): blocked processes, deadlocks, memory - Page & Buffer Pool (3): lookups, free list stalls - Wait Statistics (3): latch waits, lock timeouts - Transaction & Log (4): active transactions, log metrics - Database I/O (4): io, latency, operations, Resource Governor queue - Connection & Workload (3): login/logout rates, session count - Index & Query (3): index searches, full scans, errors - TempDB (2): space usage, version store New metrics: - sqlserver.database.io.stall_queued (Resource Governor I/O queue wait) - sqlserver.session.count (sessions by state)
- Fix console detection: set db.system.name in resource attributes - Fix query sample filter: use false for SQL Server/MySQL - Add DB Load connector configuration (7 metrics) - Add Top SQL signal-to-metrics connector - Add transform processor for start time fix - Configure full DBI metrics pipeline DB Load metrics: - active_sessions.count (total sessions) - active_sessions.by_wait (by wait type) - active_sessions.by_user (by username) - active_sessions.by_db (by database) - active_sessions.by_sql (by SQL hash) - active_sessions.by_sql_wait (by SQL + wait) - active_sessions.by_host (by host) Fixes: - Console now detects SQL Server instances correctly - Query samples flow to CloudWatch Logs
Component Registration: - Register SQL Server receiver in default components - Update common component utilities - Update configuration schema - Add platform-specific tests Changes: - Add sqlserverreceiver to default receiver list - Update component factory to include SQL Server - Add SQL Server to configuration validation - Add test coverage for new components
- Add SQL Server DBI to combined EC2 sample configs - Add SQL Server DBI to combined EKS sample configs - Update go.mod/go.sum with SQL Server receiver dependencies - Use local receiver via replace directive for development Configuration examples show: - SQL Server endpoint (localhost:1433) - Monitoring user credentials (passfile) - Instance naming - Integration with other DBI engines (PostgreSQL, MySQL)
SQL Server Detector: - Detects SQL Server process (sqlservr, sqlservr.exe) - Extracts port from command line flags (-p) or environment (MSSQL_TCP_PORT) - Defaults to port 1433 - Supports custom installation paths - Platform: Linux and Windows Port Extraction: - Parses -p flag from command line arguments - Reads MSSQL_TCP_PORT environment variable - Command line takes precedence over environment - Falls back to default port 1433 Detection logic: - Searches for sqlservr process in ps output - Excludes sqlcmd (client tool, not server) - Handles both default and custom SQL Server installations - Works on Linux and Windows platforms Files: - internal/detector/sqlserver/sqlserver.go - Main detector - internal/detector/sqlserver/extract/port.go - Port extraction logic - internal/detector/metadata.go - Register SQL Server detector
- Add SQL Server process scraping to hostmetrics pipeline - Configure sqlservr.* process pattern for SQL Server detection - Add merge rule tests for SQL Server database_insights config - Enable process metrics (CPU, memory, threads) for SQL Server instances
…ort for tracking SQL Server sessions by application name, matching PostgreSQL's existing application.name dimension for DBI workload analysis. Changes: - Add sqlserver.active_sessions.by_app connector metric with sqlserver.application.name attribute dimension - Add replace_match transform for sqlserver.application.name to set 'unknown' default for NULL values - Update tests to expect 8 connector metrics (was 7) and 6 transform statements (was 5) - Ensure 13 Windows-only Performance Monitor metrics are correctly disabled on Linux (lock.wait_time.avg, page metrics, transaction metrics, and transaction_log metrics) - Add SQLServerKey constant and DatabaseInsightsSqlServerKey to common package - Integrate SQL Server receiver creation in databaseinsights translator - Update go.mod to point to committed sqlserverreceiver with application.name support (v0.0.0-20260729175257-2c50db1bb14e) - Regenerate all sample YAML configs with new metrics and transforms The application.name attribute enables DBAs to: - Identify which applications are generating database load - Track performance and resource consumption per application - Correlate application behavior with database metrics Testing: - All unit tests passing for connectors, processors, receivers - All pipeline integration tests passing - Linux config verified with Windows-only metrics disabled - Windows config verified with all 13 Performance Monitor metrics enabled
Adjust SQL Server query sample configuration to optimize resource usage: - Increase MaxRowsPerQuery from 500 to 5000 (aligns with PostgreSQL) - Decrease MaxQuerySampleCount from 5000 to 1000 (conservative limit) This balances query plan collection with CloudWatch Logs limits while maintaining the existing gzip+base64 compression for XML ShowPlan data.
Adjust MySQL query sample configuration to optimize resource usage: - Increase MaxRowsPerQuery from 500 to 5000 (aligns with PostgreSQL) - Decrease MaxQuerySampleCount from 5000 to 1000 (conservative limit) This balances query plan collection with CloudWatch Logs limits.
Merge upstream main branch to incorporate latest changes including: - GitHub workflow updates - PostgreSQL detector additions - Release notes updates Resolved conflicts in: - cmd/workload-discovery/discovery.go - internal/detector/metadata.go
…registry On Windows, SQL Server named instances store their TCP port in the registry rather than passing it via command-line flags. The workload discovery tool was falling back to the default port 1433 for all instances, causing incorrect port reporting for named instances. Add a Windows-specific registry port extractor that: 1. Extracts the instance name from the process command line (-s flag) 2. Resolves the instance name to its registry ID via Instance Names\SQL 3. Reads the TCP port from SuperSocketNetLib\Tcp\IPAll (static or dynamic) The extractor chain on Windows is now: registry -> cmdline -> env -> default. On Linux/other platforms, behavior is unchanged: cmdline -> env -> default.
Windows SQL Server writes its ERRORLOG in UTF-16LE encoding, causing server log entries to appear as garbled null-interleaved text in CloudWatch Logs (e.g. 'S\u0000t\u0000a\u0000r\u0000t\u0000i\u0000n\u0000g'). Add a runtime.GOOS check to set the filelog receiver encoding to 'utf-16le' when running on Windows. On Linux, SQL Server writes ERRORLOG in UTF-8, so no encoding override is needed. Verified on both instances: - Windows (i-01992cda09d257426): server logs now appear as readable text - Linux (i-0c4670d5ab711c0dd): server logs already UTF-8, no issue
…386b) Update go.mod to point to the latest commit on the upstream dbi-mysql branch which includes PR #617 (InnoDB deadlocks and transaction metrics).
Update mysqlreceiver to latest dbi-mysql branch commit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add complete SQL Server Database Insights implementation with workload detection and application-level session tracking.
Overview
This PR adds SQL Server support to Database Insights, enabling monitoring of SQL Server instances on both Windows and Linux with automatic workload detection.
Key Features
SQL Server Receiver Integration
Windows-Specific Support
Application-Level Monitoring
sqlserver.application.nameattributesqlserver.active_sessions.by_appconnector metricWorkload Detection
Configuration & Tuning
Dependencies
go.modto point to sqlserverreceiver with application.name supportgo.modfor latest mysqlreceiver from dbi-mysql branchTesting
Components Added
translator/translate/otel/receiver/sqlserver/)internal/detector/sqlserver/)