-
Notifications
You must be signed in to change notification settings - Fork 1.5k
feat(tools/mysql): add list-table-stats-tool to list table statistics in MySQL and Cloud SQL MySQL source. #2938
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 6 commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
98d8842
Added list_table_stats tool for MySQL
sumeetdhingra-google f63fe80
Merge branch 'googleapis:main' into main
sumeetdhingra-google be4c7c3
changes added to verify connected db
sumeetdhingra-google 2f93a19
minor fixes
sumeetdhingra-google 747fbb0
updated tool defination
sumeetdhingra-google 5d5b35d
Merge branch 'googleapis:main' into main
sumeetdhingra-google ea9307d
Merge branch 'googleapis:main' into main
sumeetdhingra-google 9391180
updated validation block and corrected indentation
sumeetdhingra-google 17cd410
Merge branch 'googleapis:main' into main
sumeetdhingra-google 223a00a
Merge branch 'googleapis:main' into main
sumeetdhingra-google 260f2bf
Fixed errors and tool definition
sumeetdhingra-google df6adaf
Merge branch 'googleapis:main' into main
sumeetdhingra-google 578e37b
Applied Suggested changes
sumeetdhingra-google 18c48b7
added RunMySQLListTableStatsTest to cloud-sql-mysql
sumeetdhingra-google 4a62f20
Merge branch 'googleapis:main' into main
sumeetdhingra-google 4503986
added annotations
sumeetdhingra-google 0130402
Merge branch 'main' into main
Yuan325 31ad5b1
Update imports.go
Yuan325 ae3fdde
moved advance usage to about in docs file
sumeetdhingra-google 32d2934
Merge branch 'main' into main
sumeetdhingra-google ffece2c
Merge branch 'main' into main
sumeetdhingra-google 178f364
changes to merge with new repo name mcp-toolbox
sumeetdhingra-google 688ed1a
corrected db name in Cloud SQL MySQL Integration test
sumeetdhingra-google abf0b3e
Merge branch 'main' into main
sumeetdhingra-google 600dd8c
minor changes in format
sumeetdhingra-google 31913b6
Merge branch 'main' into main
sumeetdhingra-google 543ebca
fixed integration test
sumeetdhingra-google 0dd4d0c
fixed indentation
sumeetdhingra-google 4a0939e
Merge branch 'main' into main
Yuan325 b27a38f
Merge branch 'main' into main
Yuan325 429a929
Merge branch 'main' into main
Yuan325 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
75 changes: 75 additions & 0 deletions
75
docs/en/integrations/mysql/tools/mysql-list-table-stats.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| "--- | ||
| title: ""mysql-list-table-stats"" | ||
| type: docs | ||
| weight: 1 | ||
| description: > | ||
| A ""mysql-list-table-stats"" tool report table statistics including table size, total latency, rows read, rows written, read and write latency for entire instance, a specified database, or a specified table. | ||
|
shuzhou-gc marked this conversation as resolved.
Outdated
|
||
| --- | ||
|
|
||
| ## About | ||
|
|
||
| A `mysql-list-table-stats` tool provides a table level statistics by checking table size, estimated rows count, total latency, rows fetched, rows inserted, rows updated, rows deleted, number of IO reads and IO latency, number of IO writes and IO write latency, number of IO misc operations and IO misc latency, IO latency of reads and writes on the table. | ||
|
|
||
| `mysql-list-table-stats` outputs detailed overview of table level resource consumption including estimated row counts, table size, a complete breakdown of CRUD activity (rows fetched, inserted, updated, and deleted) and table level IO statistics such as total latency, read latency, write latency and misc latency. The output is a JSON formatted array of the top 10 MySQL tables ranked by total latency. | ||
|
|
||
|
shuzhou-gc marked this conversation as resolved.
|
||
| This tool takes 4 optional input parameters: | ||
|
|
||
| - `table_schema` (optional): The database where table stats check is to be | ||
| executed. Check all tables visible to the current database if not specified. | ||
| - `table_name` (optional): Name of the table to be checked. Check all tables | ||
| visible to the current user if not specified. | ||
| - `sort_by` (optional): The column to sort by. Valid values are `row_count`, `rows_fetched`, `rows_inserted`, `rows_updated`, `rows_deleted`, `total_latency_secs` (defaults to `total_latency_secs`) | ||
|
shuzhou-gc marked this conversation as resolved.
|
||
| - `limit` (optional): Max rows to return, default 10. | ||
|
|
||
| ## Compatible Sources | ||
|
|
||
| {{< compatible-sources others="integrations/cloud-sql-mysql">}} | ||
|
|
||
| ## Example | ||
|
|
||
| ```yaml | ||
| kind: tools | ||
| name: list_table_stats | ||
| type: mysql-list-table-stats | ||
| source: my-mysql-instance | ||
| description: Display table statistics including table size, total latency, rows read, rows written, read and write latency for entire instance, a specified database, or a specified table. Specifying a database name or table name filters the output to that specific db or table. Results are limited to 10 by default, with a maximum allowable limit of 1000. | ||
|
shuzhou-gc marked this conversation as resolved.
Outdated
|
||
| ``` | ||
| The response is a json array with the following fields: | ||
| ```json | ||
| [ | ||
| { | ||
| ""table_schema"": ""The schema/database this table belongs to"", | ||
| ""table_name"": ""Name of this table"", | ||
| ""size_MB"": ""Size of the table data in MB"", | ||
| ""row_count"": ""Number of rows in the table"", | ||
| ""total_latency_secs"": ""total latency in secs"", | ||
| ""rows_fetched"": ""total number of rows fetched"", | ||
| ""rows_inserted"": ""total number of rows inserted"", | ||
| ""rows_updated"": ""total number of rows updated"", | ||
| ""rows_deleted"": ""total number of rows deleted"", | ||
| ""io_reads"": ""total number of io read requests"", | ||
| ""io_read_latency"": ""io read latency in seconds"", | ||
| ""io_write_latency"": ""io write latency in seconds"", | ||
| ""io_misc_latency"": ""io misc latency in seconds"", | ||
| } | ||
| ] | ||
| ``` | ||
|
|
||
| ## prerequisite | ||
|
|
||
| - `performance_schema` should be turned ON for this tool to work. | ||
|
|
||
| ## Reference | ||
|
|
||
| | **field** | **type** | **required** | **description** | | ||
| |-------------|:--------:|:------------:|----------------------------------------------------| | ||
| | type | string | true | Must be ""mysql-list-table-stats"". | | ||
| | source | string | true | Name of the source the SQL should execute on. | | ||
| | description | string | true | Description of the tool that is passed to the LLM. | | ||
|
|
||
|
|
||
| ## Use Cases | ||
|
|
||
| - **Finding hottest tables**: Identify tables with highest total latency, read or writes based on the `sort_by` column. | ||
| - **Finding tables with most reads**: Identify tables with highest reads by sorting on `rows_fetched`. | ||
| - **Monitoring growth**: Track `row_count` and `size_MB` of table over time to estimate growth." | ||
|
sumeetdhingra-google marked this conversation as resolved.
Outdated
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
210 changes: 210 additions & 0 deletions
210
internal/tools/mysql/mysqllisttablestats/mysqllisttablestats.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,210 @@ | ||
| // Copyright 2026 Google LLC | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package mysqllisttablestats | ||
|
|
||
| import ( | ||
| "context" | ||
| "database/sql" | ||
| "fmt" | ||
| "net/http" | ||
| "os" | ||
|
|
||
| yaml "github.com/goccy/go-yaml" | ||
| "github.com/googleapis/genai-toolbox/internal/embeddingmodels" | ||
| "github.com/googleapis/genai-toolbox/internal/sources" | ||
| "github.com/googleapis/genai-toolbox/internal/tools" | ||
| "github.com/googleapis/genai-toolbox/internal/util" | ||
| "github.com/googleapis/genai-toolbox/internal/util/parameters" | ||
| ) | ||
|
|
||
| const resourceType string = "mysql-list-table-stats" | ||
|
|
||
| const listTableStatsStatement = ` | ||
| SELECT | ||
| t.table_schema AS 'table_schema', | ||
| t.table_name AS 'table_name', | ||
| ROUND((t.data_length + t.index_length)/1024/1024,2) AS 'size_MB', | ||
| t.TABLE_ROWS AS 'row_count', | ||
| ROUND(ts.total_latency / 1000000000000, 2) AS 'total_latency_secs', | ||
| ts.rows_fetched AS 'rows_fetched', | ||
| ts.rows_inserted AS 'rows_inserted', | ||
| ts.rows_updated AS 'rows_updated', | ||
| ts.rows_deleted AS 'rows_deleted', | ||
| ts.io_read_requests as 'io_reads' , | ||
| ROUND(ts.io_read_latency / 1000000000000, 2) AS 'io_read_latency', | ||
| ts.io_write_requests AS 'io_writes', | ||
| ROUND(ts.io_write_latency / 1000000000000, 2) AS 'io_write_latency', | ||
| ts.io_misc_requests AS 'io_misc_requests', | ||
| ROUND(ts.io_misc_latency / 1000000000000, 2) AS 'io_misc_latency' | ||
| FROM | ||
| information_schema.tables AS t | ||
| INNER JOIN | ||
| sys.x$schema_table_statistics AS ts | ||
| ON (t.table_schema = ts.table_schema AND t.table_name = ts.table_name) | ||
| WHERE | ||
| t.table_schema NOT IN ('sys', 'information_schema', 'mysql', 'performance_schema') | ||
| AND (COALESCE(?, '') = '' OR t.table_schema = ?) | ||
| AND (COALESCE(?, '') = '' OR t.table_name = ?) | ||
| ORDER BY | ||
| CASE | ||
| WHEN ? = 'row_count' THEN row_count | ||
| WHEN ? = 'rows_fetched' THEN rows_fetched | ||
| WHEN ? = 'rows_inserted' THEN rows_inserted | ||
| WHEN ? = 'rows_updated' THEN rows_updated | ||
| WHEN ? = 'rows_deleted' THEN rows_deleted | ||
| ELSE ts.total_latency | ||
| END DESC | ||
| LIMIT ?; | ||
| ` | ||
|
|
||
| func init() { | ||
| if !tools.Register(resourceType, newConfig) { | ||
|
shuzhou-gc marked this conversation as resolved.
Outdated
shuzhou-gc marked this conversation as resolved.
Outdated
|
||
| panic(fmt.Sprintf("tool type %q already registered", resourceType)) | ||
| } | ||
| } | ||
|
|
||
| func newConfig(ctx context.Context, name string, decoder *yaml.Decoder) (tools.ToolConfig, error) { | ||
| actual := Config{Name: name} | ||
| if err := decoder.DecodeContext(ctx, &actual); err != nil { | ||
| return nil, err | ||
| } | ||
| return actual, nil | ||
| } | ||
|
|
||
| type compatibleSource interface { | ||
| MySQLPool() *sql.DB | ||
| RunSQL(context.Context, string, []any) (any, error) | ||
| } | ||
|
|
||
| type Config struct { | ||
| Name string `yaml:"name" validate:"required"` | ||
| Type string `yaml:"type" validate:"required"` | ||
| Source string `yaml:"source" validate:"required"` | ||
| Description string `yaml:"description" validate:"required"` | ||
| AuthRequired []string `yaml:"authRequired"` | ||
|
Yuan325 marked this conversation as resolved.
Outdated
|
||
| } | ||
|
|
||
| // validate interface | ||
| var _ tools.ToolConfig = Config{} | ||
|
|
||
| func (cfg Config) ToolConfigType() string { | ||
| return resourceType | ||
| } | ||
|
|
||
| func (cfg Config) Initialize(srcs map[string]sources.Source) (tools.Tool, error) { | ||
| allParameters := parameters.Parameters{ | ||
| parameters.NewStringParameterWithDefault("table_schema", "", "(Optional) The database where statistics is to be executed. Check all tables visible to the current user if not specified"), | ||
|
Yuan325 marked this conversation as resolved.
|
||
| parameters.NewStringParameterWithDefault("table_name", "", "(Optional) Name of the table to be checked. Check all tables visible to the current user if not specified."), | ||
| parameters.NewStringParameterWithDefault("sort_by", "", "(Optional) The column to sort by"), | ||
| parameters.NewIntParameterWithDefault("limit", 10, "(Optional) Max rows to return, default is 10"), | ||
|
shuzhou-gc marked this conversation as resolved.
|
||
| } | ||
| mcpManifest := tools.GetMcpManifest(cfg.Name, cfg.Description, cfg.AuthRequired, allParameters, nil) | ||
|
|
||
| // finish tool setup | ||
| t := Tool{ | ||
| Config: cfg, | ||
| allParams: allParameters, | ||
| manifest: tools.Manifest{Description: cfg.Description, Parameters: allParameters.Manifest(), AuthRequired: cfg.AuthRequired}, | ||
| mcpManifest: mcpManifest, | ||
| } | ||
| return t, nil | ||
| } | ||
|
|
||
| // validate interface | ||
| var _ tools.Tool = Tool{} | ||
|
|
||
| type Tool struct { | ||
| Config | ||
| allParams parameters.Parameters `yaml:"parameters"` | ||
| manifest tools.Manifest | ||
| mcpManifest tools.McpManifest | ||
| } | ||
|
|
||
| func (t Tool) Invoke(ctx context.Context, resourceMgr tools.SourceProvider, params parameters.ParamValues, accessToken tools.AccessToken) (any, util.ToolboxError) { | ||
| source, err := tools.GetCompatibleSource[compatibleSource](resourceMgr, t.Source, t.Name, t.Type) | ||
| if err != nil { | ||
| return nil, util.NewClientServerError("source used is not compatible with the tool", http.StatusInternalServerError, err) | ||
| } | ||
|
|
||
| paramsMap := params.AsMap() | ||
|
|
||
| table_schema, ok := paramsMap["table_schema"].(string) | ||
| if !ok { | ||
| return nil, util.NewAgentError("invalid 'table_schema' parameter; expected a string", nil) | ||
| } | ||
| table_name, ok := paramsMap["table_name"].(string) | ||
| if !ok { | ||
| return nil, util.NewAgentError("invalid 'table_name' parameter; expected a string", nil) | ||
| } | ||
| sort_by, ok := paramsMap["sort_by"].(string) | ||
| if !ok { | ||
| return nil, util.NewAgentError("invalid 'sort_by' parameter; expected a string", nil) | ||
| } | ||
| limit, ok := paramsMap["limit"].(int) | ||
| if !ok { | ||
| return nil, util.NewAgentError("invalid 'limit' parameter; expected an integer", nil) | ||
| } | ||
|
sumeetdhingra-google marked this conversation as resolved.
|
||
|
|
||
| // Log the query executed for debugging. | ||
| logger, err := util.LoggerFromContext(ctx) | ||
| if err != nil { | ||
| return nil, util.NewClientServerError("error getting logger", http.StatusInternalServerError, err) | ||
| } | ||
| logger.DebugContext(ctx, fmt.Sprintf("executing `%s` tool query: %s", resourceType, listTableStatsStatement)) | ||
| sliceParams := []any{table_schema, table_schema, table_name, table_name, sort_by, sort_by, sort_by, sort_by, sort_by, limit} | ||
| resp, err := source.RunSQL(ctx, listTableStatsStatement, sliceParams) | ||
| if err != nil { | ||
| return nil, util.ProcessGeneralError(err) | ||
| } | ||
| connected_schema := os.Getenv("MYSQL_DATABASE") | ||
|
shuzhou-gc marked this conversation as resolved.
Outdated
|
||
| if table_schema != connected_schema && table_schema != "" { | ||
| err := fmt.Errorf("error: connected schema '%s' does not match queried schema '%s'", connected_schema, table_schema) | ||
| return nil, util.NewClientServerError("error getting logger", http.StatusInternalServerError, err) | ||
|
shuzhou-gc marked this conversation as resolved.
Outdated
|
||
| } | ||
|
|
||
| return resp, nil | ||
|
sumeetdhingra-google marked this conversation as resolved.
Outdated
|
||
| } | ||
|
sumeetdhingra-google marked this conversation as resolved.
|
||
|
|
||
| func (t Tool) EmbedParams(ctx context.Context, paramValues parameters.ParamValues, embeddingModelsMap map[string]embeddingmodels.EmbeddingModel) (parameters.ParamValues, error) { | ||
| return parameters.EmbedParams(ctx, t.allParams, paramValues, embeddingModelsMap, nil) | ||
| } | ||
|
|
||
| func (t Tool) Manifest() tools.Manifest { | ||
| return t.manifest | ||
| } | ||
|
|
||
| func (t Tool) McpManifest() tools.McpManifest { | ||
| return t.mcpManifest | ||
| } | ||
|
|
||
| func (t Tool) Authorized(verifiedAuthServices []string) bool { | ||
| return tools.IsAuthorized(t.AuthRequired, verifiedAuthServices) | ||
| } | ||
|
|
||
| func (t Tool) RequiresClientAuthorization(resourceMgr tools.SourceProvider) (bool, error) { | ||
| return false, nil | ||
| } | ||
|
|
||
| func (t Tool) ToConfig() tools.ToolConfig { | ||
| return t.Config | ||
| } | ||
|
|
||
| func (t Tool) GetAuthTokenHeaderName(resourceMgr tools.SourceProvider) (string, error) { | ||
| return "Authorization", nil | ||
| } | ||
|
|
||
| func (t Tool) GetParameters() parameters.Parameters { | ||
| return t.allParams | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.