Open
Description
Overview of the Issue
Traffic mirroring is currently broken because routing rules are not being taken into account.
Thanks to @joengmin-otterai for reporting the issue in Slack: https://vitess.slack.com/archives/C0PQY0PTK/p1740702333779269
Reproduction Steps
Run the first two steps from the local example:
$ ./101_initial_cluster.sh
$ ./201_customer_tablets.sh
Run MoveTables
:
$ vtctldclient MoveTables --target-keyspace customer --workflow commerce2customer create --source-keyspace commerce --tables 'corder'
MirrorTraffic
:
vtctldclient MoveTables --target-keyspace customer --workflow commerce2customer MirrorTraffic --percent 99.0
Run queries:
use commerce; select * from corder;
Inspect QPS Queries
field of /debug/vars
of the primary tablet in the customer
keyspace. They should increment with each select * from corder
, but do not.
The problem can be see with vexplain plan
:
mysql> vexplain plan select * from commerce.corder\G
*************************** 1. row ***************************
JSON: {
"OperatorType": "Mirror",
"Variant": "PercentBased",
"Percent": 100,
"Inputs": [
{
"OperatorType": "Route",
"Variant": "Unsharded",
"Keyspace": {
"Name": "commerce",
"Sharded": false
},
"FieldQuery": "select order_id, customer_id, sku, price from corder where 1 != 1",
"Query": "select order_id, customer_id, sku, price from corder",
"Table": "corder"
},
{
"OperatorType": "Route",
"Variant": "Unsharded",
"Keyspace": {
"Name": "commerce",
"Sharded": false
},
"FieldQuery": "select order_id, customer_id, sku, price from corder where 1 != 1",
"Query": "select order_id, customer_id, sku, price from corder",
"Table": "corder"
}
]
}
1 row in set (0.02 sec)
mysql>
The query should be mirrored to the customer
keyspace, but it is being mirrored to the commerce
keyspace. In other words, the same query is being sent twice to the same keyspace.
Binary Version
❯ ./bin/vttablet --version
vttablet version Version: 22.0.0-SNAPSHOT (Git revision 96f45350bba6c71c24938d3cbfe3e702a15ff850 branch 'main') built on Tue Mar 11 12:15:04 EDT 2025 by [email protected] using go1.24.0 darwin/arm64
Operating System and Environment details
macOS Sequoia 15.3.1
❯ uname -sr
Darwin 24.3.0
❯ uname -m
arm64