Skip to content

Bug Report: VReplication workflows can fail on statement events on unrelated tables #18715

@mattlord

Description

@mattlord

Overview of the Issue

DDL, user/account management statements (CREATE/ALTER/DROP user etc), and trigger statements are always replicated as statement based binary log events regardless of the binlog-format setting. MySQL binary log events for these statements like ALTER USER or CREATE TRIGGER have the same binary log event metadata as ALTER TABLE — meaning that they are all effectively treated as DDL in the binary logs.

By default, DDL in VReplication is IGNOREd. This is controlled by the on-ddl flag: https://vitess.io/docs/reference/vreplication/vreplication/#handle-ddl

It is not uncommon, however, to use the EXEC option so that you can allow for schema changes to tables as they are being migrated (which can take days or even weeks). When using this option on external or unmanaged MySQL instances (migrations into Vitess) and executing any of these noted statements for which the Vitess parser has no support such as CREATE TRIGGER or ALTER USER on the source side external mysqld instance (or directly against a mysqld instance within a normal Vitess cluster), your VReplication workflow from that external mysqld can fail when those statements are streamed if the tables involved in the trigger or the user being altered do not exist on the target.

The cause of that is here:

  • case sqlparser.StmtDDL:
    if mustSendDDL(q, vs.cp.DBName(), vs.filter, vs.vse.env.Environment().Parser()) {
    vevents = append(vevents, &binlogdatapb.VEvent{
    Type: binlogdatapb.VEventType_GTID,
    Gtid: replication.EncodePosition(vs.pos),
    }, &binlogdatapb.VEvent{
  • func mustSendDDL(query mysql.Query, dbname string, filter *binlogdatapb.Filter, parser *sqlparser.Parser) bool {
    if query.Database != "" && query.Database != dbname {
    return false
    }
    ast, err := parser.Parse(query.SQL)
    // If there was a parsing error, we send it through. Hopefully,
    // recipient can handle it.
    if err != nil {
    return true
    }

We return true there as the parser.Parse() call returns an error as the Vitess parser does not support e.g. ALTER USER or CREATE TRIGGER statements and it returns an error. This then results in these statements being sent in the VReplication stream to the target (the vplayer component running on the target primary tablet) where they later fail.

Reproduction Steps

git checkout main && make build

cd examples/local

alias vtctldclient='command vtctldclient --server=localhost:15999'

./101_initial_cluster.sh && ./201_customer_tablets.sh

vtctldclient MoveTables --workflow commerce2customer --target-keyspace customer create --source-keyspace commerce --tables "customer,corder" --on-ddl EXEC

sleep 5

commerce_primary_uid=$(vtctldclient GetTablets --keyspace commerce --tablet-type primary --shard "0" | awk '{print $1}' | cut -d- -f2 | bc)

command mysql -u root --socket "${VTDATAROOT}/vt_0000000${commerce_primary_uid}/mysql.sock" vt_commerce -e  'CREATE TABLE account (acct_num INT, amount DECIMAL(10,2)); CREATE TRIGGER ins_sum BEFORE INSERT ON account FOR EACH ROW SET @sum = @sum + NEW.amount;'

sleep 5

vtctldclient --compact movetables --target-keyspace customer --workflow commerce2customer show --include-logs=false | grep message

The end result being:

              "message": "error applying event: You do not have the SUPER privilege and binary logging is enabled (you *might* want to use the less safe log_bin_trust_function_creators variable) (errno 1419) (sqlstate HY000) during query: CREATE DEFINER=`root`@`localhost` TRIGGER ins_sum BEFORE INSERT ON account FOR EACH ROW SET @sum = @sum + NEW.amount",

Binary Version

vtgate version Version: 23.0.0-SNAPSHOT (Git revision f4658bd2044eeca02e8371ddae6c36070d53308a branch 'main') built on Fri Oct  3 16:56:45 UTC 2025 by [email protected] using go1.25.1 darwin/arm64

Operating System and Environment details

N/A

Log Fragments

Metadata

Metadata

Assignees

Type

Projects

Status

Prioritized

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions