Skip to content

DM: MariaDB 11.8 DPANIC due to GTID set update failure #12423

@dveeden

Description

@dveeden

What did you do?

Source: MariaDB 11.8.5

podman run --rm --name mariadb -e MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1 -p 3306:3306 mariadb:11.8 --log-bin=mariadb-log --binlog-format=ROW --binlog-legacy-event-pos

Source

source-id: "mariadb-01"
from:
  host: "127.0.0.1"
  port: 3306
  user: "root"

Task

name: mariadb-to-tidb
task-mode: all
mysql-instances:
  - source-id: "mariadb-01"
    block-allow-list: "test"
target-database:
  host: "127.0.0.1"
  port: 4000
  user: "root"
block-allow-list:
  test:
    do-dbs: ["*"]

Statements on source:

create schema test;
use test
create table t (id int primary key);
insert into t values (1);
alter table t add column c1 varchar(255);
update t set c1='test 123';

What did you expect to see?

Regular working replication

What did you see instead?

[2025/11/27 08:57:14.191 +01:00] [DPANIC] [binlog_locations.go:160] ["failed to update GTID set"] [GTID=0-1-5] [error="invalid GTID format, must UUID:interval[:interval]"] [errorVerbose="invalid GTID format, must UUID:interval[:interval]\ngithub.com/go-mysql-org/go-mysql/mysql.ParseUUIDSet\n\tgithub.com/go-mysql-org/[email protected]/mysql/mysql_gtid.go:214\ngithub.com/go-mysql-org/go-mysql/mysql.ParseMysqlGTIDSet\n\tgithub.com/go-mysql-org/[email protected]/mysql/mysql_gtid.go:411\ngithub.com/go-mysql-org/go-mysql/mysql.(*MysqlGTIDSet).Update\n\tgithub.com/go-mysql-org/[email protected]/mysql/mysql_gtid.go:473\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.(*locationRecorder).setCurEndGTID\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/binlog_locations.go:158\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.(*locationRecorder).update.func2\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/binlog_locations.go:228\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.(*locationRecorder).update\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/binlog_locations.go:180\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.locationStream.GetEvent\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/streamer_controller.go:123\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.(*StreamerController).getEvent\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/streamer_controller.go:533\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.(*StreamerController).GetEvent\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/streamer_controller.go:337\ngithub.com/pingcap/tiflow/dm/syncer.(*Syncer).Run\n\tgithub.com/pingcap/tiflow/dm/syncer/syncer.go:2153\ngithub.com/pingcap/tiflow/dm/syncer.(*Syncer).Process\n\tgithub.com/pingcap/tiflow/dm/syncer/syncer.go:757\nruntime.goexit\n\truntime/asm_amd64.s:1693"] [stack="github.com/pingcap/tiflow/dm/syncer/binlogstream.(*locationRecorder).setCurEndGTID\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/binlog_locations.go:160\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.(*locationRecorder).update.func2\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/binlog_locations.go:228\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.(*locationRecorder).update\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/binlog_locations.go:180\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.locationStream.GetEvent\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/streamer_controller.go:123\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.(*StreamerController).getEvent\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/streamer_controller.go:533\ngithub.com/pingcap/tiflow/dm/syncer/binlogstream.(*StreamerController).GetEvent\n\tgithub.com/pingcap/tiflow/dm/syncer/binlogstream/streamer_controller.go:337\ngithub.com/pingcap/tiflow/dm/syncer.(*Syncer).Run\n\tgithub.com/pingcap/tiflow/dm/syncer/syncer.go:2153\ngithub.com/pingcap/tiflow/dm/syncer.(*Syncer).Process\n\tgithub.com/pingcap/tiflow/dm/syncer/syncer.go:757"]
invalid GTID format, must UUID:interval[:interval]
github.com/go-mysql-org/go-mysql/mysql.ParseUUIDSet
	github.com/go-mysql-org/[email protected]/mysql/mysql_gtid.go:214
github.com/go-mysql-org/go-mysql/mysql.ParseMysqlGTIDSet
	github.com/go-mysql-org/[email protected]/mysql/mysql_gtid.go:411
github.com/go-mysql-org/go-mysql/mysql.(*MysqlGTIDSet).Update
	github.com/go-mysql-org/[email protected]/mysql/mysql_gtid.go:473
github.com/pingcap/tiflow/dm/syncer/binlogstream.(*locationRecorder).setCurEndGTID
	github.com/pingcap/tiflow/dm/syncer/binlogstream/binlog_locations.go:158
github.com/pingcap/tiflow/dm/syncer/binlogstream.(*locationRecorder).update.func2
	github.com/pingcap/tiflow/dm/syncer/binlogstream/binlog_locations.go:228
github.com/pingcap/tiflow/dm/syncer/binlogstream.(*locationRecorder).update
	github.com/pingcap/tiflow/dm/syncer/binlogstream/binlog_locations.go:180
github.com/pingcap/tiflow/dm/syncer/binlogstream.locationStream.GetEvent
	github.com/pingcap/tiflow/dm/syncer/binlogstream/streamer_controller.go:123
github.com/pingcap/tiflow/dm/syncer/binlogstream.(*StreamerController).getEvent
	github.com/pingcap/tiflow/dm/syncer/binlogstream/streamer_controller.go:533
github.com/pingcap/tiflow/dm/syncer/binlogstream.(*StreamerController).GetEvent
	github.com/pingcap/tiflow/dm/syncer/binlogstream/streamer_controller.go:337
github.com/pingcap/tiflow/dm/syncer.(*Syncer).Run
	github.com/pingcap/tiflow/dm/syncer/syncer.go:2153
github.com/pingcap/tiflow/dm/syncer.(*Syncer).Process
	github.com/pingcap/tiflow/dm/syncer/syncer.go:757
runtime.goexit
	runtime/asm_amd64.s:1693

Versions of the cluster

DM version (run dmctl -V or dm-worker -V or dm-master -V):

Release Version: v9.0.0-beta.2.pre-43-gd105a2b4b
Git Commit Hash: d105a2b4b780a1d4b6dd9bbf6f43bd7af081efdf
Git Branch: master
UTC Build Time: 2025-11-27 07:50:16
Go Version: go1.25.4 X:nodwarf5
Failpoint Build: false

Upstream MySQL/MariaDB server version:

11.8.5

Downstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

v8.5.3

How did you deploy DM: tiup or manually?

(leave TiUP or manually here)

Other interesting information (system version, hardware config, etc):

>
>

current status of DM cluster (execute query-status <task-name> in dmctl)

» query-status mariadb-to-tidb
{
    "result": true,
    "msg": "",
    "sources": [
        {
            "result": true,
            "msg": "",
            "sourceStatus": {
                "source": "mariadb-01",
                "worker": "127.0.0.1:8262",
                "result": null,
                "relayStatus": null
            },
            "subTaskStatus": [
                {
                    "name": "mariadb-to-tidb",
                    "stage": "Running",
                    "unit": "Sync",
                    "result": null,
                    "unresolvedDDLLockID": "",
                    "sync": {
                        "totalEvents": "8",
                        "totalTps": "0",
                        "recentTps": "0",
                        "masterBinlog": "(mariadb-log.000002, 610)",
                        "masterBinlogGtid": "0-1-2",
                        "syncerBinlog": "(mariadb-log.000002, 610)",
                        "syncerBinlogGtid": "",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                        ],
                        "synced": true,
                        "binlogType": "remote",
                        "secondsBehindMaster": "0",
                        "blockDDLOwner": "",
                        "conflictMsg": "",
                        "totalRows": "8",
                        "totalRps": "0",
                        "recentRps": "0",
                        "ioTotalBytes": "1375503",
                        "dumpIOTotalBytes": "2621"
                    },
                    "validation": null
                }
            ]
        }
    ]
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    MariaDBThis issue is found with MariaDB upstreamarea/dmIssues or PRs related to DM.severity/moderatetype/bugThe issue is confirmed as a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions