Open
Description
Description
按时间字段,将数据分散到不同的mysql实例,报错
tail -f dtle.log
2024-09-12T04:42:31.680Z [INFO] applier: doFullCopy: loop: rowCopyComplete: job=dtle-demo-src-dst2
2024-09-12T04:42:31.680Z [INFO] extractor: initBinlogReader: job=dtle-demo-src-dst2
2024-09-12T04:42:31.680Z [INFO] reader: Connecting binlog streamer: job=dtle-demo-src-dst2 file=bin.000004 pos=3297 gtid=4d7bd416-6f5a-11ef-9c56-0242ac120004:1-22
2024-09-12T04:42:31.684Z [INFO] extractor: monitor and update job status: job=dtle-demo-src-dst2
2024-09-12T04:42:31.684Z [INFO] extractor: Beginning streaming: job=dtle-demo-src-dst2
2024-09-12T04:42:31.684Z [INFO] reader: Rotate to next binlog: job=dtle-demo-src-dst2 name=bin.000004
2024-09-12T04:42:31.685Z [ERROR] extractor: onError: job=dtle-demo-src-dst2 err="StreamEvents encountered unexpected error: 'where' predicate does not eval to bool. demo.demo_tbl create_time>='2023-01-01'" hasShutdown=false
2024-09-12T04:42:31.685Z [INFO] extractor: extractor shutdown: job=dtle-demo-src-dst2
2024-09-12T04:42:31.687Z [INFO] DestroyTask: id=4ea0319f-4101-d066-d05a-9bcafe20f734/src/9979b45c
2024-09-12T04:42:31.785Z [INFO] extractor: Shutting down: job=dtle-demo-src-dst2
Steps to reproduce the issue
- 创建表结构
mysql> show create table demo_tbl \G
*************************** 1. row ***************************
Table: demo_tbl
Create Table: CREATE TABLE `demo_tbl` (
`id` int(11) NOT NULL,
`name` varchar(12) DEFAULT NULL,
`create_time` datetime DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
1 row in set (0.00 sec)
- 源表数据
mysql> select * from demo_tbl;
+----+------+---------------------+
| id | name | create_time |
+----+------+---------------------+
| 1 | a | 2023-01-02 14:00:00 |
| 2 | b | 2023-01-02 14:01:00 |
| 3 | c | 2023-01-02 14:02:00 |
| 4 | d | 2024-01-02 14:02:00 |
| 5 | e | 2024-01-03 14:02:00 |
| 6 | e | 2024-06-03 14:02:00 |
| 7 | e | 2024-06-03 14:02:00 |
| 8 | e | 2024-06-03 14:02:00 |
| 9 | e | 2022-06-03 14:02:00 |
| 10 | e | 2022-06-03 14:02:00 |
| 11 | e | 2024-06-03 14:02:00 |
+----+------+---------------------+
11 rows in set (0.00 sec)
- dtle任务文件
cat src_dst1.json
{
"Job": {
"ID": "dtle-demo-src-dst1",
"Datacenters": ["dc1"],
"TaskGroups": [{
"Name": "src",
"Tasks": [{
"Name": "src",
"Driver": "dtle",
"Config": {
"Gtid": "",
"ReplicateDoDb": [{
"TableSchema": "demo",
"Tables": [{
"TableName": "demo_tbl",
"Where":"create_time<'2023-01-01'"
}]
}],
"SrcConnectionConfig": {
"Host": "mysql-src",
"Port": 3306,
"User": "root",
"Password": "pass"
},
"DestConnectionConfig": {
"Host": "mysql-dst1",
"Port": 3306,
"User": "root",
"Password": "pass"
}
}
}]
}, {
"Name": "dest",
"Tasks": [{
"Name": "dest",
"Driver": "dtle",
"Config": {
"DestType": "mysql"
}
}]
}]
}
}
cat src_dst2.json
{
"Job": {
"ID": "dtle-demo-src-dst2",
"Datacenters": ["dc1"],
"TaskGroups": [{
"Name": "src",
"Tasks": [{
"Name": "src",
"Driver": "dtle",
"Config": {
"Gtid": "",
"ReplicateDoDb": [{
"TableSchema": "demo",
"Tables": [{
"TableName": "demo_tbl",
"Where":"create_time>='2023-01-01'"
}]
}],
"SrcConnectionConfig": {
"Host": "mysql-src",
"Port": 3306,
"User": "root",
"Password": "pass"
},
"DestConnectionConfig": {
"Host": "mysql-dst2",
"Port": 3306,
"User": "root",
"Password": "pass"
}
}
}]
}, {
"Name": "dest",
"Tasks": [{
"Name": "dest",
"Driver": "dtle",
"Config": {
"DestType": "mysql"
}
}]
}]
}
}
Describe the results you received
tail -f dtle.log
2024-09-12T04:42:31.680Z [INFO] applier: doFullCopy: loop: rowCopyComplete: job=dtle-demo-src-dst2
2024-09-12T04:42:31.680Z [INFO] extractor: initBinlogReader: job=dtle-demo-src-dst2
2024-09-12T04:42:31.680Z [INFO] reader: Connecting binlog streamer: job=dtle-demo-src-dst2 file=bin.000004 pos=3297 gtid=4d7bd416-6f5a-11ef-9c56-0242ac120004:1-22
2024-09-12T04:42:31.684Z [INFO] extractor: monitor and update job status: job=dtle-demo-src-dst2
2024-09-12T04:42:31.684Z [INFO] extractor: Beginning streaming: job=dtle-demo-src-dst2
2024-09-12T04:42:31.684Z [INFO] reader: Rotate to next binlog: job=dtle-demo-src-dst2 name=bin.000004
2024-09-12T04:42:31.685Z [ERROR] extractor: onError: job=dtle-demo-src-dst2 err="StreamEvents encountered unexpected error: 'where' predicate does not eval to bool. demo.demo_tbl create_time>='2023-01-01'" hasShutdown=false
2024-09-12T04:42:31.685Z [INFO] extractor: extractor shutdown: job=dtle-demo-src-dst2
2024-09-12T04:42:31.687Z [INFO] DestroyTask: id=4ea0319f-4101-d066-d05a-9bcafe20f734/src/9979b45c
2024-09-12T04:42:31.785Z [INFO] extractor: Shutting down: job=dtle-demo-src-dst2
Describe the results you expected
Output of ./dtle version
:**
dtle 4.23.04.3
(paste your output here)
Additional information
(e.g. issue happens only occasionally)
Additional details (log, config, job config etc):
Metadata
Metadata
Assignees
Labels
No labels