-
Notifications
You must be signed in to change notification settings - Fork 237
Description
Describe the bug
When I deploy a replication cluster for Redis 7.4.5, and configured aof continuous backup. I found errors and the aof backup file was not uploaded correctly.
To Reproduce
Steps to reproduce the behavior:
After deploy kubeblocks v1.0.1 and redis-addon 1.0.1.
- create a redis replication cluster with version 7.4.5:
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
name: redis-7
namespace: test-redis
spec:
terminationPolicy: Delete
clusterDef: redis
topology: replication
services:
- componentSelector: redis
name: redis-7-internet
serviceName: redis-7-internet
roleSelector: primary
spec:
ipFamilyPolicy: PreferDualStack
ports:
- name: redis
port: 6379
protocol: TCP
targetPort: redis
type: LoadBalancer
componentSpecs:
- name: redis
env:
- name: REDIS_DEFAULT_PASSWORD
value: ""
serviceVersion: "7.4.5"
disableExporter: false
replicas: 2
#services:
#- name: redis-advertised
#serviceType: LoadBalancer
#podService: true
resources:
limits:
cpu: '0.5'
memory: 1Gi
requests:
cpu: '0.5'
memory: 1Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 2Gi
- name: redis-sentinel
serviceVersion: "7.4.5"
disableExporter: false
replicas: 3
resources:
limits:
cpu: '0.5'
memory: 0.5Gi
requests:
cpu: '0.5'
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 2Gi
- Then configure BackupRepo and BackupSchedule to start aof incremental backup.
$ ku get backup -n test-redis
NAME POLICY METHOD REPO STATUS TOTAL-SIZE DURATION DELETION-POLICY CREATION-TIME COMPLETION-TIME EXPIRATION-TIME
ff67ea12-redis-7-redis-backup-aof redis-7-redis-backup-policy aof s3-compatible-repo Running 1160 Delete 2025-10-11T11:10:04Z
- Observe that the statefulset for aof has been successfully created and running.
$ kubectl get po -n test-redis
NAME READY STATUS RESTARTS AGE
ff67ea12-redis-7-redis-backup-aof-0 2/2 Running 0 74m
- But the log of the job pod shows some errors.
$ kubectl
environment: line 201: [: -gt: unary operator expected
environment: line 201: [: -gt: unary operator expected
environment: line 201: [: -gt: unary operator expected
environment: line 201: [: -gt: unary operator expected
environment: line 201: [: -gt: unary operator expected
environment: line 201: [: -gt: unary operator expected
environment: line 201: [: -gt: unary operator expected
2025-10-11 12:10:03 INFO: Update file: /data/appendonlydir/appendonly.aof.2.incr.aof
2025-10-11 12:10:04 INFO: save_backup_status: empty start_time from backup repo, use current time
environment: line 201: [: -gt: unary operator expected
environment: line 201: [: -gt: unary operator expected
environment: line 201: [: -gt: unary operator expected
environment: line 201: [: -gt: unary operator expected
environment: line 201: [: -gt: unary operator expected
And the "start" and "end" field for the Backup
are always the same.
$ kubectl get backup -n test-redis ff67ea12-redis-7-redis-backup-aof -oyaml | yq '.status.timeRange'
{
"end": "2025-10-11T12:10:04Z",
"start": "2025-10-11T12:10:04Z"
}
$ ku get backup -n test-redis ff67ea12-redis-7-redis-backup-aof -oyaml | yq '.status.timeRange'
{
"end": "2025-10-11T12:10:37Z",
"start": "2025-10-11T12:10:37Z"
}
$ ku get backup -n test-redis ff67ea12-redis-7-redis-backup-aof -oyaml | yq '.status.timeRange'
{
"end": "2025-10-11T12:10:45Z",
"start": "2025-10-11T12:10:45Z"
}
Check the S3 bucket, the aof file was not uploaded as expected.
Expected behavior
The "start" and "end" field of Backup
should not always be the same, and aof files should be uploaded correctly.
Screenshots
Desktop (please complete the following information):
- Version: Kubeblocks v1.0.1,Redis Addon v1.0.1
Additional context
I tried to debug the script pitr-backup.sh
and found that the redis tools version is 7.2.10,which cannot match the Redis version 7.4.5. This makes the get_base_file_ctime()
method unable to get the correct result, which leads to a "-gt: unary operator expected" problem. The if conditional statement was not executed and the aof file is not uploaded.
if [ $(get_base_file_ctime "$base_file") -gt ${global_aof_last_modify_time} ]; then
datasafed push "${base_file}" "${backup_files_prefix}.dir/${AOF_DIR}/$(basename "${base_file}")"
datasafed push "${backup_manifest}" "${backup_files_prefix}.dir/${backup_manifest}"
datasafed push "${DATA_DIR}/users.acl" "${backup_files_prefix}.dir/users.acl"
DP_log "Upload file: $base_file $backup_manifest users.acl"
fi
Here is the redis-check-rdb
tool error log:
root@redis-backup-aof-test-0:/# redis-check-rdb /data/appendonlydir/appendonly.aof.2.base.rdb
[offset 0] Checking RDB file /data/appendonlydir/appendonly.aof.2.base.rdb
--- RDB ERROR DETECTED ---
[offset 9] Can't handle RDB format version 12
[additional info] While doing: start
[additional info] Reading type 0 (string)
[info] 0 keys read
[info] 0 expires
[info] 0 already expired
I have checked the redis addon v1.0.2. and I think this problem has not been resolved in redis addon v1.0.2.