CloudFlare Node Backup uploads fail #87
Description
Hi, thanks for publishing this project. It's been super helpful to learn from to deploy our own chain.
The node backups role does not work on Cloudflare r2. It fails at the upload of files.txt. Oddly, it had worked to synchronize the db folder fine. Researching this, it seemed to be a common s3 compatibility issue.
<6>INFO : full/000930.sst: Copied (new)
<6>INFO :
Transferred: 357.202 MiB / 357.202 MiB, 100%, 69.344 MiB/s, ETA 0s
Transferred: 123 / 123, 100%
Elapsed time: 6.0s
+ sort
+ rclone lsf -R --fast-list --files-only R2backups:argon-testnet-backup/argon-testnet/20241209-192549
+ set +x
+ rclone copyto -v /tmp/remote-files-argon-testnet-20241209-192549 R2backups:argon-testnet-backup/argon-testnet/20241209-192549/files.txt
<3>ERROR : remote-files-argon-testnet-20241209-192549: Failed to copy: AccessDenied: Access Denied
status code: 403, request id: , host id:
<3>ERROR : Attempt 1/3 failed with 1 errors and: AccessDenied: Access Denied
status code: 403, request id: , host id:
<3>ERROR : remote-files-argon-testnet-20241209-192549: Failed to copy: AccessDenied: Access Denied
status code: 403, request id: , host id:
<3>ERROR : Attempt 2/3 failed with 1 errors and: AccessDenied: Access Denied
status code: 403, request id: , host id:
<3>ERROR : remote-files-argon-testnet-20241209-192549: Failed to copy: AccessDenied: Access Denied
status code: 403, request id: , host id:
<3>ERROR : Attempt 3/3 failed with 1 errors and: AccessDenied: Access Denied
status code: 403, request id: , host id:
<6>INFO :
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.5s
Failed to copyto: AccessDenied: Access Denied
status code: 403, request id: , host id:
It fails if you upload directly as well rclone copyto -v /tmp/remote-files-argon-testnet-20241209-192549 R2backups:argon-testnet-backup/argon-testnet/20241209-192549/files.txt
Single file uploads will apparently only work if you add --s3-no-check-bucket to the upload (this is using rclone 1.63.1).
As an aside, the rclone specific features seem quite minimal. Why did you choose to configure this role to have specific configurations for Cloudflare but not other s3 providers? It seems like it would only be a matter of configuring rclone with the right provider, which could easily be left to the ansible user.