S3 BACKUP DATABASE Target

[gramian]

Dear All,

it would be very "cloudy" to have S3 object storage as target for a database backup. Particularly, for the SQL BACKUP DATABASE command. For example, this could look like:

BACKUP DATABASE s3://my-storage.com/my-bucket

The biggest advantage is that the ArcadeDB host system does not need to mount a network drive or share to allow external backups.

The issue is to provide credentials (access key and secret key) safely: the keys could be stored in a database setting, but then it would be readable by other users. Alternatively, the keys could be passed as options to the command, but this would mean if a backup command is issued from a script that it contains secrets. However, if the locations of the secrets would be passed in the command then this would be somewhat safer, so:

BACKUP DATABASE s3://my-storage.com/my-bucket WITH access=/run/secrets/bucket-access secret=/run/secrets/bucket-secret

and at the same time tie in well with secret passing of container orchestrators.

Feel free to add to this idea.

PS: This idea was discussed some time ago already on Discord.

[gramian]

Another approach would be to expand the server.backupDirectory setting such that the server/bucket can be passed and new settings are added for credentials, for example:

-Darcadedb.server.backupDirectory=s3:my-storage.com/my-bucket \
-Darcadedb.server.backupAccessPath=/run/secrets/access-key \
-Darcadedb.server.backupSecretPath=/run/secrets/secret-key

This would be less invasive then the solution above and additionally a backup location could be set via environment variable.

So probably this is a better way as the SQL grammar would not need changes.

[noveroa]

this is exaclty what we were looking into for azure ... automated backups (ie daily, monthly) would also be a great feature - (aws databases does have that option . eh?)

is this still an active topic or pursuit by the arcade db team?