Splitter

Splitter

Available as of Backup 3.0.20

Backup::Model.new(:my_backup, 'Description for my_backup') do
  ##
  # Split [Splitter]
  #
  # Split the backup file in to chunks of 250 megabytes
  # if the backup file size exceeds 250 megabytes
  #
  split_into_chunks_of 250

  # etc...
end

By default, all generated models will be setup to utilize the Splitter, using a 250MB chunk size.

This can be disabled using the --no-splitter option with the Generator:

$ backup generate:model --trigger my_backup --no-splitter (etc...)

What does it do?

It removes the file size constraints of remote storage locations such as Amazon S3, Rackspace Cloud Files, Ninefold and Dropbox. For example, you cannot store files larger than 5GB in size on Amazon S3. You can now configure backup with split_into_chunks_of(5000) to tell Backup to split the entire backup file in to chunks of 5000 megabytes (5gb) before storing it to Amazon S3. So assuming your backup is 15GB in size, it will split the backup file in 3 chunks and upload them separately.

Enabling backup splitting

Every Backup model now consists of a new option: split_into_chunks_of(int) in megabytes. For example, say you wish to split any backups that exceed 500 megabytes in size.

Backup::Model.new(:my_backup, "My chunked backup in chunks of 500mb") do
  split_into_chunks_of(500)

  # ... the rest of the usual configuration: storages, syncers, encryptors, compressors, notifiers, etc.
end

Simply specify the split_into_chunks_of(int) in your Backup::Model and backup will ensure all files exceeding 500mb will be split to ensure no files larger than 500mb will be uploaded to the storage location(s).

Assembling a backup that has been split in to multiple chunks

Just download the chunks you would like to assemble to your local or remote machine and perform the following cat command.

cat my-backup.tar.gz.enc-aa my-backup.tar.gz.enc-ab my-backup.tar.gz.enc-ac > my-backup.tar.gz.enc

The end result will be a fully assembled my-backup.tar.gz.enc.

A shorthand way of assembling a backup that has been split in to multiple chunks can be done like so:

cat my-backup.tar.gz.enc-a[a-c] > my-backup-tar.gz.enc