Skip to content

Clone (not copy) data chunks when extracting from uncompressed chunks #299

@HaleTom

Description

@HaleTom

I made my chunk store uncompressed via:

  "store-options": {
    "/btrbk/ssh-in/desync-store.home-var/": {
      "uncompressed": true
    }

My hope was to save disk space (the btrfs volume itself uses compression): that when extracting, data, the extracted file would be cloned from BOTH any uncompressed chunks in the store AND other seeds, taking up a trivially small amount of space.

However, this indicates that only seed file data is cloned, uncompressed chunks are ignored:
https://github.com/copilot/share/0073513a-01a4-8002-8813-260884952024

There may be file system fragmentation implications in a store with very small chunks -- so cloning chunks would best determined by the user.

Would you consider an option --clone-chunks for cloning any uncompressed chunks?

The the user could decide that the average chunk size is sufficiently large to clone the underlying file system extents instead of copying the data in them. This would be especially useful for chunks in the megabyte size range.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions