Data Transfer Fails Between Vaults with Different Chunk Sizes

[S4tvara]

Describe the bug
If two vaults have different chunk sizes, data transfer fails.

To Reproduce
Steps to reproduce the behavior:

1. Create two vaults with no encryption but different chunk sizes.
2. Attempt to sync data between the vaults.
3. Observe that chunks fail to transfer.

Expected behavior
The vault should:

• Retrieve all data from the source vault.
• Reassemble it into a complete file.
• Re-chunk the file according to the target vault’s chunk size.
• Successfully store the chunks.

Screenshots

Additional context
This issue occurs because the transfer process does not handle differing chunk sizes. Implementing a reassemble-and-rechunk step will ensure reliable data transfer between vaults with different chunk configurations.

[Udayan853]

I tried reproducing this issue but got slightly different behavior.
The chunks are being transferred successfully, and I’m also able to retrieve the content from both vaults — even when their respective chunk_size values don’t match.

Steps I followed:

• Created two unencrypted vaults: dune1 (chunk_size: 2MB) and dune2 (chunk_size: 4MB)
• Added a sample.pdf (~5.8MB) to each, one at a time and called sietch sync for both cases
• Verified that the chunks transferred and retrieval via get worked in both cases

When both vaults share the same filepath, the chunks are transferred but become orphaned; still, retrieval remains successful.

Could it be that this issue is caused by something other than differing chunk_sizes?

[S4tvara]

@Udayan853 ,
That's interesting lemme try it again. Btw can you check the chunk size on dune2

[Udayan853]

[S4tvara]

@Udayan853,
Surprisingly, I was able to replicate it again xD.

The configurations for my vaults were:

• vault1 → No encryption, chunk size: 120MB
• vault2 → No encryption, chunk size: 100MB

The file I used was around 4gb.
Try transferring chunks from a bigger chunk size to smaller.

[Udayan853]

This is odd, now when I try to transfer the files, the chunks get orphaned, and the file is not transferred
Its the same configuration as above, I deleted dune1 and dune2 and reinitialized them

[Udayan853]

On making dune1 listen for connections

[S4tvara]

@Udayan853,
weird lemme deep dive into this problem.
can you share the vault configs here?

[Udayan853]

[S4tvara]

This is odd, now when I try to transfer the files, the chunks get orphaned, and the file is not transferred Its the same configuration as above, I deleted dune1 and dune2 and reinitialized them

@Udayan853 , can you try this again but after exchanging keys using discover

[Udayan853]

Same result