feat(services/swift): add SLO multipart upload for large objects

[benroeder]

Summary

• Fixes feat: Swift service should support multipart upload for large objects #7211
• Replace oio::OneShotWriter with oio::MultipartWriter backed by Swift's Static Large Object (SLO) API
• initiate_part() generates a local UUID (no server call needed, unlike S3)
• write_part() uploads segments to .segments/{path}/{upload_id}/{part_number:08}
• complete_part() PUTs a JSON manifest to {path}?multipart-manifest=put
• abort_part() lists and deletes orphaned segments
• Small writes still go through write_once() as a single PUT — oio::MultipartWriter handles the decision automatically
• Supports concurrent part uploads via args.concurrent()
• Adds uuid dependency for upload ID generation

Reference: https://docs.openstack.org/swift/latest/overview_large_objects.html

Test plan

• test_writer_write, test_writer_write_with_overwrite, test_writer_write_with_concurrent, test_writer_sink, test_writer_abort and other writer tests now execute
• All 93 behavior tests pass against both local SAIO and a real Swift cluster

[dosubot]

Related Documentation

Checked 0 published document(s) in 1 knowledge base(s). No updates required.

^{How did I do? Any feedback?}  

[Xuanwo]

Thank you for this great work!

[Xuanwo]

Is the path defined by Swift, or did we choose it ourselves? Will this path be listed out by users?

[benroeder]

The .segments/ prefix is a convention, not defined by Swift — SLO just needs segments to be reachable objects anywhere in the container. This convention is widely used by Swift clients (e.g. python-swiftclient uses the same pattern).

Segments won't appear in OpenDAL listings because they're outside the user's root prefix — swift_list uses build_abs_path(&self.root, path) as the prefix filter, and .segments/ sits at the container root. They would be visible if someone lists the container directly via the Swift API, but that's the same behavior as python-swiftclient.

[Xuanwo]

Got it, thanks for the explanation. I would greatly appreciate it if you could add those explanations directly in the code as comments.

[Xuanwo]

I'm guessing we need to carry the user metadata here?

[benroeder]

Good catch — the manifest PUT should carry user metadata and content headers (Content-Type, Content-Disposition, etc.) from OpWrite. Currently only segments get the raw bytes and the manifest gets none of the user's metadata. I'll fix this.

[Xuanwo]

Maybe we can add size: Option<usize> in oio::MultipartPart so that we can remove this lock.

[benroeder]

Agree, that's cleaner. I'll add size: Option<u64> to oio::MultipartPart and remove the HashMap<usize, u64> + Mutex from the writer.

[benroeder]

Done — added size: Option<u64> to oio::MultipartPart and removed the HashMap<usize, u64> + Mutex from the Swift writer.

Swift's SLO manifest format requires size_bytes for each segment — Swift validates that each segment's actual size matches the manifest. S3/GCS don't need size in their complete calls (only part number and etag), which is why MultipartPart didn't have it before.

Since this is a core struct, this required updating all 11 existing MultipartPart constructors across other services (S3, GCS, COS, OSS, OBS, B2, Upyun, Vercel Blob, object_store integration, and the test mock) to include size: None. Let me know if you'd prefer this split into a separate PR.

Also addressed your other feedback:

• Manifest PUT now forwards user metadata (X-Object-Meta-* headers) from OpWrite
• Rebased onto current main (includes the 4 merged Swift PRs)

[Xuanwo]

Thank you, really nice!