The opendal storage implementation, when chunking (multipart) is enabled requires gathering all data in RAM before upload. There is no stream there.
The RAM usage is hence inflated. The main effect was curbed due to #6492 but we would benefit from reducing the part size a notch. The trade off is PUT requests costs.
Bringing it down from 5GB to 1GB for instance would really help without hurting user's wallets to much.
The opendal storage implementation, when chunking (multipart) is enabled requires gathering all data in RAM before upload. There is no stream there.
The RAM usage is hence inflated. The main effect was curbed due to #6492 but we would benefit from reducing the part size a notch. The trade off is PUT requests costs.
Bringing it down from 5GB to 1GB for instance would really help without hurting user's wallets to much.