[Feature] Implement AWS S3 Support for BanyanDB's Remote Filesystem Interface

[hanahmily]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

To enhance the backup capabilities of BanyanDB, support for AWS S3 as a remote filesystem must be implemented. This will allow users to sync data directly to S3, leveraging its scalability and reliability. The implementation should include:

• S3 Client Integration: Use the AWS SDK for Go to interact with S3.
• Filesystem Operations: Implement methods for listing, reading, writing, and deleting files in S3.
• Configuration: Add configuration options for S3 bucket name, region, and credentials.
• Error Handling: Ensure robust error handling for network issues and S3-specific errors.
• Testing: Provide unit and integration tests to validate the S3 integration.

This feature will empower users to securely and efficiently back up their data to S3, aligning with modern cloud-native storage practices.

Use case

No response

Related issues

No response

Are you willing to submit a pull request to implement this on your own?

• Yes I am willing to submit a pull request on my own!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Tsukilc]

Hi, I have experience using the AWS SDK for file upload and read operations. I'm interested in this issue and would love to work on it. Could you assign it to me?

[wu-sheng]

Hi, I have experience using the AWS SDK for file upload and read operations. I'm interested in this issue and would love to work on it. Could you assign it to me?

Please be familiar with BanyanDB as well. Ww are not doing this, you could try poc locally, and provide some draft.

[Tsukilc]

Hello, I have completed a basic version of s3 and verified it locally.
apache/skywalking-banyandb#622

Could you please help me to see if there are any errors in the direction, so that I can further improve it? If not, I will write a detailed proposal and improve it!
Thank you so much for your time and assistance!

Hi, I have experience using the AWS SDK for file upload and read operations. I'm interested in this issue and would love to work on it. Could you assign it to me?

Please be familiar with BanyanDB as well. Ww are not doing this, you could try poc locally, and provide some draft.