Skip to content

S3ArchivePlugin Cleanup and Error Handling Improvements #1166

@ata-nas

Description

@ata-nas

Story Form

As a Block Node Developer
I want to have a clean, well managed plugin with improved error handling capabilities
So that it is easily manageable and updatable if need be

Technical Notes

  • the S3ArchivePlugin is in need of some cleanup
  • the plugin will have to also check available multipart uploads at startup in order to ensure proper state of the archive:
    • query mulitipart uploads at start
      • if any:
        • close the multipart
        • check the filename for what blocks it contains
          • if we have those blocks, delete the file and send those blocks again
          • if we do not have all of the blocks, download that file to ensure we have all the blocks, then delete the partial file and recreate, continuing with later blocks to complete the file.
      • if not:
        • query the latest tar and determine the latest persisted
  • the plugin is in the role of the caller of the s3 client, we need to make sure that the plugin has as less as possible responsibilities in terms of making requests to the bucket
  • this issues is connected to S3Client Cleanup and Error Handling Improvements #1118 as changes in that issue will affect this one

Metadata

Metadata

Assignees

No one assigned

    Labels

    Block NodeIssues/PR related to the Block Node.S3 Cloud ArchiveIssues related to the S3 Cloud Archive functionality

    Type

    Projects

    Status

    🧊 Backlog

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions