We've got a lot of code everywhere talking to S3 directly. It's pretty error-prone to remember the correct incantations for object key formatting, checksum calculation, etc.
I've been thinking for a while that we want some sort of "storage" abstraction that provides a thin layer over raw S3 requests with structured parameters so it's impossible to forget to add a checksum or mess up the key construction.
Some open questions:
- Should bake the S3 bucket into the service instance? Doing so saves us an extra field to carry around, but means that different repositories and packages will no longer be able to be saved in different buckets (this also requires a schema migration). We don't have anyone using this functionality today I think, but we also have a user who wants to separate their APT and YUM buckets.
- Should this be a storage abstraction, or just an APT repository abstraction? After all, YUM and the like have different repository metadata. Does this mean we need multiple instances created at startup?
- How should we handle parameters like S3 prefixes? Should they be raw strings, newtyped strings, or private fields pulled from domain model objects that act as capability tokens that provide proof that they were pulled from the database?
We've got a lot of code everywhere talking to S3 directly. It's pretty error-prone to remember the correct incantations for object key formatting, checksum calculation, etc.
I've been thinking for a while that we want some sort of "storage" abstraction that provides a thin layer over raw S3 requests with structured parameters so it's impossible to forget to add a checksum or mess up the key construction.
Some open questions: