The current implementation conflates the idea of a local content store on disk with the configuration for which content stores are used, and when. This has some undesireable implications for performance (you can't stream an object from a backend, for example, it has to be fetched and stored locally first), as well as complicating the code significantly.
Instead, the frontend service should just be an interface to a configuration of content stores. These would have three types: read stores (for fetching an object), synchronous write stores (an object being stored must be successfully stored in each of these before the frontend operation completes), and asynchronous write stores (an object being stored will be stored in each of these, but the operation will be scheduled asynchronously and the frontend operation will complete without waiting on this).
This would also pave the way for having multiple service endpoints which expose different content store configurations.
The current implementation conflates the idea of a local content store on disk with the configuration for which content stores are used, and when. This has some undesireable implications for performance (you can't stream an object from a backend, for example, it has to be fetched and stored locally first), as well as complicating the code significantly.
Instead, the frontend service should just be an interface to a configuration of content stores. These would have three types: read stores (for fetching an object), synchronous write stores (an object being stored must be successfully stored in each of these before the frontend operation completes), and asynchronous write stores (an object being stored will be stored in each of these, but the operation will be scheduled asynchronously and the frontend operation will complete without waiting on this).
This would also pave the way for having multiple service endpoints which expose different content store configurations.