-
Notifications
You must be signed in to change notification settings - Fork 6
Stream optional interfaces #11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Created simpleblob.NewReader and simpleblob.NewWriter functions, along with ReaderStorage and WriterStorage interface types. Those functions allow to provide an optimized way to write to and read from backend if those interfaces are satsfied. If an interface is not satisfied by backend, a fallback implementation is used, akin to `fs.GlobFS`.
Provide io.ReadCloser and io.WriteCloser to interact with files to way the `os` package does it. The `os` package has byte-slice based functions that rely on streams under the hood, we're doing the same here.
The Object returned by MinIO client satisfies io.ReadCloser, so this implementation is very straightforward. For writing, the MinIO client takes a reader and needs the full size of the stored object. Thus, we'd need to exhaust the writer into a buffer, take its size and provide it to the client, which is basically what the fallback implementation of NewWriter does.
Confront the results from simpleblob.NewReader and simpleblob.NewWriter with simpleblob.Interface.
Co-authored-by: wojas <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly nits and improvements, the logic LGTM, except for the fs writer.
Co-authored-by: wojas <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First pass, will do a more in depth one soon.
General name validation was dropped in favour of documenting safe names.
@@ -0,0 +1,90 @@ | |||
package fs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks solid, I guess you have looked at other similar solutions like this one:
https://github.com/google/renameio
just to see we are not missing any corner cases?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the recommendation. I gave it a look and while it looks like it can handle a broader set of situations (that we don't need here), I didn't see something that would benefit us here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nothing to add since last time. If @wojas comments are all closed it seems like it's time to finalize this.
Per #3, and inspired by
fs.FS
optional interfaces (e.g.fs.GlobFS
), this PR introduces two optional interfaces:WriterStorage
andReaderStorage
.Those interfaces are created using global functions:
In case
st
doesn't satisfyReaderStorage
orWriterStorage
, it will be wrapped in a struct, allowing abstraction but no performance gain.Corresponding tests have been implemented.
Closes #3