Move (de)compression to Web Workers #72
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
To be simplified and replaced with renderer-side web workers.
Remove a temporary console.log statement as well as now-unused msgpack module.
Does not fix the savegame list bugs. Note that savegame index serialization happens a few times for some reason, this is out of scope for this commit though.
Implement DefaultCompression class along with a generic interface to facilitate easy to use compression in a background thread, and make use of this class in Storage implementation by default.
While the browser engine can clean up workers automatically, it's a good idea to terminate them as soon as we don't need those anymore. It's generally suggested to reuse a single worker for repetitive operations, but implementing such a system is much harder. In addition, spawning a new worker for each operation allows (de)compressing multiple blobs in parallel :)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Alternative to #71 - does almost the same in a slightly different way. This PR once again adds a separate class for built-in compression implementation (gzipped MessagePack) and makes use of it in the Storage implementation. Unlike upstream, this compression is applied on a lower level, so all files are compressed instead of just RWProxy and other explicit usages. There needs to be a way to bypass this, and that's exactly why the Storage constructor optionally accepts a different storage interface. This may be useful for mods once they get own data directories.
Unlike #71 this implementation creates a new worker for each operation, which may help with intensive compression operations; workers are terminated after use. Error handling for workers may be simplified once
Error.isErrorbecomes available in Electron.