-
-
Notifications
You must be signed in to change notification settings - Fork 424
re: New higher-performance rnote file format, and atomic file saving #1578
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…ation, lots left to do
…mplement cli and legacy loading, as well as clean things up
…ole (not finished though)
…nor changes to `set_compression`, added documentation for v1
…ements to save (less unwraps)
|
@flxzt, @Doublonmousse, I'd say this PR is ready for review. Take all the time you need of course, and feel free to ask me for information, changes, etc. 🫡 |
|
Another thought on the atomic saving denomination. That would be the case for a file format where any change we do inside the file is mapped back to the file through an efficient diff (and not a full write of the new engine state). For sqlite (#1173 (comment)), that would be the case because modifications to the file can be mapped back to the file through a push/commit |
I have seen "atomic saving" be used to mean somewhat different things, but for the most part it seems to mean what we are doing here (Geany wiki, 15 year old stackoverflow thread, etc...). A more efficient saving mechanism using diffs might be interesting, but would probably be very complex to implement and then have to deal with forward- and backward-compatibility... That being said, we should maybe tinker a bit more with the implementation as for instance metadata, permissions, and symlinks are not handled... |
|
@Doublonmousse Apologies for the delay, let me know what you think |
The overall goal of this PR is to improve the handling of
.rnotesave files.tl;dr
* improvement factor measured only with respect to
EngineSnapshot <-> bytesMain changes:
zstdinstead ofgzipHere's a diagram of the simplified process to get from
EngineSnapshotto a.rnotesave file and vice-versa.fixes #1128
Note that this PR is a re-write of #1177, some of the important changes being:
(+)parallelism for serialization and compression(+)the introduction of a file version separate from the version of Rnote(-)scrapped save preferences (not worth complexity cost)(-)scrapped alternative serialization methods (for compatibility reasons)