feat: Implement KV as a write-through cache #2732
Open
+754
−410
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We've had reports from users observing long run times from
atuin kv list
andatuin kv get
; this can happen when the record store has lots of KV entries, even if the actual number of unique keys is not that high (e.g. from automatically creating entries regardless of the current value). Both the aforementioned commands built a map by scanning the entire record store each time.This PR changes this so that the KV store is a write through cache. During sync (and manually with
atuin kv rebuild
), we iterate over the record store and build up the cache, which we then use to query from.set
anddelete
calls write to both the cache and the record store.atuin-kv
cratekv.db
, the location of which can be overridden withATUIN_KV__DB_PATH
or set viakv.db_path
atuin kv rebuild
to rebuild the KV cache (also rebuilt during sync)atuin kv delete
atuin kv ls
as an alias toatuin kv list
atuin kv set -n "" -k key value
; the other commands don't perform this check, in case users have existing empty-string-namespace entries they want to use or deleteI also tried to allow setting a value via
atuin kv set key value
(without the explicit--key
argument), but was unable to get this to work while maintaining backward compatibility.Checks