Skip to content

feat: Implement KV as a write-through cache #2732

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

BinaryMuse
Copy link
Member

@BinaryMuse BinaryMuse commented May 2, 2025

We've had reports from users observing long run times from atuin kv list and atuin kv get; this can happen when the record store has lots of KV entries, even if the actual number of unique keys is not that high (e.g. from automatically creating entries regardless of the current value). Both the aforementioned commands built a map by scanning the entire record store each time.

This PR changes this so that the KV store is a write through cache. During sync (and manually with atuin kv rebuild), we iterate over the record store and build up the cache, which we then use to query from. set and delete calls write to both the cache and the record store.

  • Introduce the atuin-kv crate
  • Introduce a new database file, kv.db, the location of which can be overridden with ATUIN_KV__DB_PATH or set via kv.db_path
  • Add new command atuin kv rebuild to rebuild the KV cache (also rebuilt during sync)
  • Add the ability to pass multiple keys to atuin kv delete
  • Add atuin kv ls as an alias to atuin kv list
  • Disallow setting a key with an empty namespace via atuin kv set -n "" -k key value; the other commands don't perform this check, in case users have existing empty-string-namespace entries they want to use or delete
  • Add help messaging for all commands and options

I also tried to allow setting a value via atuin kv set key value (without the explicit --key argument), but was unable to get this to work while maintaining backward compatibility.

Checks

  • I am happy for maintainers to push small adjustments to this PR, to speed up the review cycle
  • I have checked that there are no existing pull requests for the same thing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant