Use for Blob Storage ?

[MentalGear]

Thank you so much for developing these low-level lo-fi primitives and sharing them openly, important pioneering work !

I'd like to use your libraries in my local-first file sync engine (files first saved to the browser, then optional cloud sync). At the moment, it uses localforage, but if your storage library could be modified to also support Blob Storage, I would much rather use it with it's local vault encryption.

For my use case, it is essential that files remain encrypted not only at rest on the client device but also when transferred to and stored on a central synchronization server. Could you clarify whether the vault library currently encrypts data exclusively at rest/on‑device, or whether encrypted files can also be exported, shared, and transmitted securely while remaining encrypted?

Thank you again for your work on this project. I believe your library could be an excellent fit for my engine, and I am eager to understand its capabilities in this regard.

[getify]

Hmmm! Thanks for bringing this up. There are some challenges to adjusting the KV store system to work like this, but... it's definitely an interesting idea. And I would be thrilled to help you explore using the library.

---

Just to be clear: this repo (Storage) is just the plain unsecured KV Store (across the different browser storage backing mechanisms). I have other libraries that use Storage, including Local-Vault and Local-Data-Lock, which do the encryption stuff. So we might want to move the encryption specific discussions to one of those two library's issue threads, and keep this issue thread focused on the BLOB storage idea.

But in short: Local-Data-Lock does expose the mechanisms to do encryption/decryption of data that you're planning to transmit, so you can essentially "re-encrypt" data you read out of the vault before sending it on. If you wanted to transfer the entire vault (in its already encrypted state), you could just read its value out of the KV Store and push it wherever you wanted.

---

The KV store uses standard JSON serialization on any non-primitive value stored in a key, so that would need to change to allow a non-object binary value to be stored without that JSON round-tripping. This may require some tweaks to the way the API methods work, not sure.

Moreover, a big limitation is that some of the storage backing mechanisms flat out can't hold binary -- namely, the cookie, localStorage, and sessionStorage mechanisms -- so for those we would have to do string serialization (base64) which can be expensive (performance + storage size). The alternative (not preferred) would be to not allow binary data storage in those backing mechanisms.

IndexedDB is easy to support binary blobs stored.

OPFS has another little nuance: the OPFS-Worker mechanism requires serializing data across the Worker process/event boundary. This can support binary data, but it's a little more complicated (using transferrable objects and array buffers), and a little narrower on browser support (not bad, but... not as broad as current).

---

Another side note: for the Local-Vault library, it stores your entire vault inside a wrapping JSON object, so we'd need further limitations/changes on that library for encrypting and storing a blob (separately from the vault). Probably would need to devise a way to link a field in the vault (JSON object) to a separate key alongside the vault, that is an encrypted blob value.

This is all possible, but will require a decent amount of effort.

[MentalGear]

Thank you so much for the detailed explanation!

I agree that continuing the encryption-specific discussion in the local-data-lock repo will help keep things more organized: mylofi/local-data-lock#13

Regarding expanding storage to support blobs or binary data: using base64 for legacy browser formats makes sense to ensure broad compatibility and feature parity. For local-vault, if storage is enhanced to handle binary/blob types, I think it would be very valuable to update local-vault to manage these files separately.

Thx again for openness to explore these ideas, I think it will expand storage and the encryption libraries greatly in their usefulness for the lo-fi sector, as at the moment a lo-fi binary storage (and sync engine) is what's still missing the space.

1