Moving to a content addressed cache for SQLite

[brecert]

This is a wishlist item of mine for the upcoming switch to SQLite for the database, I ask that the cache be considered to be made content addressed- that is the content hash of the file is the "address" (file name in this case) of it.

I believe the cloud already does this for assets, so I'd like to see it happen for the cache too.

External references could be made into fs level content addressed links pointing to the actual content they point to, this approach would also work for asset variants.

source "https://example.com/example.png" -> link "7821e3..." -> content "123456..."

Some immediate benefits of this:

• the database being corrupted or needing to be rebuilt no longer means the entire cache is invalid.
• deduplication of assets happens automatically
  • this is relevant to me because there have been many instances of many duplicate assets with the current cache
    (I checked this using a script I made some time ago because I was curious)
  • multiple instances can all use the same cache without duplicates
    (similarly, due to often running multiple different instances, I end up with a lot of duplicate items in the cache)
• sharing a cache between computers, and having it on a network device becomes much easier and less error prone

Some potential future benefits:

• mesh networked transfers of assets become much easier to implement, all of the metadata required for the transfer is part of the cache itself.
  this can be done by viewing the content addressed cache as a distributed hash table.
• resonitepackages wouldn't have to reimport everything for everyone every time they're loaded in, instead being able to rely on the content addressed nature of the cache.

A minor downside to this approach (file based) to content addressing is that you're more reliant on the host fs but I don't believe it'd be too much of an issue, I've implemented fs based content addressed storage in the past that was multiplatform without major issues.