Disable caching for a certain subfolder of a cached folder

[dsavenko]

Hi,

Just a question. If I do

remoteStorage.caching.enable('/a/')
remoteStorage.caching.disable('/a/b/')

I expect caching will be enabled for /a/ and everything inside of it, but not for /a/b/ and everything inside of it. Am I right?

I'm asking, because the docs are a bit confusing. They say:

When caching is enabled for a folder, it causes all subdirectories to be cached as well.

but don't mention the case when I specifically disable caching for a subdirectory.

[raucao]

Good question. Did you try it and just check in your IndexedDB if it's cached or not?

The way sync works is that the library checks the highest parent folder of cached paths for a changed ETag (i.e. something in a subfolder was updated), and then traverse the folder hierarchy until it finds the one that was updated, and then in it the document that was updated, added, or deleted. In theory it's totally possible to ignore explicitly uncached subfolders, but personally, I don't know if that extra code is there off the top of my head.

In general, I think it's a good idea to not have different cache settings for things down the tree, but instead use a folder layout where you have one subfolder that is cache-worthy and one that is not.

Aside from it definitely being worth it to answer this question properly, I think it would also be interesting to know your use case for this. If it's a good one, and we find out that the feature doesn't exist, then we have more data about weather it's worth adding it to the library or not.

[dsavenko]

I checked IndexedDB, it looks like the disabled folder's content is not cached (which is what I expected).

At the beginning of the development, I decided to keep my settings in the file config.json at the root of my scope (which seems OK).

After some time in dev, I realized I needed something like a remotely synchronized Set, so I could check if a particular key exists in it or not. I decided to create a folder in my scope for that, and put (almost) empty files there with the names being md5 hashes of my keys (I'm not concerned about collisions). This way, I can easily check if the file is there or not.

But when this Set-folder grew to a couple of hundred files, some things became very slow, especially the initial loading, when the user logs in via Google Drive (I use it as the backend). For instance, my config file may not be loaded for a really long time, because many of those md5-files are being loaded first (it seems there's no particular order). And I wanted to get the config as fast as possible.

So, I disabled caching for it (and it helped). But I wanted to keep caching enabled for the config file which still resides in the root of the scope. I don't think I could've planned ahead to keep my settings in a sub-folder because of caching.

If you're wondering, here's the project I'm talking about: https://github.com/dsavenko/procrastinator

I know you recommend to use data modules instead of dealing with files manually. But it just doesn't agree with my engineering habits :)