Support for Multiple Queries of variable size.

[DellanX]

The Short Version

It'd be nice to have a way for being able to query multiple feeds(of a variable size), and to aggregate their data together. This is something that would probably need discussion, as I figure there are multiple strategies for it.

To me, the main reason I ask, is to see about a way to support multiple queries with caching, pagination, deduplication and status information

Long Version

I'm trying to figure out how to combine multiple queries under one state machine. I'm pretty sure that this is possible, just don't know the best way to pull this off with the API. (I figure some application of useQueryCache())

I have a complex scenario where I am pulling a user-defined amount of feeds (say size M), and each feed is paginated (default 20 elements). This feed system is unfortunately the key feature of my application, and thus not something I can really simplify.

Since my UI only needs to render the latest created 20 elements, I sort the aggregation of each feed, and slice 20 from that dataset. Now comes the fun part, how to handle user pagination, reactivity and caching?

My old system (based on tanstack):

1. I have a custom class called MultiPaginator, that tracks all the configured feeds, as well as defines an aggregate page position.
2. I execute the initial fetch by doing a Promise.all(feeds) operation, and I pass the returned data into a new instance of MultiPaginator
3. When the user requests to load the next page, MultiPaginator will do some checks to see the next page has reached the end of each feed, and if so, it'll fetch the next 20 elements for those feeds. (Similar process for previous page). Each fetch operation runs through the cache and thus populates for future queries.
4. MultiPaginator will do some logic to determine variables such as cursor position, length, and totalLength
5. The exposed nextPage() and prevPage() methods were wrapped with a useAsyncData() composable from vueuse.

Where the challenges are for migrating to this library (by step):

1. As of now, I don't see a documented way to execute a query without reactivity. So I cannot nest my queries as I did earlier. A workaround, I could technically create a composable that creates an array of M instances of useQuery, That said, I foresee trouble when the user clicks a button to add another feed. (In the past I used Tanstacks QueryClient.fetchQuery )
2. This step is pretty unaffected, I could create an instance of MultiPaginator just fine, as a lot of the aggregation functions are the same.
3. The hard part, if the user triggers the nextPage() or prevPage() function on my MultiPaginator class, the class will retrieve data, but I don't know how to inject it into the cache.
4. This step is complete unaffected.
5. I could technically define queries around my pagination functions, but the query keys wouldn't really be valid for a cursor based implementation.

I'm starting to wonder, perhaps there is a better way. And to which, I'd love to help contribute to this library to pave the way for one. (I'll likely still build something for myself in the meantime).

I did see #102, my main problem, is that my queries are of a variable size.

I'd especially like if I had a single UseQueryReturn, instead of the 3 I'll have with my current development path.

[posva]

This seems to be what #178 does

[DellanX]

Ooo, you're right! I think I overlooked it, as I was imagining merge as merging the same dataset... but it could definitely be used to merge and paginate multiple ones!

I think if I combine #178 with 'Optimistic updates', I think I can pull off everything here. Specifically:

1. I'd use useInfiniteQueries to handle pulling and managing data. (This will, unfortunately, use something like calendars as the cache key)
2. I'd use optimistic updates to fetch/inject data into the cache for optimization. (so if the user turns off bob's calendar key ['calendars', '123', 'events'] then turns it back on, no refetching happens as the data is already cached). That is, queryCache.setQueryData and queryCache.getQueryData.

I may need to see if there is a way to ensure that the calendars cache key doesn't do anything funky. For example, if the user refreshes the page, the feed selection needs to match the dataset pulled from the cache.

That said... I think this is going to be much more elegant code and better performance than my older system.

By the way, I really enjoy the way that optimistic updates were implemented. It will grant me much more flexibility than I had in the past. (Perhaps similar was possible with Tanstack, but I didn't find a lot of documentation on the API for these purposes).