Feature request: Support for relational actions

[jarvisluong]

I'm building an application that support offline usage and found this enqueue to be helpful! However as what I saw in the code the queue only support 1-level merging. What I mean is that if, for example, I created a Tag (which also have its own id from server after commit) and then I make a Todo which use the tag. Before the Tag is created, we don't know its id for the TODO be created. :/

[sorodrigo]

We still have some TODOs before releasing into the wild and could use some help. The main pending issue is the merging strategy, we want to expose the function see #4 . But if I understood correctly what you want is to relate different resources optimistically. Do you have a proposal on how we could do this?

[sorodrigo]

This is me thinking out loud:

• We are using a unique key to merge resources in the queue, and sometimes that unique key is not an id because the resource hasn't been created in the backend, we'll call this key optimistic id.

• When you create two resources and link them optimistically, you have to ensure that the resource A is created in the database before the resource B can be created in the database, so that way it can reference its id.

• Also when creating the resource B, sending the optimistic id as a foreign key won't work. So you need a mechanism that updates Action B with the correct foreign key once that Action A has resolved successfully.

• Actually I see this possible in the following way:

  • on COMMIT we save the DB resource in our state, but we also save a meta key that corresponds to the optimistic id. On peek we replace the optimistic id with the DB's is, we find this some way using the optimistic id.

  • Actually if this is the solution, peek would need access to the whole app's state, which is not great.

[jarvisluong]

What you thought out loud was actually the same as mine! I was even thinking that if we have the ability to listen to each time state changed just like redux-saga the implementation would be much easier. However I just got an idea like this:

• If I understand the library logic correctly, if the request is successful then the middleware can dispatch the COMMIT, how about we expose the event that the request is successful and users can listen to that event with the value of request response, and they can decide if a new action should be dispatched (in this case it can be the action that replace the optimistic id with the real database id)

[calumpeak]

I've been looking at offline relational resource resolution at work and we have a solution possible solution that I was going to give back to redux-offline when it's ready.

We were looking at doing it via dequeue as that gets the action that's just been resolved (action.meta.completed). If it was of type CREATE you can go back through through the queue and update ID's/Keys that had the existing key on it (commit action would need to carry the 'temp' id, the same way that you can resolve it in reducers after the fact) and return out the new outbox.

The main problem with this method is that you cannot merge over CREATE actions as if there are relationships between created entities on updates (Entity 1 has a pointer ID to Entity 2) then you'll hit the database with that relationship for an ID that

1. doesn't exist
2. the ID from entity 2 has yet to be generated from the db so entity 1 will point to a bad link

The queue order matters for relational entities, so as far as I've found, you can either have your queue squashing/merging, or you don't squash, and allow updates via dequeue across the outbox to make sure entities that have been created with tempId's can be updated when a completed action comes through.

[sorodrigo]

I don’t love the idea of triggering a global event to share the commit payload. It’s like going outside of redux and feels dangerous.

The dequeue approach sounds way better, if we pass the optimistic id to the meta of the commit we then can try to replace its occurrences. I don’t completely get what’s the catch with this.

With dequeue all pieces seem to fit naturally! 😛

[calumpeak]

There's no issues on the dequeue method if you don't use smart-queue/merging. It's if you want to use something like smart-queue where the issues arise.

E.g.
Given I'm offline and I create an entity 1
{ CREATE 1}
And I create entity 2
{CREATE 1} - {CREATE 2}
And I update entity 1 to have a relationship with entity 2 (no squash)
{CREATE 1} - {CREATE 2} - {UPDATE 1, payload: entity 2} //This is fine, no issues
When update 1 gets squashed into create
{CREATE 1, payload: entity 2} - {CREATE 2}// Pointer created to entity 2 before entity 2 exists.
And I go online.

The first CREATE carries a payload to entity 2 which has been given an optimistic ID. That payload is stored in the DB. Create 2 will come back with a completely different ID to what is now stored in Entity 1's pointer to Entity 2.
If you do this you can't squash (at all, UPDATE will have the same issue), or you have to re-order, but that will not always be possible and is more likely to introduce more issues.

[sorodrigo]

Oh now I get it! Yeah that's true, what if the relation was created in the other direction?
{CREATE 1}
||
{CREATE 2}
||
{UPDATE 2, payload: entity 1}
||
{CREATE 1} - {CREATE 2, payload: entity 1}

Which sounds maybe difficult to achieve always... I'm out of ideas

Edit: I think this is a really specific use case, and very hard to solve in a general way. If we're exposing the merge fn maybe this should be handled in userland.

[calumpeak]

Yeah, if it's backwards it's fine as the first entity will get resolved first, on dequeue, you can walk the outbox and replace the references to it. You can do it without squashing fine as it keeps the order perfectly and you can walk the outbox after each resolved action, they just won't be merged down.

If you're allowed to create your IDs client side then it's fine too (Inc squashing) but you have know way of knowing conflicts, and that's not advisable.

The only other thing you can do if you want squashing but have relationships is have separate actions for them that don't contain the additional metadata for smart queue so they get added to the outbox in the correct sequence.