Immediate mode: instant, synchronous document updates

[puppybits]

What is the feature?
When doing something like drag and drop the updates to firestore take too long so the UI jumps. This is even worse with transactions.

What version would this apply to?
v0.15

Do you have thoughts/examples on how the feature would be used (i.e. API)?

Yes. I already forked it and made it work: Optimal-AI#1

// Single document change w/ synchronous, optimistic data in Redux
firestore.mutate({ collection: 'collection/path', doc: 'document-id', someChange: 'value' });

// Batch multiple document changes w/ synchronous, optimistic data in Redux 
// (batches > 500 will be bundled and committed together)
firestore.mutate([
  { collection: 'collection/path', doc: 'document-id', someChange: 'value' },
  { collection: 'collection/path', doc: 'other-id', otherChange: 'otherValue' }
]);

// transactions w/ synchronous, optimistic data in Redux
firestore.mutate({
  read: { 
     myLockedDocument:  { 
        collection: 'path', 
        doc: 'document-id',
     }
  },
  write: [({ myLockedDocument }) => ({
      collection: 'path', 
      doc: 'document-id' , 
      someNumber: myLockedDocument.someNumber + 10
    })]
});

[prescottprue]

Interesting approach, thanks for posting! I would say one downside to this methodology is that if the write breaks a database rule (not confirmed until after the promise completes), then we would also need a way to revert the optimistic change. We would probably also want this to be opt in since it could cause a flash for some cases (would not be expected)

Have you run into those cases?

[puppybits]

@prescottprue I've updated our fork and integrated it into our app. Here's an illustration of the issue and the difference between the 2 different reducers.

Before.

After.

We're doing internal QA on our app but when it all passes we'd be happy to send a PR.

It does handle both success and failure cases when the firestore save reaches a resolution. Here's a link to our internal PR review to see the details Optimal-AI#4.

Also we have an additional follow-on feature planned which would unify single document changes, batches, transactions and optimistic updates into a single API call so manually sending an optimistic update would no longer be needed.

[prescottprue]

@puppybits awesome! Thanks for the work making the reducer and sharing - looking forward to the PR

Really like the idea of optimistic updates, so let me know if there is anything I can do to speed up the process of getting this in

[puppybits]

We've tested the cache reducer locally and it's all working. But the optimistic update message API was only designed to be used internally. It's too verbose for a consumer.

We just pushed an internal PR to the fork. It both unifies updating the cache reducer and firestore solo, batching and transactions into and single API. It's going through PR review and testing this week.

Optimal-AI#5

Changes included:

1. Simpler API for changing firestore documents.
2. Automatically updated the cache reducer to instantly reflect changes in Redux
3. Exposed a config to optionally enable onSnapshot includeMetadataChange
4. Added a config for a global FirestoreDataConvertor (We generate global decoder from our Typescript definitions).

const transactionPromise = firestore.mutate({
  read: { 
     myLockedDocument:  { 
        collection: 'path', 
        doc: 'document-id',
     }
  },
  write: [({ myLockedDocument }) => ({
      collection: 'path', 
      doc: 'document-id' , 
      someNumber: myLockedDocument.someNumber + 10
    })]
});

// cache reducer has already added the requested changes to 
// the in-memory overrides as an optimistic update

transactionPromise.then(() => {
  /* change has completed from firestore*/
  /* cache reducer has already removed the optimistic update and is using firestore's full query results */
}).catch(() => {
  /* failures will also trigger the cache to remove the optimistic overrides */
  /* use any of the existing failure messages to be notified and create custom failure functionality */
});

[puppybits]

@prescottprue Submitted a PR for this new reducer. The headline is that is' 100x faster for all changes.

The reason is that even single save operations have to round-trip over the network before the snapshot listener fires. For us saving a Firestore transaction in production took a minimum of 1.2s to show the results to the user. That dropped to 6ms. That's over 100x faster.

[cabello]

I am suffering from this issue also as I implemented drag & drop in one of by useFirestoreConnect queries.

@puppybits super excited about taking a look at your PR #335 I noticed the changeset is 68k lines @prescottprue do you have a suggestion on breaking this down into something that could be merged into main and released?

I agree with the architectural, data flow issue: the React component is backed by Redux connected to Firebase, the drag & drop sends the change request to Firebase, that takes ~150ms (most times is faster), the UI flickers between old state and new state, would be lovely to have an optimistic UI system in place for these. ❤️