Brainstorming GraphQL to orpc migration

[strblr]

Hello,

After having tested orpc on a couple of personal projects, this will be my go-to library for every new project going forward, no doubts! Awesome work.

I'm working since a few years on a big app for a client, and we're using GraphQL over websocket with graphql-live-query, and urql on the client. The idea is that we have a bunch of queries that stay alert for updates. On the backend, graphql-live-query allows me to invalidate them using keys which reruns targetted queries and updates are then sent to all relevant clients.

It works well but I've become frustrated with GraphQL lately especially after working with orpc. So I've been brainstorming a migration plan to orpc. Here are the two main challenges:

1. Using zod to resolve some fields by fetching a database. To avoid n+1 query problem, I'd use dataloaders. And because zod doesn't allow for custom parse context (to pass the dataloaders), I need to use AsyncLocalStorage.
2. Pushing query updates from backend to frontend

Two designs come to mind:

1. Using event iterators for live queries

Basically something similar to a trpc plugin I wrote some months ago: trpc-live.

Advantages:

• Simple to think about
• No network overhead, fresh data sent as soon as available

Problems:

• Not possible before websocket support, I've way too many live queries (SEE over http/1 has a max of 6 concurrent requests)
• AsyncLocalStorage loses context in async generators so output validation would need to happen "inside" the generator
• Unsure how auth / permission could be re-checked before a rerun
• Tricky tanstack query integration (having to use a custom version of streamedQuery)

I've been thinking of something like this where a middleware would transform the output to a stream:

export const test = os
  .input(z.object({ id: z.string() }))
  .use(async ({ next }, { id }) => {
    async function* run() {
      const result = await next();
      yield result.output;
      for await (const _ of eventSource(id)) {
        const result = await next();
        yield result.output;
      }
    }
    return { output: run() };
  })
  .use(auth)
  .output(z.object({ num: z.number() }))
  .handler(async ({ input: { id } }) => {
    return { num: Number(id) + 1 };
  });

This of course doesn't work and the typing is all wrong because the middleware can't know the type of .output.
But the idea would be to have regular output validators and handlers, and have the generator part somewhere higher up. This way, I can use middlewares for auth and injecting AsyncLocalStorage after the async generator stuff, and this way I bypass all the issues about context loss in async generators and rerunning auth/permission checks on live updates.

Is there a way to do something like that, or is there even a better way?

2. Normal queries + single SEE stream for update events + client-side refetch

Having just a single never-ending stream of updates that tells the clients what queries to refetch.

Advantage:

• No need for websocket
• No need for server-side invalidation and async generators for live queries, so overall way simpler

Problems:

• 2 roundtrips before clients actually have the updated data (update event from server => client, refetch from client => server, fresh data from server => client)
• On live update, the server will get requests from every client subscribed to the data all at once

Please let me know your thoughts.
Thank you!

[unnoq]

Thanks for your thoughts.

Could you help me understand a bit more about these points?

• AsyncLocalStorage loses context in async generators so output validation would need to happen "inside" the generator
• Unsure how auth / permission could be re-checked before a rerun

I believe bellow example almost the same: , every yield is validated, except the auth middleware only run on first time

export const test = os
.input(z.object({ id: z.string() }))
.use(auth)
.output(eventIterator(z.object({ num: z.number() })))
.handler(async function* ({ input: { id } }) => {
  for await (const _ of eventSource(id)) {
      yield { num: Number(id) + 1 };
    }
});

I also plant to support https://crossws.unjs.io/ which has widely adapter from websocket to sse

[strblr]

Thanks for the quick answer.

• About AsyncLocalStorage, I want to use zod transforms to resolve certain fields GraphQL-style (for example transforming an id into an object fetched from the database). If you have an array of n elements, each with a field to resolve, that would be n calls to the database. This is a typical problem with GraphQL, mitigated with dataloaders. To use them in zod transforms, they would need to be passed as parse context. Zod doesn't allow for custom context so one solution is AsyncLocalStorage, where you can pass a context value to an async callstack.

The context of AsyncLocalStorage gets lost in generators, this is discussed here: nodejs/node#42237, so it's not suited for orpc event iterator handlers. However, I just figured that I could write my own dataloader that just batches db calls until the next tick so I don't need AsyncLocalStorage at all.

• "Unsure how auth / permission could be re-checked before a rerun"

If we have an auth middleware like the example you give, it does not recheck permission for every yield. But this would be necessary in live queries. Because otherwise, a user that is subscribed to data updates will keep getting them, even if the permission gets revoked later on. This would be a security problem.

Crossws looks nice, I didn't know it existed!

[unnoq]

AsyncLocalStorage

I believe this is not node.js problem, bellow example still works as expect if AsyncLocalStorage run inside a yield cycle (it almost the same your custom middleware, where you await next() will run inside a yield cycle too)

  const s = new AsyncLocalStorage()

  async function normal() {
    console.log('async fn', s.getStore())
    return 'hello'
  }

  async function* gen() {
    yield await s.run('hello', normal)
  }

  await gen().next()

...even if the permission gets revoked later on

I believe we cann't support this usecase with middleware, or you can create two separate procedure and one call other

[strblr]

Yes it works when used like this, but when wrapping orpc output validation in an asyncLocalStorage run and that output is an async generator, there is no way to preserve the context I believe, unless we call zod parse manually in the handler. Anyway it's not a problem because I just published this to bypass it: https://github.com/strblr/superload

Do you have a rough time estimation for websocket support? I think this is the only remaining blocker for my migration plan from GraphQL.

[unnoq]

after v1 I will work on it, v1 will release in 15h this month

[strblr]

Awesome!

[unnoq]

@strblr websocket support now 1.2.0