Increasing the number of overloads for pipe()?

[jrista]

I am a huge fan of RxJs, and have been using it heavily for a number of years now as a basis for purely functionally reactive programming in JavaScript. By that, I mean I no longer write imperative code, I pretty much write functional code (with occasional objects where methods are also implemented with RxJs, and in a functional manner). I've started making pretty heavy use of pipes...both the Observable.prototype.pipe() and the unbound, free pipe() function. The latter I use as my primary means of implementing custom operators, which are then composed into Observable pipes.

One of the challenges I have run into is the 9-operator limit of the existing pipe overloads...both on the Observable.prototype.pipe() and the unbound pipe() function. As soon as you surpass 9 operators, you suddenly lose typing, as the 10th overload's return type is UnaryFunction<T, unknown>, so this overload is generally unusable, and TypeScript always freaks out:

export declare function pipe(): typeof identity;
export declare function pipe<T, A>(fn1: UnaryFunction<T, A>): UnaryFunction<T, A>;
export declare function pipe<T, A, B>(fn1: UnaryFunction<T, A>, fn2: UnaryFunction<A, B>): UnaryFunction<T, B>;
export declare function pipe<T, A, B, C>(fn1: UnaryFunction<T, A>, fn2: UnaryFunction<A, B>, fn3: UnaryFunction<B, C>): UnaryFunction<T, C>;
export declare function pipe<T, A, B, C, D>(fn1: UnaryFunction<T, A>, fn2: UnaryFunction<A, B>, fn3: UnaryFunction<B, C>, fn4: UnaryFunction<C, D>): UnaryFunction<T, D>;
export declare function pipe<T, A, B, C, D, E>(fn1: UnaryFunction<T, A>, fn2: UnaryFunction<A, B>, fn3: UnaryFunction<B, C>, fn4: UnaryFunction<C, D>, fn5: UnaryFunction<D, E>): UnaryFunction<T, E>;
export declare function pipe<T, A, B, C, D, E, F>(fn1: UnaryFunction<T, A>, fn2: UnaryFunction<A, B>, fn3: UnaryFunction<B, C>, fn4: UnaryFunction<C, D>, fn5: UnaryFunction<D, E>, fn6: UnaryFunction<E, F>): UnaryFunction<T, F>;
export declare function pipe<T, A, B, C, D, E, F, G>(fn1: UnaryFunction<T, A>, fn2: UnaryFunction<A, B>, fn3: UnaryFunction<B, C>, fn4: UnaryFunction<C, D>, fn5: UnaryFunction<D, E>, fn6: UnaryFunction<E, F>, fn7: UnaryFunction<F, G>): UnaryFunction<T, G>;
export declare function pipe<T, A, B, C, D, E, F, G, H>(fn1: UnaryFunction<T, A>, fn2: UnaryFunction<A, B>, fn3: UnaryFunction<B, C>, fn4: UnaryFunction<C, D>, fn5: UnaryFunction<D, E>, fn6: UnaryFunction<E, F>, fn7: UnaryFunction<F, G>, fn8: UnaryFunction<G, H>): UnaryFunction<T, H>;
export declare function pipe<T, A, B, C, D, E, F, G, H, I>(fn1: UnaryFunction<T, A>, fn2: UnaryFunction<A, B>, fn3: UnaryFunction<B, C>, fn4: UnaryFunction<C, D>, fn5: UnaryFunction<D, E>, fn6: UnaryFunction<E, F>, fn7: UnaryFunction<F, G>, fn8: UnaryFunction<G, H>, fn9: UnaryFunction<H, I>): UnaryFunction<T, I>;
export declare function pipe<T, A, B, C, D, E, F, G, H, I>(fn1: UnaryFunction<T, A>, fn2: UnaryFunction<A, B>, fn3: UnaryFunction<B, C>, fn4: UnaryFunction<C, D>, fn5: UnaryFunction<D, E>, fn6: UnaryFunction<E, F>, fn7: UnaryFunction<F, G>, fn8: UnaryFunction<G, H>, fn9: UnaryFunction<H, I>, ...fns: UnaryFunction<any, any>[]): UnaryFunction<T, unknown>;

I have two cases where sometimes I need more than 9 operators in a pipe:

1. When I am adding logging (often temporary) to debug something (notably when debugging Ionic apps on actual devices, where using a proper debugger is not an option.)
2. When composing a larger number of custom operators into higher level functions or larger scale processes.

Most of the time, its the logging scenario, where I may need to temporarily add some taps between operators to figure out what's going on, and that will frequently bust the 9-operator limit. I am also finding more value in doing basic performance timing using custom operators...one operator placed at the beginning of a sequence of operators to start timing, and another placed at the end to report the timing. This is often combined with taps for other logging. In any case, because the moment we hit that unknown return type once we surpass 9 operators, it can make some debugging in certain circumstances quite challenging.

The other case that is becoming more common, as I rely more and more on custom operators to encapsulate business behavior, are my higher level Observable pipes. Sometimes I may compose 10+ operators, and in those cases to get around the 9-overload limit, I am creating additional operators simply to keep my total operator limit under 10, without any other real design value.

I'm wondering if we could increase the number of strongly typed overloads of the pipe() function and the Observable.prototype.pipe() method. Could we expand it to 20 operators, or so? That would make temporary logging with taps much, much easier, especially if you already have 9 operators and need to tap between each to track the state of your value as it moves through the pipe, you can hit 20 operators easy.

Kind Regards!

[voliva]

Although it can be annoying sometimes, I'm not fully convinced it's necessary tbh. Specially because it's easily fixable:

something$.pipe(
  A(),
  B(),
// ...
  H(),
  I(),
  J(),
  K()
);

is equivalent to:

something$.pipe(
  A(),
  B(),
// ...
  H()
).pipe(
  I(),
  J(),
  K()
);

I usually don't make longer pipes than 5 operators.... by that point I usually have a meaningful name I can give to that stream so I can keep composing it down.

Do you already have a pipe with 8 operators and you want to debug it by adding 3-4 taps? Then just put in a ).pipe( in between and that fixes it.

In a way I like having a soft limit at 9, because this way if I ever reach it, I get reminded that probably I need to break that stream into smaller bits, which does help in readability.

[jrista]

Yes, I've done this on occasion. I don't really like doing it as it is a bit jarring to have the extra pipes in there, when it shouldn't really be necessary. A 9-operator limit is pretty small. I try to write code that "tells a story" so it is very clear, and with higher level code, composing descriptive operators sequentially makes for really nice code, and it would be great to be able to compose more operators in a single pipe without having to break it down for no other reason than an arbitrary limit.

An example of a custom operator that composes several other validation operators:

export const verifyRequest = (
  authorityId: number,
  cardNumber: string,
): UnaryFunction<Observable<Authority>, Observable<Authority>> =>
  pipe(
    tap(measureTiming()),
    verifyAuthorityExists(),
    verifyCardNumberForAuthority(authorityId),
    verifyAuthorityIsActive(),
    verifyBinCodeIsEnabled(cardNumber),
    verifyIntegrationSpecified(cardNumber),
    tap(reportTimingObserver('verifyRequest')),
  );

A later method that composes the above validation operator, with other operators, to perform a specific higher level workflow:

this.authorities.loadAll().pipe(
      tap(measureTiming()),
      findAuthority(cardNumber),
      verifyRequest(authorityId, cardNumber),
      resolveIntegration(this.integrationResolver.knownIntegrations, cardNumber),
      queryBenefits({
          cardNumber,
          householdId,
          benefitsCriteria: query,
          includeOriginal,
      }),
      normalizeBenefits(),
      sortEmptyBenefitsLast(),
      logError(),
      tap(reportTimingObserver('resolve')),
);

A mere 9 operators is a very arbitrary and tight limit. I already do what I can to break things down. I aim to follow SRP as much as I can. The validateRequest operator is an example of breaking down the operators in a single pipe by aggregating some of them into another operator. This pipe in fact has numerous custom operators, each of which compose standard rxjs operators and possibly other custom operators. So it isn't like I'm not trying to write clean, concise code, I am. However, there are cases where you just need more than 9 operators...without meaning the code becomes hard to read or anything like that. In fact, some longer pipes can be extremely clear, if written properly, about what behavior they compose and the order in which the data flows through them.

If I needed to debug the above stream...which in fact I did, and what lead to creating this request...I already had 9 operators. The moment I added a tap for logging, the typing fell apart. I feel very strongly that we could benefit from more pipe() overloads.

I'll also offer that, when adding taps for debugging purposes, they are often temporary. This makes the whole chained pipes option far less than ideal, as it means temporarily breaking up the pipe just so you can add temporary taps for logging. So you have to change the code more than necessary to solve runtime problems in certain environments. When you resolve the issue and remove the taps, you would then have to merge multiple pipes back into one.

Its all very unnecessary, if we could get a pipe that supports more than just 9 operators.