Support in-process channels in non-test code

[Meijuh]

I believe we have a valid case where in-process channels are useful in production code. Sometimes, a process in my app knows it can deal with a request itself, making it unnecessary to use a Netty-based channel. And always using RPCs also simplifies the architecture of my app.

Would it be possible to support a GrpcChannelFactory for in-process channels? Mostly for the reason that I then don't have to copy code from DefaultGrpcChannelFactory#destroy(). Also, this line of code https://github.com/spring-projects-experimental/spring-grpc/blob/main/spring-grpc-spring-boot-autoconfigure/src/main/java/org/springframework/grpc/autoconfigure/client/GrpcChannelFactoryConfigurations.java#L64 makes it a bit cumbersome to define my own GrpcChannelFactory, as I then don't have an instance of the Netty-based factory by default (which I still need).

[mkleine]

Thanks for raising this issue! We also have this requirement for production code.

[dsyer]

I'm pretty sure the in-process channel is designed by the Google team for testing only. What is it that means you can't call your service directly if it is in the local JVM? Also, @Meijuh could you share a small sample that has both a netty channel factory and in-process channels? I foresee issues with autowiring if you have multiple beans of type GrpcChannelFactory.

[Meijuh]

I'm pretty sure the in-process channel is designed by the Google team for testing only.

The Javadoc here: https://grpc.github.io/grpc-java/javadoc/io/grpc/inprocess/InProcessChannelBuilder.html says "The channel is intended to be fully-featured, high performance, and useful in testing." I don't read this as "for testing only". In fact, the way it is formulated made me use it in production. But I am happy to go in a different direction, if somewhere else it says in-process channels should only be used in tests.

What is it that means you can't call your service directly if it is in the local JVM?

I actually can, it is not a very big issue. However, aside from making the implementation simpler (you can see this in my example below), it also allows me to execute code inside the JVM asynchronously, as I am using the client executor from the in-process channel (which is not a "direct executor", but a ThreadPoolTaskExecutor).

Also, @Meijuh could you share a small sample that has both a netty channel factory and in-process channels?

@Component
public class ClientChannelManager implements DisposableBean {

  private final ConcurrentMap<String, ManagedChannel> channels;

  private final ManagedChannel inProcessChannel;

  private final GrpcChannelFactory channelFactory;

  private final String authority;

  public ClientChannelManager(
      ConcurrentMap<String, ManagedChannel> channels,
      @Qualifier("inProcess") ManagedChannel inProcessChannel,
      GrpcChannelFactory channelFactory,
      @Qualifier("grpcServerAuthority") String authority) {
    this.channels = channels;
    this.inProcessChannel = inProcessChannel;
    this.channelFactory = channelFactory;
    this.authority = authority;
  }

  public ManagedChannel getChannel(String target) {
    return authority.equals(target)
        ? inProcessChannel
        : channels.compute(
            target,
            (key, value) -> {
              final ManagedChannel channel;
              if (value == null || value.isShutdown()) {
                channel = channelFactory.createChannel(target);
              } else {
                channel = value;
              }
              return channel;
            });
  }

  @Override
  public void destroy() {
    inProcessChannel.shutdown();
    try {
      inProcessChannel.awaitTermination(30, TimeUnit.SECONDS);
    } catch (InterruptedException e) {
      inProcessChannel.shutdownNow();
    }
    // other channels are shutdown by the GrpcChannelFactory
  }
}

In the example above, you can see that doing something in-process is done using a simple ternary expression (where this.authority is the authority of the current JVM). As an alternative, I would have the same business logic twice; once in the gRPC server, and once where authority.equals(target) == true. The injected GrpcChannelFactory is the factory for Netty channels you where asking for.

I foresee issues with autowiring if you have multiple beans of type GrpcChannelFactory.

This is indeed the problem I was facing, where I would not inject the in-process channel in the ClientChannelManager, but my own InProcessGrpcChannelFactory. I don't have a good solution for this problem. Maybe something along the lines of a CompositeGrpcChannelFactory? Or maybe a way where developers can resolve the problem of having multiple beans of type GrpcChannelFactory themselves?
If you think it is a good idea too to have in-process channels in production code, then maybe a bunch of code from spring-grpc-test can be moved to spring-grpc-core, or a new spring-grpc-inprocess?

[svametcalf]

We also use the InProcess transport mechanism as a migration path. If you are in the process of splitting out an existing service into multiple services, it can be nice to do it progressively where one part of the application can be calling gRPC and the other can be JVM direct. Eventually, when the migration is complete, it just becomes a property change (or a feature flag) to swap over to network transport.

We have been running one of these services in production for many months now and have had no issues

[dsyer]

Maybe the best option for you both is to simply extend one of the existing channel factories and override the newChannelBuilder() method to look for a URL with a special prefix ("inprocess:" or something) and just create an in-process channel? That seems simpler than the snippet from @Meijuh above - but I'd still like to have an actual sample to play with to see if that's true - and the existing autoconfiguration would just back off.

It would be really helpful if you would try it. Maybe it could be a feature.

[Meijuh]

Maybe the best option for you both is to simply extend one of the existing channel factories and override the newChannelBuilder() method

I just did a brief attempt; DefaultGrpcChannelFactory is parameterized by ManagedChannelBuilder, so I would need to create my own MyChannelBuilder (which is neither an InprocessChannelBuilder nor NettyChannelBuilder, but more or less both), this is doable by delegating. However, I believe I would then also need to create my own "global" List<GrpcChannelBuilderCustomier<MyChannelBuilder>> that somehow reuses the auto configured global customizer for NettyChannelBuilders. I believe this makes it more complex than my current solution listed above.

Thinking about it a bit conceptually. I don't think having a single factory for both types of channels is the best way to go:

1. On in-process channels I would never perform health probes (to the HealthGrpc Service).
2. I would never enable retries on in-process channels.
3. Using a factory pattern for singleton in-process channel bean does not sound right.

Regarding 1. and 2.; if my type of managed channels are opaque, I would not know whether or not to enable/disable health probes and retries.

but I'd still like to have an actual sample to play with to see if that's true

Sadly, the code base I am working on is not open source, and it would be quite a lot of work to extract a minimum working example which I would be able to share publicly.

It would be really helpful if you would try it. Maybe it could be a feature.

I'd be happy to try out more suggestions and report back here. I'd be willing too to help out with a PR so that Spring gRPC supports in-process channels for production code, once I know which direction to go is best. It would be nice if the solution supports auto configuration for in-process servers too.

[dsyer]

I take the point about the interceptors, but how did the snippet above solve that problem? The thing that creates in-process channels is going to have to know which global interceptors are needed and which can be discarded. I don't see a good way to do that yet.

[Meijuh]

This is in fact one part I could not solve nicely:

@Bean("inProcess")
public ManagedChannel inProcessChannel(List<ClientInterceptor> interceptors) {
  if (interceptors.size() == 1
      && interceptors.getFirst().getClass().equals(ObservationGrpcClientInterceptor.class)) {
    log.info("Adding observation interceptor to in-process channel: {}", interceptors.getFirst());
    return InProcessChannelBuilder.forName(InProcessGrpcServer.NAME)
        .executor(clientExecutor)
        .intercept(interceptors)
        .disableRetry()
        .build();
  } else {
    // if at any point in the future this exception is thrown, reevaluate adding interceptors, at
    // the moment we only want to add an observation interceptor
    throw new IllegalStateException(
        "Unexpected interceptor(s) found: %s".formatted(interceptors));
  }
}

As you can see above, I wanted to reuse the only auto configured interceptor related to monitoring, with some safety built-in for future changes. The clientExecutor is the executor that is shared among all managed channels.