Open
Description
Steps to Reproduce
- Create a spring boot app with lognet
- Create any simple grpcService
- Create a restController having a reference to GrpcServerRunner
- Expose a rest end point in this restController to stop grpc server
- Deploy and run app in your local kubernetes cluster (You can run as spring boot app as well, doesn't have to be deployed to cluster)
- Tomcat and grpc server will be started. RestController and grpc services will get registered
- grpc service will be discovered and its status will be set to SERVING by default
- Now once everything is up and looking good hit the rest endpoint to shutdown grpcServer
- Now tomcat server is up and grpcServer is down
- Send a request to /actuator/health/grpc end point and it will continue to report the service as SERVING
Analysis
- Lognet's DefaultHealthStatusService seems to be having a bug
- It extends ManagedHealthStatusService whose onShutdown() method's documentation clearly indicates that "Implementation is advised to set status of all services as ServingStatus.NOT_SERVING"
- But DefaultHealthStatusService's onShutdown() implementation simply delegates the call to healthStatusManager.enterTerminalState(); without changing the statuses of services to NOT_SERVING in its locally maintained map
References
- https://github.com/LogNet/grpc-spring-boot-starter/blob/master/grpc-spring-boot-starter/src/main/java/org/lognet/springboot/grpc/autoconfigure/actuate/GRpcActuateAutoConfiguration.java
- https://github.com/LogNet/grpc-spring-boot-starter/blob/master/grpc-spring-boot-starter/src/main/java/org/lognet/springboot/grpc/health/ManagedHealthStatusService.java
- https://github.com/LogNet/grpc-spring-boot-starter/blob/master/grpc-spring-boot-starter/src/main/java/org/lognet/springboot/grpc/health/DefaultHealthStatusService.java
Fix
Any local state maintained by DefaultHealthStatusService should be cleared out and marked as NOT_SERVING upon onShutdown() method call.
Metadata
Metadata
Assignees
Labels
No labels