Skip to content

Possible NPE due to race condition when updating token issuer map during adapter restart in Choreo Connect #3688

@sajith-madhusanka

Description

@sajith-madhusanka

Description

When the adapter server restarts, there is a possibility that the tokenIssuer instance is set to null, which can lead to a NullPointerException (NPE) during token validation. As a result, the client application receives the following error message:

{"description":"Error during validating the request","code":"102500","message":"Unclassified Validation Failure"}

Steps to Reproduce

  • Download the choreo-connect-with-apim.zip from the G Drive [6] and extract it.
  • Navigate to the choreo-connect-with-apim directory and run:
    docker-compose up -d
  • Tail the enforcer logs:

docker-compose logs -f enforcer

  • In the config.toml, disable the gateway token cache by adding the following configuration:
[enforcer]
[enforcer.cache]
  enabled = false
  maximumSize = 10000
  expiryTime = 15
  • Once all components are up, deploy the Pizzashack sample API.

  • Subscribe to the API and generate an access token.

  • Download the load_test.jmx file from G Drive [6] and execute a load test.

  • While the load test is in progress, restart the adapter using:

    docker restart choreo-connect-with-apim-adapter-1

  • Observe the below error stack trace in the enforcer logs.


enforcer-1  | Jun 05, 2025 5:53:28 PM io.grpc.internal.SerializingExecutor run
enforcer-1  | SEVERE: Exception while executing runnable io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed@71af37f1
enforcer-1  | java.lang.NullPointerException
enforcer-1  | 	at org.wso2.carbon.apimgt.common.gateway.jwttransformer.DefaultJWTTransformer.getTransformedConsumerKey(DefaultJWTTransformer.java:42)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.security.jwt.validator.JWTValidator.getConsumerKey(JWTValidator.java:169)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.security.jwt.validator.JWTValidator.validateToken(JWTValidator.java:92)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.security.jwt.validator.JWTValidator.validateJWTToken(JWTValidator.java:74)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.security.jwt.JWTAuthenticator.getJwtValidationInfo(JWTAuthenticator.java:561)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.security.jwt.JWTAuthenticator.authenticate(JWTAuthenticator.java:188)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.security.AuthFilter.authenticate(AuthFilter.java:205)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.security.AuthFilter.handleRequest(AuthFilter.java:162)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.api.API.executeFilterChain(API.java:42)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.api.RestAPI.process(RestAPI.java:192)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.server.HttpRequestHandler.process(HttpRequestHandler.java:63)
enforcer-1  | 	at org.wso2.choreo.connect.enforcer.grpc.ExtAuthService.check(ExtAuthService.java:79)
enforcer-1  | 	at io.envoyproxy.envoy.service.auth.v3.AuthorizationGrpc$MethodHandlers.invoke(AuthorizationGrpc.java:245)
enforcer-1  | 	at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
enforcer-1  | 	at io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
enforcer-1  | 	at io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
enforcer-1  | 	at io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
enforcer-1  | 	at io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)
enforcer-1  | 	at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:340)
enforcer-1  | 	at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:866)
enforcer-1  | 	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
enforcer-1  | 	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
enforcer-1  | 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
enforcer-1  | 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
enforcer-1  | 	at java.base/java.lang.Thread.run(Unknown Source)
enforcer-1  |

[1] https://drive.google.com/drive/folders/1IFJIlrI0l8aYcZuInj-ly8OZlXG7ZZ_G?usp=sharing

Version

1.0.0

Environment Details (with versions)

No response

Relevant Log Output

Related Issues

No response

Suggested Labels

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions