Persistent "Token expired" error after JWT expires

[moritzploss-k]

Hi and thanks for making this library!

We are exploring to use brod_oauth for SASL/OAUTHBEARER authentication towards Kafka. Our JWT tokens are valid for 30 minutes, and it works fine to connect to Kafka while the token is valid. However, after the token expires, we get errors like this, which are only resolved by an application restart:

exception error: {sasl_auth_error,<<"Token expired at: 1742309468000 (2025-03-18T14:51:08 UTC) (ErrId: 2791d48d)">>}

Looking at the code in brod_oauth, my understanding is that tokens are only used to establish a connection, but that connections may live longer than the corresponding token, and that tokens for a given connection don't get refreshed dynamically. Is that correct?

Any help would be much appreciated!

Our Config

Our brod config looks like this:

{brod, [
    {clients, [
        {client_name, [
            {endpoints, [{"${KAFKA_HOST}", ${KAFKA_PORT}}]},
            {ssl, true},
            {sasl, {callback, brod_oauth, #{token_callback => fun module:callback/1}}}
        ]}
    ]}
]}

And a call to module:callback/1 returns the following, where Token is a JWT access token:

{ok, #{token => Token}}

Subsequent calls to module:callback/1 return the same token as long as the token is not expired, and a new token after the token expires.

[vikas15bhardwaj]

Hi @moritzploss-k your callback should refresh token once it is expired. In one of our implementation we are caching token in ets and keep track of its expiry time and before it expires we update token in ets. So when brod auth calls it, it always get a valid token.

[moritzploss-k]

Hi @vikas15bhardwaj, thanks for the reply! My callback does refresh tokens once they expire, using a similar mechanism as the one you describe.

[vikas15bhardwaj]

Interesting, we haven't seen this problem. Can you replicate this problem locally maybe by adding debug flag to config e.g.

{brod, [
    {clients, [
        {client_name, [
            {endpoints, [{"${KAFKA_HOST}", ${KAFKA_PORT}}]},
            {ssl, true},
            {sasl, {callback, brod_oauth, #{debug => true, token_callback => fun module:callback/1}}}
        ]}
    ]}
]}

debug true prints some output to console, maybe helpful to debug it. Its worth adding some debug to your callback as well.

[vikas15bhardwaj]

Also what version of Kafka product you are using? Is it on-prem, cloud?

[moritzploss-k]

thanks, I'll give that a try and add some more debug logging. We're running Kafka 3.7.1 with Strimzi's Kafka operator in a self-hosted setup.

[whesleybarnard]

@moritzploss-k were you able to resolve the issue on your side? We are also looking to use this plugin and will probably face the same token expire issues.