-
Notifications
You must be signed in to change notification settings - Fork 336
Description
What is the bug?
We noticed this issue in new clusters running version 3.2.0, where the bundled Java version is 24 instead of 21. When using Java 24, we encounter the following error on the server:
{"type": "server", "timestamp": "2025-09-18T14:44:51,862+02:00", "level": "ERROR", "component": "o.o.s.a.h.k.HTTPSpnegoAuthenticator", "cluster.name": "monitcopy1", "node.name": "osaitos376401-monitcopy1_client4", "message": "Login exception due to", "cluster.uuid": "U8GacOsoSV6zJZCizDTPRA", "node.id": "A_5_PZzqRO6Y75TS7zITgQ" ,
"stacktrace": ["javax.security.auth.login.LoginException: Security Exception",
"at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:682) ~[?:?]",
"at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:460) ~[?:?]",
"at org.opensearch.security.auth.http.kerberos.util.JaasKrbUtil.loginUsingKeytab(JaasKrbUtil.java:88) ~[opensearch-security-3.2.0.0.jar:3.2.0.0]",
"at org.opensearch.security.auth.http.kerberos.HTTPSpnegoAuthenticator.extractCredentials0(HTTPSpnegoAuthenticator.java:214) [opensearch-security-3.2.0.0.jar:3.2.0.0]",
"at org.opensearch.security.auth.http.kerberos.HTTPSpnegoAuthenticator$2.run(HTTPSpnegoAuthenticator.java:184) [opensearch-security-3.2.0.0.jar:3.2.0.0]",
"at org.opensearch.security.auth.http.kerberos.HTTPSpnegoAuthenticator$2.run(HTTPSpnegoAuthenticator.java:181) [opensearch-security-3.2.0.0.jar:3.2.0.0]",
"at java.base/java.security.AccessController.doPrivileged(AccessController.java:74) [?:?]",
"at org.opensearch.security.auth.http.kerberos.HTTPSpnegoAuthenticator.extractCredentials(HTTPSpnegoAuthenticator.java:181) [opensearch-security-3.2.0.0.jar:3.2.0.0]",
"at org.opensearch.security.auth.BackendRegistry.authenticate(BackendRegistry.java:330) [opensearch-security-3.2.0.0.jar:3.2.0.0]",
"at org.opensearch.security.filter.SecurityRestFilter.checkAndAuthenticateRequest(SecurityRestFilter.java:330) [opensearch-security-3.2.0.0.jar:3.2.0.0]",
"at org.opensearch.security.ssl.http.netty.Netty4HttpRequestHeaderVerifier.channelRead0(Netty4HttpRequestHeaderVerifier.java:90) [opensearch-security-3.2.0.0.jar:3.2.0.0]",
"at org.opensearch.security.ssl.http.netty.Netty4HttpRequestHeaderVerifier.channelRead0(Netty4HttpRequestHeaderVerifier.java:37) [opensearch-security-3.2.0.0.jar:3.2.0.0]",
"at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) [netty-codec-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) [netty-codec-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289) [netty-handler-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:107) [netty-codec-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1519) [netty-handler-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1377) [netty-handler-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1428) [netty-handler-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530) [netty-codec-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469) [netty-codec-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) [netty-codec-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:796) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:697) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:660) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) [netty-transport-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998) [netty-common-4.1.121.Final.jar:4.1.121.Final]",
"at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.121.Final.jar:4.1.121.Final]",
"at java.base/java.lang.Thread.run(Thread.java:1447) [?:?]",
"Caused by: java.lang.SecurityException",
"at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:683) ~[?:?]",
"... 49 more"] }
If we switch back to the same JDK we were using with version 2.18 (Java 21), everything works correctly. The error itself is not very informative, but the problem seems directly related to the Java version change.
How can one reproduce the bug?
Configure Kerberos authentication in OpenSearch and try to do a request. Our opensearch.yml
contains:
plugins.security.kerberos.acceptor_keytab_filepath: krb5.keytab
plugins.security.kerberos.acceptor_principal: HTTP/[email protected]
plugins.security.kerberos.krb5_filepath: "/etc/krb5.conf"
And in config.yml
is:
...
kerberos_auth_domain:
description: "Authenticate via Kerberos"
http_enabled: true
transport_enabled: false
order: 3
http_authenticator:
type: kerberos
challenge: true
config:
krb_debug: false
strip_realm_from_principal: true
authentication_backend:
type: noop
What is the expected behavior?
A clear and concise description of what you expected to happen.
What is your host/environment?
- OS: AlmaLinux 9
- Version 3.2.0
- Plugins:
opensearch-alerting
opensearch-asynchronous-search
opensearch-cross-cluster-replication
opensearch-custom-codecs
opensearch-flow-framework
opensearch-geospatial
opensearch-index-management
opensearch-job-scheduler
opensearch-knn
opensearch-ml
opensearch-neural-search
opensearch-notifications
opensearch-notifications-core
opensearch-observability
opensearch-reports-scheduler
opensearch-security
opensearch-security-analytics
opensearch-skills
opensearch-sql
opensearch-system-templates
query-insights
repository-s3
Do you have any screenshots?
Not applicable
Do you have any additional context?
Unfortunately, the stacktrace doesn’t provide much detail. We are not doing anything special in our setup, and the only difference appears to be the Java version. If there is a way to enable more verbose logging or additional debug output in this area, I would be happy to test it by building and deploying a custom version.