Malloc error on z/OS with Java 25 from Netty beta

[epj]

Describe the bug
Verbosegc and full stack will be included below, but error first presents as follows. Since the transport is failing, subsequent connection errors or related problems can happen later in the logs:

CWWKE0701E: bundle com.ibm.ws.transport.http:1.0.109.cl26012025121
6-0302 (99)[com.ibm.ws.http.internal.HttpEndpointImpl(214)] : The activate method has thrown an exception java.lang.NoClassDefFoundError: io.netty.buffer.ByteBufAlloca
tor (initialization failure)
...
Caused by: java.lang.OutOfMemoryError: malloc(2) failed to allocate 0 bytes

Steps to Reproduce
Several fat and zfat tests when run on z/OS with Beta enabled will get the error.

Expected behavior
No exceptions caused by malloc errors.

Diagnostic information:

• OpenLiberty Version: 26.0.0.1. Hasn't been seen on 25.xxxx levels w/ Java25. Also not seen on distr platforms.
• Affected feature(s)
• Java Version: 25.0.1
• server.xml configuration:

[epj]

From messages.log:
[12/16/25, 11:17:52:940 EST] 0000001e LogService-99-com.ibm.ws.transport.http E CWWKE0701E: bundle com.ibm.ws.transport.http:1.0.109.cl26012025121
6-0302 (99)[com.ibm.ws.http.internal.HttpEndpointImpl(214)] : The activate method has thrown an exception java.lang.NoClassDefFoundError: io.netty.buffer.ByteBufAlloca
tor (initialization failure)
at java.base/java.lang.J9VMInternals.initializationAlreadyFailed(J9VMInternals.java:139)
at io.netty.channel.DefaultChannelConfig.(DefaultChannelConfig.java:60)
at io.netty.channel.DefaultChannelConfig.(DefaultChannelConfig.java:75)
at io.netty.bootstrap.FailedChannel.(FailedChannel.java:30)
at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:336)
at io.netty.bootstrap.AbstractBootstrap.register(AbstractBootstrap.java:243)
at io.openliberty.netty.internal.tcp.TCPUtils.startHelper(TCPUtils.java:278)
at io.openliberty.netty.internal.tcp.TCPUtils.startInbound(TCPUtils.java:323)
at io.openliberty.netty.internal.impl.NettyFrameworkImpl.startInbound(NettyFrameworkImpl.java:676)
at com.ibm.ws.http.netty.NettyChain.startNettyChannel(NettyChain.java:264)
at com.ibm.ws.http.netty.NettyChain.update(NettyChain.java:215)
at com.ibm.ws.http.internal.HttpEndpointImpl$5.run(HttpEndpointImpl.java:247)
at com.ibm.ws.http.internal.HttpEndpointImpl.processHttpChainWork(HttpEndpointImpl.java:560)
at com.ibm.ws.http.internal.HttpEndpointImpl.modified(HttpEndpointImpl.java:463)
at com.ibm.ws.http.internal.HttpEndpointImpl.activate(HttpEndpointImpl.java:314)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java.base/java.lang.reflect.Method.invoke(Method.java:571)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:257)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41)
at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:701)
at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:544)
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:317)
at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:307)
at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:354)
at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:115)
at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:1002)
at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:975)
at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:920)
at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse$1.run(ServiceFactoryUse.java:229)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:87)
at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse.factoryGetService(ServiceFactoryUse.java:226)
at org.eclipse.osgi.internal.serviceregistry.ServiceFactoryUse.getService(ServiceFactoryUse.java:120)
at org.eclipse.osgi.internal.serviceregistry.ServiceConsumer$2.getService(ServiceConsumer.java:48)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.getService(ServiceRegistrationImpl.java:588)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.getService(ServiceRegistry.java:590)
at org.eclipse.osgi.internal.framework.BundleContextImpl.getService(BundleContextImpl.java:666)
at org.osgi.util.tracker.ServiceTracker.addingService(ServiceTracker.java:419)
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:947)
at org.osgi.util.tracker.ServiceTracker$Tracked.customizerAdding(ServiceTracker.java:1)
at org.osgi.util.tracker.AbstractTracked.trackAdding(AbstractTracked.java:257)
at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:230)
at org.osgi.util.tracker.ServiceTracker$Tracked.serviceChanged(ServiceTracker.java:907)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:134)
at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:988)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:151)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:999)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:933)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:146)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:321)
at org.eclipse.osgi.internal.framework.BundleContextImpl.registerService(BundleContextImpl.java:495)
at org.apache.felix.scr.impl.manager.AbstractComponentManager$3.register(AbstractComponentManager.java:929)
at org.apache.felix.scr.impl.manager.AbstractComponentManager$3.register(AbstractComponentManager.java:915)
at org.apache.felix.scr.impl.manager.RegistrationManager.changeRegistration(RegistrationManager.java:133)
at org.apache.felix.scr.impl.manager.AbstractComponentManager.registerService(AbstractComponentManager.java:984)
at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:752)
at org.apache.felix.scr.impl.manager.AbstractComponentManager.enableInternal(AbstractComponentManager.java:674)
at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:437)
at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:671)
at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:310)
at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:613)
at org.apache.felix.scr.impl.Activator.access$200(Activator.java:75)
at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:480)
at org.apache.felix.scr.impl.AbstractExtender.createExtension(AbstractExtender.java:196)
at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:169)
at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:49)
at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:490)
at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:1)
at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:233)
at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:452)
at org.eclipse.osgi.internal.framework.BundleContextImpl.dispatchEvent(BundleContextImpl.java:975)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:151)
at org.eclipse.osgi.internal.framework.EquinoxEventPublisher.publishBundleEventPrivileged(EquinoxEventPublisher.java:237)
at org.eclipse.osgi.internal.framework.EquinoxEventPublisher.publishBundleEvent(EquinoxEventPublisher.java:136)
at org.eclipse.osgi.internal.framework.EquinoxEventPublisher.publishBundleEvent(EquinoxEventPublisher.java:128)
at org.eclipse.osgi.internal.framework.EquinoxContainerAdaptor.publishModuleEvent(EquinoxContainerAdaptor.java:232)
at org.eclipse.osgi.container.Module.publishEvent(Module.java:534)
at org.eclipse.osgi.container.Module.start(Module.java:518)
at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel$2.run(ModuleContainer.java:2110)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
at java.base/java.lang.Thread.run(Thread.java:1485)
Caused by: java.lang.OutOfMemoryError: malloc(2) failed to allocate 0 bytes
at io.netty.util.internal.CleanerJava24Linker.malloc(CleanerJava24Linker.java:214)
at io.netty.util.internal.CleanerJava24Linker$CleanableDirectBufferImpl.(CleanerJava24Linker.java:232)
at io.netty.util.internal.CleanerJava24Linker$CleanableDirectBufferImpl.(CleanerJava24Linker.java:227)
at io.netty.util.internal.CleanerJava24Linker.allocate(CleanerJava24Linker.java:198)
at io.netty.util.internal.PlatformDependent.allocateDirect(PlatformDependent.java:600)
at io.netty.buffer.EmptyByteBuf.(EmptyByteBuf.java:43)
at io.netty.buffer.AbstractByteBufAllocator.(AbstractByteBufAllocator.java:82)
at io.netty.buffer.PooledByteBufAllocator.(PooledByteBufAllocator.java:274)
at io.netty.buffer.PooledByteBufAllocator.(PooledByteBufAllocator.java:231)
at io.netty.buffer.PooledByteBufAllocator.(PooledByteBufAllocator.java:220)
at io.netty.buffer.PooledByteBufAllocator.(PooledByteBufAllocator.java:206)
at io.netty.buffer.PooledByteBufAllocator.(PooledByteBufAllocator.java:188)
at io.netty.buffer.ByteBufUtil.(ByteBufUtil.java:87)
at io.netty.buffer.ByteBufAllocator.(ByteBufAllocator.java:24)
at io.netty.channel.DefaultChannelConfig.(DefaultChannelConfig.java:60)
at io.netty.channel.socket.DefaultServerSocketChannelConfig.(DefaultServerSocketChannelConfig.java:51)
at io.netty.channel.socket.nio.NioServerSocketChannel$NioServerSocketChannelConfig.(NioServerSocketChannel.java:212)
at io.netty.channel.socket.nio.NioServerSocketChannel$NioServerSocketChannelConfig.(NioServerSocketChannel.java:210)
at io.netty.channel.socket.nio.NioServerSocketChannel.(NioServerSocketChannel.java:106)
at io.netty.channel.socket.nio.NioServerSocketChannel.(NioServerSocketChannel.java:98)
at io.netty.channel.socket.nio.NioServerSocketChannel.(NioServerSocketChannel.java:81)
at io.netty.channel.socket.nio.NioServerSocketChannel.(NioServerSocketChannel.java:74)
at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483)
at io.netty.channel.ReflectiveChannelFactory.newChannel(ReflectiveChannelFactory.java:44)
at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:326)
... 78 more

[epj]

Verbosegc:
...

<heap-resize id="1462" type="expand" space="tenure" amount="2686976" count="1" timems="0.282" reason="insufficient free space following gc" timestamp="2025-12-16T11:17:52.963" />
<gc-end id="1463" type="global" contextid="1448" durationms="12.508" usertimems="10.000" systemtimems="10.000" stalltimems="9.883" timestamp="2025-12-16T11:17:52.963" activeThreads="3">
  <mem-info id="1464" free="17918696" total="49348608" percent="36">
    <mem type="nursery" free="9136272" total="20185088" percent="45">
      <mem type="allocate" free="9136272" total="14680064" percent="62" />
      <mem type="survivor" free="0" total="5505024" percent="0" />
    </mem>
    <mem type="tenure" free="8782424" total="29163520" percent="30" micro-fragmented="499649" macro-fragmented="0">
      <mem type="soa" free="7324248" total="27705344" percent="26" />
      <mem type="loa" free="1458176" total="1458176" percent="100" />
    </mem>
    <pending-finalizers system="489" default="0" reference="797" classloader="0" />
    <remembered-set count="7785" />
  </mem-info>
</gc-end>
<cycle-end id="1465" type="global" contextid="1448" timestamp="2025-12-16T11:17:52.963" />
<exclusive-end id="1466" timestamp="2025-12-16T11:17:52.963" durationms="12.665" />

[ERROR   ] CWWKE0701E: bundle com.ibm.ws.transport.http:1.0.109.cl260120251216-0302 (99)[com.ibm.ws.http.internal.HttpEndpointImpl(214)] : The activate method has thrown an exception java.lang.NoClassDefFoundError: io.netty.buffer.ByteBufAllocator (initialization failure)
	at java.base/java.lang.J9VMInternals.initializationAlreadyFailed(J9VMInternals.java:139)
	at io.netty.channel.DefaultChannelConfig.<init>(DefaultChannelConfig.java:60)
	at [internal classes]
Caused by: java.lang.OutOfMemoryError: malloc(2) failed to allocate 0 bytes
	at io.netty.util.internal.CleanerJava24Linker.malloc(CleanerJava24Linker.java:214)
	... 1 more```

[epj]

I raised issue 16054 with Netty.

[epj]

Netty has pull request for malloc error. Determining if we should patch it into 4.2.7 now, or wait and get 4.2.10 which will be out soon.

[epj]

Current thinking: wait for next Netty drop, which will be January anyway. Anyone hitting this problem in the mean time can avoid it by turning Beta off in their jvm.options or use java<25.