Bug Report: CDI deployment failure for 6.2025.2

[rignicat]

Brief Summary

deployment failure because some internal string is unexpectedly null:

org.glassfish.deployment.common.DeploymentException: CDI deployment failure:Cannot invoke "String.contains(java.lang.CharSequence)" because "beanArchiveId" is null -- Cannot invoke "String.contains(java.lang.CharSequence)" because "beanArchiveId" is null at org.jboss.weld.bootstrap.BeanDeployments.getFinalId(BeanDeployments.java:39) at org.jboss.weld.bootstrap.WeldStartup.startContainer(WeldStartup.java:189) at org.jboss.weld.bootstrap.WeldBootstrap.startContainer(WeldBootstrap.java:73) at org.glassfish.weld.WeldDeployer.startWeldBootstrap(WeldDeployer.java:578) at org.glassfish.weld.WeldDeployer.processApplicationLoaded(WeldDeployer.java:533) at org.glassfish.weld.WeldDeployer.event(WeldDeployer.java:422) at org.glassfish.kernel.event.EventsImpl.send(EventsImpl.java:135) at org.glassfish.internal.data.ApplicationInfo.load(ApplicationInfo.java:344) at com.sun.enterprise.v3.server.ApplicationLifecycle.prepare(ApplicationLifecycle.java:580) at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:570) at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:556) at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:552) at java.base/java.security.AccessController.doPrivileged(AccessController.java:399) at java.base/javax.security.auth.Subject.doAs(Subject.java:376) at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:551) at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:582) at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:574) at java.base/java.security.AccessController.doPrivileged(AccessController.java:399) at java.base/javax.security.auth.Subject.doAs(Subject.java:376) at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:573) at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1497) at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1879) at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1755) at com.sun.enterprise.admin.cli.embeddable.CommandExecutorImpl.executeCommand(CommandExecutorImpl.java:183) at com.sun.enterprise.admin.cli.embeddable.CommandExecutorImpl.run(CommandExecutorImpl.java:96) at fish.payara.boot.runtime.BootCommand.execute(BootCommand.java:69) at fish.payara.boot.runtime.BootCommands.executeCommands(BootCommands.java:166) at fish.payara.boot.runtime.BootCommands.executeCommands(BootCommands.java:160) at com.sun.enterprise.v3.bootstrap.BootCommandService.doBootCommands(BootCommandService.java:81) at com.sun.enterprise.v3.bootstrap.BootCommandService.postConstruct(BootCommandService.java:91) at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:303) at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:351) at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:466) at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:281) at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:65) at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2103) at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93) at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:67) at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1213) at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1144) at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$UpOneLevel.run(CurrentTaskFuture.java:762) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840)

Looks like this might've been introduced by #7032 @lprimak

Expected Outcome

My app deploying normally, as it would on 6.2025.1

Current Outcome

Deployment failure

Reproducer

TBD, will work on creating one

Operating System

Debian Linux 12

JDK Version

OpenJDK 17

Payara Distribution

Payara Server Full Profile

[lprimak]

Do you have a reproducer?
Waiting for a reproducer...

[rignicat]

No I am not able to build a small reproducer, I tried recreating the structure of my project but I have no idea what the issue could be, so not really able to produce any results

[lprimak]

I can try running your app if that's at all possible. Is it using containers at all?

[lprimak]

Is your app a WAR or EAR? Can you post a "file tree" from the WAR file?

[rignicat]

Hi, sorry for forgetting to respond. Yeah, sharing the ear isn't an option sadly, but I can share the file tree for the ear. I'll be anonymising some stuff, is there anything specifically you're looking for?

[lprimak]

Can you try the fix in #7211 please?
I am just guessing here since there is no reproducer, but hoping for the best

You can try either building the branch / PR or get the pre-built here:
https://nexus.hope.nyc.ny.us/repository/maven-releases/fish/payara/distributions/payara/6.2025.2-1/payara-6.2025.2-1.zip

[rignicat]

sadly looks like the same issue is still present:

[#|2025-02-24T10:08:41.618+0000|SEVERE|Payara 6.2025.3|javax.enterprise.system.core|_ThreadID=44;_ThreadName=RunLevelControllerThread-1740391673967;_TimeMillis=1740391721618;_LevelValue=1000;_MessageID=NCLS-CORE-00026;| Exception during lifecycle processing org.glassfish.deployment.common.DeploymentException: CDI deployment failure:Cannot invoke "String.contains(java.lang.CharSequence)" because "beanArchiveId" is null -- Cannot invoke "String.contains(java.lang.CharSequence)" because "beanArchiveId" is null at org.jboss.weld.bootstrap.BeanDeployments.getFinalId(BeanDeployments.java:39) at org.jboss.weld.bootstrap.WeldStartup.startContainer(WeldStartup.java:189) at org.jboss.weld.bootstrap.WeldBootstrap.startContainer(WeldBootstrap.java:73) at org.glassfish.weld.WeldDeployer.startWeldBootstrap(WeldDeployer.java:577) at org.glassfish.weld.WeldDeployer.processApplicationLoaded(WeldDeployer.java:532) at org.glassfish.weld.WeldDeployer.event(WeldDeployer.java:421) at org.glassfish.kernel.event.EventsImpl.send(EventsImpl.java:135) at org.glassfish.internal.data.ApplicationInfo.load(ApplicationInfo.java:344) at com.sun.enterprise.v3.server.ApplicationLifecycle.prepare(ApplicationLifecycle.java:580) at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:570) at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:556) at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:552) at java.base/java.security.AccessController.doPrivileged(AccessController.java:399) at java.base/javax.security.auth.Subject.doAs(Subject.java:376) at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:551) at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:582) at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:574) at java.base/java.security.AccessController.doPrivileged(AccessController.java:399) at java.base/javax.security.auth.Subject.doAs(Subject.java:376) at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:573) at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1497) at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1879) at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1755) at com.sun.enterprise.admin.cli.embeddable.CommandExecutorImpl.executeCommand(CommandExecutorImpl.java:183) at com.sun.enterprise.admin.cli.embeddable.CommandExecutorImpl.run(CommandExecutorImpl.java:96) at fish.payara.boot.runtime.BootCommand.execute(BootCommand.java:69) at fish.payara.boot.runtime.BootCommands.executeCommands(BootCommands.java:166) at fish.payara.boot.runtime.BootCommands.executeCommands(BootCommands.java:160) at com.sun.enterprise.v3.bootstrap.BootCommandService.doBootCommands(BootCommandService.java:81) at com.sun.enterprise.v3.bootstrap.BootCommandService.postConstruct(BootCommandService.java:91) at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:303) at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:351) at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:466) at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:281) at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:65) at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2103) at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:93) at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:67) at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1213) at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1144) at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$UpOneLevel.run(CurrentTaskFuture.java:762) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840) |#]

I tried attaching a debugger, and saw that the DeploymentImpl constructor has a return on line 168, if libJarRootBDAs is not empty. This is the case for our project, which causes the deploymentImpl constructor to not run any code that sets the contextId. Does this info help? do you need to me to look at anything else?

[TheOnlyAl]

Hi.

I have the exact same problem with one of our deployments. My analysis of the problem matches with that of @RInverid
I also could not build a reproducer for now, but maybe will try again tomorrow.

[mkarg]

We also have massive problems deploying our EAR (containing lots of WARs) on 6.2025.2, while 6.2025.1 was pretty fine.

Edit: We also have the problem that if we succeed to deploy on 6.2025.2 (after many attempts and very long waiting, it works "eventually"), then at some point we fail to @Inject a simple POJO suddenly... (always on the same target, which is in no way different to other targets). It seems your CDI change somehow makes that WELD blind for our producer, but just for this specific injection target...?! Weird... 🤔

WELD-001408: Unsatisfied dependencies for type DeploymentPlan with qualifiers @default

[lprimak]

@RInverid Can you try this guess please?

https://nexus.hope.nyc.ny.us/repository/maven-releases/fish/payara/distributions/payara/6.2025.2-2/payara-6.2025.2-2.zip

It's a combo of #7211 and #7212

Thank you

[TheOnlyAl]

Hi. For my deployment this fix sadly does not work. I am getting the same deployment error as for #7214:

[2025-02-25T07:43:57.630+0100] [Payara 6.2025.3] [SEVERE] [NCLS-CORE-00026] [javax.enterprise.system.core] [tid: _ThreadID=174 _ThreadName=admin-thread-pool::admin-listener(2)] [timeMillis: 1740465837630] [levelValue: 1000] [[
  Exception during lifecycle processing
org.glassfish.deployment.common.DeploymentException: CDI deployment failure:null -- null
        at com.sun.enterprise.deployment.Application.getPostConstructDescriptors(Application.java:624)
        at com.sun.enterprise.deployment.BundleDescriptor.getPostConstructDescriptorByClass(BundleDescriptor.java:491)
        at com.sun.enterprise.deployment.BundleDescriptor.getInjectionInfoByClass(BundleDescriptor.java:479)
        at com.sun.enterprise.deployment.Application.getInjectionInfoByClass(Application.java:661)
        at org.glassfish.weld.services.InjectionServicesImpl.registerInjectionTarget(InjectionServicesImpl.java:248)
        at org.jboss.weld.manager.InjectionTargetFactoryImpl.postProcess(InjectionTargetFactoryImpl.java:159)
        at org.jboss.weld.manager.InjectionTargetFactoryImpl.createInjectionTarget(InjectionTargetFactoryImpl.java:94)
        at org.jboss.weld.bean.ManagedBean.<init>(ManagedBean.java:102)
        at org.jboss.weld.bean.ManagedBean.of(ManagedBean.java:82)
        at org.jboss.weld.bootstrap.AbstractBeanDeployer.createManagedBean(AbstractBeanDeployer.java:275)
        at org.jboss.weld.bootstrap.BeanDeployer.createClassBean(BeanDeployer.java:278)
        at org.jboss.weld.bootstrap.BeanDeployer.createClassBeans(BeanDeployer.java:256)
        at org.jboss.weld.bootstrap.BeanDeployment.createBeans(BeanDeployment.java:256)
        at org.jboss.weld.bootstrap.WeldStartup.deployBeans(WeldStartup.java:435)
        at org.jboss.weld.bootstrap.WeldBootstrap.deployBeans(WeldBootstrap.java:87)
        at org.glassfish.weld.WeldDeployer.startWeldBootstrap(WeldDeployer.java:586)
        at org.glassfish.weld.WeldDeployer.processApplicationLoaded(WeldDeployer.java:532)
        at org.glassfish.weld.WeldDeployer.event(WeldDeployer.java:421)
        at org.glassfish.kernel.event.EventsImpl.send(EventsImpl.java:135)
        at org.glassfish.internal.data.ApplicationInfo.load(ApplicationInfo.java:344)
        at com.sun.enterprise.v3.server.ApplicationLifecycle.prepare(ApplicationLifecycle.java:573)
        at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:570)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:556)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:552)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
        at java.base/javax.security.auth.Subject.doAs(Subject.java:376)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:551)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:582)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:574)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
        at java.base/javax.security.auth.Subject.doAs(Subject.java:376)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:573)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1497)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1879)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1755)
        at org.glassfish.admin.rest.resources.admin.CommandResource.executeCommand(CommandResource.java:409)
        at org.glassfish.admin.rest.resources.admin.CommandResource.execCommandMultInMultOut(CommandResource.java:258)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
        at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:274)
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:266)
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:253)
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:696)
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:367)
        at org.glassfish.admin.rest.adapter.JerseyContainerCommandService$3.service(JerseyContainerCommandService.java:179)
        at org.glassfish.admin.rest.adapter.RestAdapter.service(RestAdapter.java:189)
        at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
        at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
        at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:174)
        at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:153)
        at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:196)
        at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:88)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:246)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:178)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:118)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:96)
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
        at org.glassfish.grizzly.portunif.PUFilter.handleRead(PUFilter.java:193)
        at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:88)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:246)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:178)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:118)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:96)
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
        at org.glassfish.grizzly.portunif.PUFilter.handleRead(PUFilter.java:193)
        at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:88)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:246)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:178)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:118)
        at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:96)
        at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:51)
        at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:510)
        at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:82)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:83)
        at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:101)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535)
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515)
        at java.base/java.lang.Thread.run(Thread.java:840)
]]

[2025-02-25T07:43:57.634+0100] [Payara 6.2025.3] [SEVERE] [] [javax.enterprise.system.core] [tid: _ThreadID=174 _ThreadName=admin-thread-pool::admin-listener(2)] [timeMillis: 1740465837634] [levelValue: 1000] [[
  Exception while loading the app]]

[2025-02-25T07:43:57.672+0100] [Payara 6.2025.3] [SEVERE] [] [javax.enterprise.system.core] [tid: _ThreadID=174 _ThreadName=admin-thread-pool::admin-listener(2)] [timeMillis: 1740465837672] [levelValue: 1000] [[
  Exception while loading the app : CDI deployment failure:null -- null]]