[TCK Challenge] ChromeDevtoolsDriver fails due to hardcoded DevTools version mismatch and lack of browser configuration

[jamezp]

TCK Details

• TCK Specification
  • Jakarta Context and Dependency Injection 4.1
• Coordinates
  • jakarta.tck:cdi-tck-ee-impl:11.0.1
• TCK Version
  • 11.0.1
• Implementation
  • WildFly, Red Hat

Description

The CDI TCK implementation (specifically tests utilizing ChromeDevtoolsDriver, such as LongRunningConversationPropagatedByFacesContextTest) fails to execute in environments where the installed version of Google Chrome is newer than version 126. This test affects Jakarta EE 11 Platform Tests and Jakarta Context and Dependency Injection 4.1.

The TCK depends on selenium-devtools-v126 (via the Selenium 4.24.0 BOM). This creates a hard dependency on the Chrome DevTools Protocol (CDP) v126. However, the test runner logic does not provide a functional mechanism to configure the browser binary location, forcing the WebDriver to default to the system-installed browser (e.g., current Chrome Stable v142).

Because CDP v126 cannot communicate with Chrome v142, the test fails immediately with a DevToolsException. Extensive attempts to configure the binary path via standard Selenium system properties (webdriver.chrome.binary), WebDriverManager properties (wdm.chromeBinary), and environment variables have failed. This indicates that the TCK runner logic (combined with WebDriverManager usage and Maven forking) isolates or ignores these configurations during execution.

Likely Cause

The class org.jboss.cdi.tck.selenium.ChromeDevtoolsDriver initializes new ChromeOptions() and passes it to new ChromeDriver(options) without propagating binary configuration properties.

Additionally, the environment implicitly uses WebDriverManager 6.1.0. While WDM manages the driver version successfully, the lack of a mechanism to force the browser binary (coupled with the inability to use wdm.dockerEnabled within the existing TCK structure) results in an unavoidable version mismatch on modern systems.

Steps to Reproduce

1. Run the TCK on a system with a current stable Google Chrome installed (e.g., v142).
2. Execute the CDI TCK Runner (Platform TCK).
3. Run the test: org.jboss.cdi.tck.tests.context.conversation.LongRunningConversationPropagatedByFacesContextTest.
4. Observe the failure during driver initialization.

Log Output

[client] ... FINE [org.openqa.selenium.remote.service.DriverFinder] (main) Skipping Selenium Manager, path to chromedriver found in system property: .../chromedriver/linux64/126.0.6478.182/chromedriver
...
[client] ... FINE [org.openqa.selenium.remote.RemoteWebDriver] (Forwarding newSession on session null to remote) Exception: newSession Could not start a new session. Response code 500. Message: session not created: This version of ChromeDriver only supports Chrome version 126
Current browser version is 142.0.7444.162 with binary path /opt/google/chrome/chrome 
...
org.openqa.selenium.devtools.DevToolsException: You are using a no-op implementation of the CDP. The most likely reason for this is that Selenium was unable to find an implementation of the CDP protocol that matches your browser.

Related Issues / References

• TCK Challenge: testConversationPropagatedAjax using HTMLUnit is incompatible with modern Typescript implementations cdi-tck#434
• Revisit htmlunit usage in CDI TCK #1228
• Add Support for Selenium Tests cdi-tck#437

Proposed Solution / Justification for Challenge

The TCK is currently unportable to standard CI/local environments that maintain up-to-date browsers.

We request that this test failure be marked as a valid TCK Challenge / Exclude until one of the following fixes is applied to the TCK:

1. Upgrade Selenium: Update the Selenium dependency to a version compatible with the current Chrome Stable channel (e.g., Selenium 4.27+).
2. Enable Configuration: Refactor ChromeDevtoolsDriver to explicitly respect webdriver.chrome.binary or wdm.chromeBinary.
3. Rewrite the tests to use something other than Selenium like PlayWrite, I don't recall where this was suggested.

[jamezp]

A little OT, but I couldn't use the challenge template because it's in the wrong location. I filed jakartaee/cdi-tck#662 to fix that.

[Ladicek]

You're not saying which CDI TCK version you're running, so I'm not sure if this issue is valid or if it should be moved to https://github.com/jakartaee/platform-tck/, which hosts the CDI EE TCK now.

[manovotn]

You're not saying which CDI TCK version you're running, so I'm not sure if this issue is valid or if it should be moved to https://github.com/jakartaee/platform-tck/, which hosts the CDI EE TCK now.

I have transferred the issue.
The test in question is indeed no longer under CDI TCK.

[manovotn]

Upgrade Selenium: Update the Selenium dependency to a version compatible with the current Chrome Stable channel (e.g., Selenium 4.27+).

IIRC, this was an endless version battle. One that TCKs would likely loose given the frequency of updates and releases.

Enable Configuration: Refactor ChromeDevtoolsDriver to explicitly respect webdriver.chrome.binary or wdm.chromeBinary.

I assume you have a good idea what would be needed there, if so, would you mind sending a PR?
It would be preferable to just excluding the test as this is bound to bite us back for 5.0 anyway.

[scottmarlow]

Upgrade Selenium: Update the Selenium dependency to a version compatible with the current Chrome Stable channel (e.g., Selenium 4.27+).

IIRC, this was an endless version battle. One that TCKs would likely loose given the frequency of updates and releases.

Enable Configuration: Refactor ChromeDevtoolsDriver to explicitly respect webdriver.chrome.binary or wdm.chromeBinary.

I assume you have a good idea what would be needed there, if so, would you mind sending a PR?
It would be preferable to just excluding the test as this is bound to bite us back for 5.0 anyway.

Even https://playwright.dev/java/docs/browsers mentions downloading of the native browser that worked for the specific version of Playwright used. This doesn't sound like a test that can work very long as platforms get updated to work with new hardware/OS/Java versions and old platforms are expected to keep working.

I'm curious as to why we needed this test so looked at the the SpecAssertion's in https://github.com/jakartaee/platform-tck/blob/main/tcks/apis/cdi-ee-tck/tck/src/main/resources/tck-audit-cdi.xml to get the descriptions for the LongRunningConversationPropagatedByFacesContextTest test file:

• @SpecAssertion(section = CONVERSATION_CONTEXT_EE, id = "l")
  The long-running conversation context associated with a request that renders a JSF view is automatically propagated to any faces request (JSF form submission) that originates from that rendered page.
• @SpecAssertion(section = CONVERSATION_CONTEXT_EE, id = "m")
  The long-running conversation context associated with a request that results in a JSF redirect (via a navigation rule) is automatically propagated to the resulting non-faces request, and to any other subsequent request to the same URL. This is accomplished via use of a request parameter named |cid| containing the unique identifier of the conversation.
• @SpecAssertion(section = BUILTIN_SCOPES, id = "ba")
  The |@ConversationScoped| annotation represents the conversation scope defined in Section 6.7.4, "Conversation context lifecycle"

From a google search I did see https://github.com/LoboEvolution/LoboEvolution?tab=readme-ov-file described as a pure java browser but even if that works is it worth depending on an HTML browser for covering the above mention spec requirements?

[jamezp]

If the browser can be downloaded that's better because it's not using what is installed on the local machine running the TCK. Really, that is where the challenge lies. What I did yesterday was try to get Selenium or Webdriver to download the browser and ignore what was installed locally. I got it to download the driver, but not the browser.

However, we can likely force it from the TCK itself. We should consider upgrading the dependency too, but IIRC there are API changes we'd need to account for there.

I'm not suggesting we need to switch to PlayWrite, it was just mention somewhere and looked interesting to me. I think it was on a Slack discussion that is no longer visible given how long ago it was.

I assume you have a good idea what would be needed there, if so, would you mind sending a PR?
It would be preferable to just excluding the test as this is bound to bite us back for 5.0 anyway.

I've got a pretty good idea yes. I did a rather deep dive yesterday into Selenium and Webdriver :) This plan makes sense to me and I can look at creating a PR for this.

[jamezp]

You're not saying which CDI TCK version you're running, so I'm not sure if this issue is valid or if it should be moved to https://github.com/jakartaee/platform-tck/, which hosts the CDI EE TCK now.

Oops, you're right. The test is in the platform-tck repository. I'll update the issue to specify the CDI version.

Challenges are supposed to be filed with the spec itself regardless of where the source lives AIUI.

[Ladicek]

Challenges are supposed to be filed with the spec itself regardless of where the source lives AIUI.

Starting with CDI 4.1 / Jakarta EE 11, the CDI EE specification also lives in the Jakarta Platform specification and no longer in the CDI specification. See https://jakarta.ee/specifications/platform/11/jakarta-platform-spec-11.0#component-integration

[jamezp]

Challenges are supposed to be filed with the spec itself regardless of where the source lives AIUI.

Starting with CDI 4.1 / Jakarta EE 11, the CDI EE specification also lives in the Jakarta Platform specification and no longer in the CDI specification. See https://jakarta.ee/specifications/platform/11/jakarta-platform-spec-11.0#component-integration

Correct, but the challenge needs to be filed here with the fix PR sent to the platform-tck, see https://jakarta.ee/committees/specification/tckprocess/#_filing_a_challenge.

Challenges MUST be filed via the specification project’s issue tracker using the label challenge and include the following information: