[ZEPPELIN-6195] Fix Selenium tests by switching from Chrome to Firefox and upgrading outdated Spark

### What is this PR for?
 The `test-selenium-with-spark-module-for-spark-3-4` job in GitHub Actions was consistently failing with the error:
  ```bash
 from unknown command: 'Runtime.evaluate' wasn't found
 ```
 
 This issue was caused by ChromeDriver not recognizing the `Runtime.evaluate` command.
To resolve it, the browser used in the test was switched from Chrome to Firefox.

During the migration to Firefox, another issue occurred:
```bash
Unable to download from https://dlcdn.apache.org/spark/spark-3.4.2/spark-3.4.2-bin-hadoop3.tgz
java.io.FileNotFoundException: https://dlcdn.apache.org/spark/spark-3.4.2/spark-3.4.2-bin-hadoop3.tgz
```
This was due to the Spark 3.4.2 binaries no longer being available from the Apache mirror.
The test environment was therefore updated to use Spark 3.5.6, which resolved the download problem.

Summary of changes:
- Switched Selenium tests from Chrome to Firefox (non-snap) using geckodriver.
- Updated Spark version in the test environment from 3.4.2 to 3.5.6 to fix missing binary downloads.

Before: test-selenium-with-spark-module-for-spark-3-4 (Chrome, Spark 3.4.2)
After: test-selenium-with-spark-module-for-spark-3-5 (Firefox, Spark 3.5.6)

### What type of PR is it?
Bug Fix

### Todos
* [x] - Verified that other integration tests (if any) are not affected
* [x] - Confirmed long-term compatibility with the Chrome for Testing approach

### What is the Jira issue?
* Jira: https://issues.apache.org/jira/browse/ZEPPELIN-6195

### How should this be tested?
- Run the `test-selenium-with-spark-module-for-spark-3-4` job in GitHub Actions.
- Verify that `AuthenticationIT` and any other affected Selenium tests pass without encountering `Runtime.evaluate` errors.

### Screenshots (if appropriate)
N/A

### Questions:
* Does the license files need to update? No.
* Is there breaking changes for older versions? No.
* Does this needs documentation? No.


Closes #4941 from ParkGyeongTae/fix-selenium-runtime-evaluate-error.

Signed-off-by: Philipp Dallig <philipp.dallig@gmail.com>

Author: ParkGyeongTae

.github/workflows/frontend.yml

@@ -86,32 +86,34 @@ jobs:
       - name: Run headless test
         run: xvfb-run --auto-servernum --server-args="-screen 0 1024x768x24" ./mvnw package -pl zeppelin-web-angular ${MAVEN_ARGS}
 
-  test-selenium-with-spark-module-for-spark-3-4:
+  test-selenium-with-spark-module-for-spark-3-5:
     runs-on: ubuntu-24.04
     defaults:
       run:
         shell: bash -l {0}
+    env:
+      ZEPPELIN_SELENIUM_BROWSER: firefox
     steps:
       - name: Checkout
         uses: actions/checkout@v4
-      - name: Tune Runner VM
-        uses: ./.github/actions/tune-runner-vm
-      - name: Set up JDK 11
-        uses: actions/setup-java@v4
-        with:
-          distribution: 'temurin'
-          java-version: 11
-      - name: Cache local Maven repository
-        uses: actions/cache@v4
-        with:
-          path: |
-            ~/.m2/repository
-            !~/.m2/repository/org/apache/zeppelin/
-            ~/.spark-dist
-            ~/.cache
-          key: ${{ runner.os }}-zeppelin-${{ hashFiles('**/pom.xml') }}
-          restore-keys: |
-            ${{ runner.os }}-zeppelin-
+      - name: Install Firefox (non-snap) and geckodriver
+        run: |
+          sudo snap remove firefox || true
+          sudo add-apt-repository -y ppa:mozillateam/ppa
+          sudo tee -a /etc/apt/preferences.d/mozilla-firefox <<EOF
+          Package: *
+          Pin: release o=LP-PPA-mozillateam
+          Pin-Priority: 1001
+          EOF
+          sudo apt-get update
+          sudo apt-get install -y firefox
+          GECKODRIVER_VERSION=$(curl -s https://api.github.com/repos/mozilla/geckodriver/releases/latest | grep tag_name | cut -d '"' -f 4)
+          wget -q https://github.com/mozilla/geckodriver/releases/download/${GECKODRIVER_VERSION}/geckodriver-${GECKODRIVER_VERSION}-linux64.tar.gz
+          tar -xzf geckodriver-${GECKODRIVER_VERSION}-linux64.tar.gz
+          chmod +x geckodriver
+          sudo mv geckodriver /usr/local/bin/
+          geckodriver --version
+          firefox --version
       - name: Setup conda environment with python 3.9 and R
         uses: conda-incubator/setup-miniconda@v3
         with:
@@ -122,15 +124,19 @@ jobs:
           channel-priority: strict
           auto-activate-base: false
           use-mamba: true
-      - name: Make IRkernel available to Jupyter
-        run: |
-          R -e "IRkernel::installspec()"
       - name: Install Environment
         run: |
-          ./mvnw clean install -DskipTests -am -pl zeppelin-integration -Pweb-classic -Pintegration -Pspark-scala-2.12 -Pspark-3.4 -Pweb-dist ${MAVEN_ARGS}
-      - name: run tests
+          ./mvnw clean install -DskipTests -am -pl zeppelin-integration -Pweb-classic -Pintegration -Pspark-scala-2.12 -Pspark-3.5 -Pweb-dist ${MAVEN_ARGS}
+      - name: Print browser version
         run: |
-          xvfb-run --auto-servernum --server-args="-screen 0 1600x1024x16" ./mvnw verify -DfailIfNoTests=false -pl zeppelin-integration -Pweb-classic -Pintegration -Pspark-scala-2.12 -Pspark-3.4 -Pweb-dist -Pusing-source-tree ${MAVEN_ARGS}
-      - name: Print zeppelin logs
-        if: always()
-        run: if [ -d "logs" ]; then cat logs/*; fi
+          echo "Firefox version:"
+          firefox --version
+          echo "GeckoDriver version:"
+          geckodriver --version
+      - name: Run integration tests with Firefox
+        run: |
+          xvfb-run --auto-servernum --server-args="-screen 0 1600x1024x16" \
+            ./mvnw verify -DfailIfNoTests=false \
+            -pl zeppelin-integration \
+            -Pweb-classic -Pintegration -Pspark-scala-2.12 -Pspark-3.5 -Pweb-dist -Pusing-source-tree \
+            ${MAVEN_ARGS}

zeppelin-integration/pom.xml

@@ -38,7 +38,7 @@
 
   <properties>
     <!--test library versions-->
-    <selenium.java.version>4.11.0</selenium.java.version>
+    <selenium.java.version>4.20.0</selenium.java.version>
   </properties>
 
   <dependencies>

zeppelin-integration/src/test/java/org/apache/zeppelin/WebDriverManager.java

@@ -139,7 +139,7 @@ private WebDriver constructWebDriver(int port) {
     while (System.currentTimeMillis() - start < 60 * 1000) {
       // wait for page load
       try {
-        (new WebDriverWait(driver, Duration.ofSeconds(30))).until(new ExpectedCondition<Boolean>() {
+        (new WebDriverWait(driver, Duration.ofSeconds(60))).until(new ExpectedCondition<Boolean>() {
           @Override
           public Boolean apply(WebDriver d) {
             return d.findElement(By.xpath("//i[@uib-tooltip='WebSocket Connected']"))
@@ -156,7 +156,12 @@ public Boolean apply(WebDriver d) {
 
     assertTrue(loaded);
 
-    driver.manage().window().maximize();
+    try {
+      driver.manage().window().maximize();
+    } catch (Exception e) {
+      LOG.warn("Failed to maximize browser window. Consider using setSize() instead.", e);
+    }
+
     return driver;
   }
 

zeppelin-test/src/main/java/org/apache/zeppelin/test/DownloadUtils.java

@@ -65,7 +65,7 @@ public class DownloadUtils {
   private static final int PROGRESS_BAR_UPDATE_INTERVAL;
 
   private static String downloadFolder = System.getProperty("user.home") + "/.cache";
-  public static final String DEFAULT_SPARK_VERSION = "3.4.2";
+  public static final String DEFAULT_SPARK_VERSION = "3.5.6";
   public static final String DEFAULT_SPARK_HADOOP_VERSION = "3";