[py] Refactored `conftest.py` in a more object oriented design approach #15495

sandeepsuryaprasad · 2025-03-23T15:46:13Z

User description

Thanks for contributing to Selenium!
A PR well described will help maintainers to quickly review and merge it

Before submitting your PR, please check our contributing guidelines.
Avoid large PRs, help reviewers by making them as simple and short as possible.

Refactored conftest.py in a more object oriented design approach

Motivation and Context

Refactored conftest.py in a more object oriented design approach so that the code is more readable and easy to make changes in the future.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist

I have read the contributing document.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.
All new and existing tests passed.

PR Type

Enhancement

Description

Refactored conftest.py to an object-oriented design for better readability.
Introduced a Driver class to encapsulate driver-related logic and properties.
Simplified driver initialization and teardown using the Driver class.
Removed redundant functions and streamlined driver configuration handling.

Changes walkthrough 📝

Relevant files

Enhancement

conftest.py Refactored `conftest.py` with a `Driver` class py/conftest.py Introduced a `Driver` class to encapsulate driver logic. Simplified driver initialization and teardown processes. Removed redundant helper functions for driver options and services. Improved code readability and maintainability with object-oriented design.	+111/-111

Need help?
Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
Check out the documentation for more information.

qodo-merge-pro · 2025-03-23T15:46:32Z

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review Incorrect Property Usage The driver property doesn't return the driver instance if it's already initialized. It only returns a value when _driver is None, which could lead to unexpected behavior. @property def driver(self): if not self._driver: return self._initialize_driver() Platform Reference Error The code references 'driver_instance.exe_platform' but this attribute doesn't exist. It should be using 'selenium_driver.exe_platform' instead. if driver_class == "Safari" and driver_instance.exe_platform != "Darwin": pytest.skip("Safari tests can only run on an Apple OS") if driver_class == "Ie" and driver_instance.exe_platform != "Windows": pytest.skip("IE and EdgeHTML Tests can only run on Windows") if "WebKit" in driver_class and driver_instance.exe_platform != "Linux": pytest.skip("Webkit tests can only run on Linux") Redundant Fixture The 'stop_driver' fixture at line 236 appears to be empty and redundant since driver cleanup is now handled by the Driver class's stop_driver method. @pytest.fixture(scope="session", autouse=True) def stop_driver(request): def fin():

qodo-merge-pro · 2025-03-23T15:46:58Z

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	✅ ~~Fix platform detection~~ Suggestion Impact: The commit directly implemented the suggestion by changing self._platform = platform to self._platform = platform.system(), fixing the platform detection bug. This change is also reflected in the platform comparison checks later in the code. code diff: - self._platform = platform + self._platform = platform.system() The property is storing the `platform` module instead of the platform name. It should call `platform.system()` to get the actual platform name. py/conftest.py [113-116] @property def exe_platform(self): - self._platform = platform + self._platform = platform.system() return self._platform `[Suggestion has been applied]` Suggestion importance[1-10]: 10 __ Why: This fixes a critical bug where the platform module object is being stored instead of the actual platform name (string). This would cause platform-specific checks to fail, as seen in lines 212-217 where platform comparisons are made.	High
	✅ ~~Return existing driver instance~~ Suggestion Impact: The suggestion was directly implemented in the commit. Line 37 adds 'return self._driver' to the driver property method, exactly as suggested, ensuring the property returns the existing driver instance when it's already initialized. code diff: if not self._driver: return self._initialize_driver() + return self._driver The `driver` property doesn't return the driver instance when it's already initialized. It should return `self._driver` if it exists, otherwise initialize and return it. py/conftest.py [184-187] @property def driver(self): if not self._driver: return self._initialize_driver() + return self._driver `[Suggestion has been applied]` Suggestion importance[1-10]: 9 __ Why: This is a critical bug fix. The current implementation only returns a value when initializing a new driver but doesn't return anything when the driver already exists, which would cause the property to return None in subsequent calls, breaking driver functionality.	High
	✅ ~~Return None for missing executable~~ Suggestion Impact: The suggestion was directly implemented in the commit. Line 31 of the diff shows the exact addition of 'return None' to the service property method when executable is falsy, exactly as suggested. code diff: @@ -180,15 +179,19 @@ module = getattr(webdriver, self._driver_class.lower()) self._service = module.service.Service(executable_path=executable) return self._service + return None The `service` property doesn't return anything when `executable` is falsy, which could cause unexpected behavior. It should return `None` explicitly in this case. py/conftest.py [176-182] @property def service(self): executable = self.driver_path if executable: module = getattr(webdriver, self._driver_class.lower()) self._service = module.service.Service(executable_path=executable) return self._service + return None `[Suggestion has been applied]` Suggestion importance[1-10]: 7 __ Why: This improves code clarity and prevents potential issues by explicitly returning None when no executable is provided. Without this fix, the property would return undefined, which could lead to unexpected behavior when the service property is accessed.	Medium
Update

cgoldberg

@sandeepsuryaprasad thanks for the PR! Overall this looks really nice.

I pulled your branch and ran some tests, and noticed that the browser now closes and re-opens between every test. Previously, it was re-using the same browser instance between tests.

For example, if you run pytest py/test/selenium/webdriver/common/typing_tests.py you will notice this. The same tests running against trunk behave different. This makes the tests take significantly longer to run. Can you look into that and make it so it behaves like it used to?

Also, the AI bot found some minor issues. Could you address these?:
#15495 (comment)

cgoldberg · 2025-03-23T17:15:09Z

There are some TypeError failures in CI that seem to be related to this PR also:

https://github.com/SeleniumHQ/selenium/actions/runs/14020548932/job/39252777029?pr=15495#annotation:18:66830

sandeepsuryaprasad · 2025-03-24T00:47:41Z

@cgoldberg sure.. I figured out what went wrong.. I will fix it and push the code once agin.

cgoldberg · 2025-03-24T00:50:19Z

@sandeepsuryaprasad great, I'll give it another look once you update.

cgoldberg · 2025-03-24T16:58:20Z

@sandeepsuryaprasad I tried again after your latest changes and there are still issues.

the browser still closes/re-opens between tests
something is wrong with driver_class?

If I try to run tests with the argument --driver=safari (I'm on Linux), the tests should be skipped. This isn't happening, and all tests attempt to run and fail.

Similarly, if I run tests with the argument --driver=webkitgtk, all tests fail with messages like this:

request = <SubRequest 'driver' for <Function test_should_type_an_integer[webkitgtk]>>

    @pytest.fixture(scope="function")
    def driver(request):
        global driver_instance
        driver_class = getattr(request, "param", "Chrome").capitalize()
        selenium_driver = Driver(driver_class, request)
>       driver_instance = selenium_driver.driver

py/conftest.py:212: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
py/conftest.py:187: in driver
    return self._initialize_driver()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <conftest.Driver object at 0x7fb251afafd0>

    def _initialize_driver(self):
        if self.options:
            self.kwargs["options"] = self.options
        if self.driver_path:
            self.kwargs["service"] = self.service
>       self._driver = getattr(webdriver, self._driver_class)(**self.kwargs)
E       AttributeError: module 'selenium.webdriver' has no attribute 'Webkitgtk'

py/conftest.py:195: AttributeError

I haven't looked into what is causing these, but they need to be fixed before we can merge this.

sandeepsuryaprasad · 2025-03-24T17:33:48Z

@cgoldberg I am working on it..running tests on my local machine..

sandeepsuryaprasad · 2025-03-24T17:57:14Z

@cgoldberg can you try it now.. except remote tests all other tests are passing when I run CI on my machine. Regarding closing and re-opening of the browser for every test, the scope of the fixture driver is at function level @pytest.fixture(scope="function") and i have not changed the scope of this fixture in my latest change.. so the setup and teardown is happening for each test method. Do you suspect any other thing going wrong here?..

cgoldberg · 2025-03-24T18:08:05Z

@sandeepsuryaprasad The issues I mentioned in my last comment are still there... I don't see any changes that fix them.

As far as the browser closing/re-opening... I haven't investigated the cause, but it didn't do that before (run the tests against trunk to see). We don't want to introduce that change, so it needs to function the same way as it did previously.

sandeepsuryaprasad · 2025-03-25T16:28:41Z

@cgoldberg I have fixed driver_class issues. Now the tests are getting skipped if there is a platform and browser mismatch. Below are the results when executed on Mac. Can you please re-validate once again.

============================= test session starts ==============================
platform darwin -- Python 3.11.1, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/sandeep/Documents/selenium/py
configfile: pyproject.toml
plugins: instafail-0.4.2, html-3.2.0, mock-3.10.0, trio-0.8.0, xdist-3.3.1, metadata-2.0.4
collected 30 items

py/test/selenium/webdriver/common/typing_tests.py::test_should_fire_key_press_events[webkitgtk] SKIPPED [  3%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_fire_key_down_events[webkitgtk] SKIPPED [  6%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_fire_key_up_events[webkitgtk] SKIPPED [ 10%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_type_lower_case_letters[webkitgtk] SKIPPED [ 13%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_be_able_to_type_capital_letters[webkitgtk] SKIPPED [ 16%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_be_able_to_type_quote_marks[webkitgtk] SKIPPED [ 20%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_be_able_to_type_the_at_character[webkitgtk] SKIPPED [ 23%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_be_able_to_mix_upper_and_lower_case_letters[webkitgtk] SKIPPED [ 26%]
py/test/selenium/webdriver/common/typing_tests.py::test_arrow_keys_should_not_be_printable[webkitgtk] SKIPPED [ 30%]
py/test/selenium/webdriver/common/typing_tests.py::test_list_of_arrow_keys_should_not_be_printable[webkitgtk] SKIPPED [ 33%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_be_able_to_use_arrow_keys[webkitgtk] SKIPPED [ 36%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_up_when_entering_text_into_input_elements[webkitgtk] SKIPPED [ 40%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_down_when_entering_text_into_input_elements[webkitgtk] SKIPPED [ 43%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_press_when_entering_text_into_input_elements[webkitgtk] SKIPPED [ 46%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_up_when_entering_text_into_text_areas[webkitgtk] SKIPPED [ 50%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_down_when_entering_text_into_text_areas[webkitgtk] SKIPPED [ 53%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_press_when_entering_text_into_text_areas[webkitgtk] SKIPPED [ 56%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_report_key_code_of_arrow_keys_up_down_events[webkitgtk] SKIPPED [ 60%]
py/test/selenium/webdriver/common/typing_tests.py::test_numeric_non_shift_keys[webkitgtk] SKIPPED [ 63%]
py/test/selenium/webdriver/common/typing_tests.py::test_numeric_shift_keys[webkitgtk] SKIPPED [ 66%]
py/test/selenium/webdriver/common/typing_tests.py::test_lower_case_alpha_keys[webkitgtk] SKIPPED [ 70%]
py/test/selenium/webdriver/common/typing_tests.py::test_uppercase_alpha_keys[webkitgtk] SKIPPED [ 73%]
py/test/selenium/webdriver/common/typing_tests.py::test_all_printable_keys[webkitgtk] SKIPPED [ 76%]
py/test/selenium/webdriver/common/typing_tests.py::test_arrow_keys_and_page_up_and_down[webkitgtk] SKIPPED [ 80%]
py/test/selenium/webdriver/common/typing_tests.py::test_delete_and_backspace_keys[webkitgtk] SKIPPED [ 83%]
py/test/selenium/webdriver/common/typing_tests.py::test_special_space_keys[webkitgtk] SKIPPED [ 86%]
py/test/selenium/webdriver/common/typing_tests.py::test_numberpad_and_function_keys[webkitgtk] SKIPPED [ 90%]
py/test/selenium/webdriver/common/typing_tests.py::test_shift_selection_deletes[webkitgtk] SKIPPED [ 93%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_type_into_input_elements_that_have_no_type_attribute[webkitgtk] SKIPPED [ 96%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_type_an_integer[webkitgtk] SKIPPED [100%]

============================= 30 skipped in 0.11s ==============================

============================= test session starts ==============================
platform darwin -- Python 3.11.1, pytest-7.4.4, pluggy-1.5.0
rootdir: /Users/sandeep/Documents/selenium/py
configfile: pyproject.toml
plugins: instafail-0.4.2, html-3.2.0, mock-3.10.0, trio-0.8.0, xdist-3.3.1, metadata-2.0.4
collected 30 items

py/test/selenium/webdriver/common/typing_tests.py::test_should_fire_key_press_events[ie] SKIPPED [  3%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_fire_key_down_events[ie] SKIPPED [  6%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_fire_key_up_events[ie] SKIPPED [ 10%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_type_lower_case_letters[ie] SKIPPED [ 13%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_be_able_to_type_capital_letters[ie] SKIPPED [ 16%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_be_able_to_type_quote_marks[ie] SKIPPED [ 20%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_be_able_to_type_the_at_character[ie] SKIPPED [ 23%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_be_able_to_mix_upper_and_lower_case_letters[ie] SKIPPED [ 26%]
py/test/selenium/webdriver/common/typing_tests.py::test_arrow_keys_should_not_be_printable[ie] SKIPPED [ 30%]
py/test/selenium/webdriver/common/typing_tests.py::test_list_of_arrow_keys_should_not_be_printable[ie] SKIPPED [ 33%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_be_able_to_use_arrow_keys[ie] SKIPPED [ 36%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_up_when_entering_text_into_input_elements[ie] SKIPPED [ 40%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_down_when_entering_text_into_input_elements[ie] SKIPPED [ 43%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_press_when_entering_text_into_input_elements[ie] SKIPPED [ 46%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_up_when_entering_text_into_text_areas[ie] SKIPPED [ 50%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_down_when_entering_text_into_text_areas[ie] SKIPPED [ 53%]
py/test/selenium/webdriver/common/typing_tests.py::test_will_simulate_akey_press_when_entering_text_into_text_areas[ie] SKIPPED [ 56%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_report_key_code_of_arrow_keys_up_down_events[ie] SKIPPED [ 60%]
py/test/selenium/webdriver/common/typing_tests.py::test_numeric_non_shift_keys[ie] SKIPPED [ 63%]
py/test/selenium/webdriver/common/typing_tests.py::test_numeric_shift_keys[ie] SKIPPED [ 66%]
py/test/selenium/webdriver/common/typing_tests.py::test_lower_case_alpha_keys[ie] SKIPPED [ 70%]
py/test/selenium/webdriver/common/typing_tests.py::test_uppercase_alpha_keys[ie] SKIPPED [ 73%]
py/test/selenium/webdriver/common/typing_tests.py::test_all_printable_keys[ie] SKIPPED [ 76%]
py/test/selenium/webdriver/common/typing_tests.py::test_arrow_keys_and_page_up_and_down[ie] SKIPPED [ 80%]
py/test/selenium/webdriver/common/typing_tests.py::test_delete_and_backspace_keys[ie] SKIPPED [ 83%]
py/test/selenium/webdriver/common/typing_tests.py::test_special_space_keys[ie] SKIPPED [ 86%]
py/test/selenium/webdriver/common/typing_tests.py::test_numberpad_and_function_keys[ie] SKIPPED [ 90%]
py/test/selenium/webdriver/common/typing_tests.py::test_shift_selection_deletes[ie] SKIPPED [ 93%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_type_into_input_elements_that_have_no_type_attribute[ie] SKIPPED [ 96%]
py/test/selenium/webdriver/common/typing_tests.py::test_should_type_an_integer[ie] SKIPPED [100%]

============================= 30 skipped in 0.12s ==============================

sandeepsuryaprasad · 2025-03-25T17:00:07Z

@cgoldberg working on browser closing/re-opening issue.

sandeepsuryaprasad · 2025-03-26T18:27:39Z

@cgoldberg Looks like there is some bug in the below code that I have taken from contest.py from trunk. The teardown code which is defined in function fin and request.addfinalizer(fin) gets executed only if
marker = request.node.get_closest_marker(f"xfail_{driver_class.lower()}") condition evaluates to truthy.

But the variable marker never gets evaluated to boolean True for the test that have been marked as expected failure. Below are the test that are marked as expected failure in typing_tests.py which are expected to fail on safari

test_will_simulate_akey_up_when_entering_text_into_input_elements
test_will_simulate_akey_down_when_entering_text_into_input_elements
test_will_simulate_akey_press_when_entering_text_into_input_elements
test_will_simulate_akey_up_when_entering_text_into_text_areas
test_will_simulate_akey_down_when_entering_text_into_text_areas
test_will_simulate_akey_press_when_entering_text_into_text_areas

If you carefully observe the execution, except for the above tests, for all other tests a new instance of chrome opens (setup part) and closes (teardown part, request.addfinalizer(fin) gets executed). But for the above mentioned tests, teardown code never gets executed.. the reason is if marker is not None: is never truthy. because marker will be None so request.addfinalizer(fin) never gets executed and typing happens on the same browser that was opened by the test test_will_simulate_akey_up_when_entering_text_into_input_elements Below is the current code in trunk/conftest.py which I suspect is buggy.

# conditionally mark tests as expected to fail based on driver
marker = request.node.get_closest_marker(f"xfail_{driver_class.lower()}")

    if marker is not None:
        if "run" in marker.kwargs:
            if marker.kwargs["run"] is False:
                pytest.skip()
                yield
                return
        if "raises" in marker.kwargs:
            marker.kwargs.pop("raises")
        pytest.xfail(**marker.kwargs)

        def fin():
            global driver_instance
            if driver_instance is not None:
                driver_instance.quit()
            driver_instance = None

        request.addfinalizer(fin)

But in the PR that I have raised, the teardown part request.addfinalizer(selenium_driver.stop_driver) is outside the condition if marker is not None: So the teardown code gets executed irrespective of weather if marker is not None: evaluates to truthy or falsy. And this should be the expected behaviour. Because all of the 6 tests that are mentioned above are independant tests and for each of the tests, the fixture driver is passed and the scope of the fixture is function. So the right behaviour is setup and teardown must be executed irrespective of what happens to if marker is not None: condition. Below is the code in my PR.

# conditionally mark tests as expected to fail based on driver
marker = request.node.get_closest_marker(f"xfail_{driver_class.lower()}")

if marker is not None:
        if "run" in marker.kwargs:
            if marker.kwargs["run"] is False:
                pytest.skip()
                yield
                return
        if "raises" in marker.kwargs:
            marker.kwargs.pop("raises")
        pytest.xfail(**marker.kwargs)

request.addfinalizer(selenium_driver.stop_driver)

Please let me know your thoughts on this. Thanks!

sandeepsuryaprasad · 2025-03-30T18:17:55Z

@cgoldberg I have fixed all the review comments. After the code change in this PR, the behaviour is exactly same as it was on trunk. Can you please re-trigger the CI. I have executed all tests on my local and it's passing.

Fixed browser closing/reopening issue
The tests will be skipped if ran on browser with incompatible platform.

cgoldberg · 2025-03-30T18:20:47Z

I just triggered CI again.

sandeepsuryaprasad · 2025-03-31T10:43:57Z

@cgoldberg there were some logical issues with newly developed Driver class. I have completely re-structured the entire class now. I ran below browser tests and all are passing without any issues on my local.

bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:common-chrome-bidi
bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:common-firefox-bidi
bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:test-chrome
bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:test-firefox
bazel test --jobs 1 //py:test-remote

Could you please validate once and re-trigger the CI. Hopefully this time everything should be okay. It was little bit trickier than I thought it would be!

cgoldberg · 2025-03-31T14:08:01Z

I just triggered CI.

sandeepsuryaprasad · 2025-03-31T14:39:45Z

@cgoldberg looks like the tests are passing except some flakey ones
//py:common-chrome-bidi-test/selenium/webdriver/common/api_example_tests.py FAILED in 3 out of 3 in 4.4s
//py:common-edge-bidi-test/selenium/webdriver/common/w3c_interaction_tests.py FAILED in 3 out of 3 in 10.4s
//py:test-remote-test/selenium/webdriver/common/quit_tests.py FAILED in 3 out of 3 in 6.0s

sandeepsuryaprasad · 2025-03-31T15:06:20Z

executed //py:common-chrome-bidi-test/selenium/webdriver/common/api_example_tests.py separately once again..

~$ bazel test //py:common-chrome-bidi-test/selenium/webdriver/common/api_example_tests.py
INFO: Analyzed target //py:common-chrome-bidi-test/selenium/webdriver/common/api_example_tests.py (0 packages loaded, 5 targets configured).
INFO: Found 1 test target...
Target //py:common-chrome-bidi-test/selenium/webdriver/common/api_example_tests.py up-to-date:
  bazel-bin/py/common-chrome-bidi-test/selenium/webdriver/common/api_example_tests.py
  bazel-bin/py/common-chrome-bidi-test/selenium/webdriver/common/api_example_tests.py-runner.py
INFO: Elapsed time: 2.470s, Critical Path: 0.31s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
//py:common-chrome-bidi-test/selenium/webdriver/common/api_example_tests.py (cached) PASSED in 35.5s

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

cgoldberg · 2025-04-01T01:04:17Z

@sandeepsuryaprasad
I pulled your branch and ran the tests that failed in CI, and they passed locally.

However, I found one issue with the tests for WebKitGTK and WPEWebKit... I don't think we run these in CI (not sure why).

for example, if I try to run any test using the --driver=webkitgtk arg, they will fail with AttributeError: module 'selenium.webdriver' has no attribute 'Webkitgtk'. Did you mean: 'webkitgtk'?.

Note the Capitalization... It is trying to instantiate Webkitgtk() rather than the correct class name WebKitGTK().

This is caused by the following line inside your driver function:
driver_class = getattr(request, "param", "Chrome").capitalize()

We can't make the assumption that the class name is always the capitalized request.param. This is not correct in the case of WebKitGTK and WPEWebKit.

You could fix it by replacing that line with:

driver_class = getattr(request, "param", "Chrome").capitalize()
if driver_class == "Webkitgtk":
    driver_class = "WebKitGTK"
elif driver_class == "Wpewebkit":
    driver_class = "WPEWebKit"

... but perhaps there is a nicer solution?

(BTW, I can't get the --driver=wpewebkit tests to run locally even on trunk, but that is unrelated to your change and I will investigate that separately)

sandeepsuryaprasad · 2025-04-01T15:21:50Z

@cgoldberg I am working on this issue to provide an elegant solution..

sandeepsuryaprasad · 2025-04-01T18:26:38Z

@cgoldberg I have modified the code, introduced one more level of abstraction for handling driver classes and options classes. Please let me know your feedback. I ran all the tests on my local and it's passing.

shbenzer · 2025-05-01T10:38:59Z

No, it isn’t. Please run tests locally using:

bazel test //py:<bazel_tests>

sandeepsuryaprasad · 2025-05-01T12:28:41Z

@shbenzer do you want me to run all browser tests as well as remote tests on my local machine.

shbenzer · 2025-05-01T13:17:35Z

@shbenzer do you want me to run all browser tests as well as remote tests on my local machine.

Considering you are editing conftest.py, that’s what I would do.

sandeepsuryaprasad · 2025-05-01T14:53:06Z

@cgoldberg @shbenzer @titusfortner I ran all the tests on my local machine... below are the results

Unit Tests

~$ bazel test //py:unit
INFO: Analyzed 19 targets (0 packages loaded, 0 targets configured).
INFO: Found 19 test targets...
INFO: Elapsed time: 0.765s, Critical Path: 0.07s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
//py:unit-test/unit/selenium/webdriver/chrome/chrome_options_tests.py (cached) PASSED in 2.5s
//py:unit-test/unit/selenium/webdriver/common/cdp_module_fallback_tests.py (cached) PASSED in 2.4s
//py:unit-test/unit/selenium/webdriver/common/common_options_tests.py (cached) PASSED in 2.4s
//py:unit-test/unit/selenium/webdriver/common/fedcm/account_tests.py (cached) PASSED in 2.3s
//py:unit-test/unit/selenium/webdriver/common/fedcm/dialog_tests.py (cached) PASSED in 2.0s
//py:unit-test/unit/selenium/webdriver/common/print_page_options_tests.py (cached) PASSED in 2.9s
//py:unit-test/unit/selenium/webdriver/edge/edge_options_tests.py (cached) PASSED in 3.8s
//py:unit-test/unit/selenium/webdriver/firefox/firefox_options_tests.py (cached) PASSED in 1.3s
//py:unit-test/unit/selenium/webdriver/remote/error_handler_tests.py (cached) PASSED in 3.0s
//py:unit-test/unit/selenium/webdriver/remote/new_session_tests.py (cached) PASSED in 2.0s
//py:unit-test/unit/selenium/webdriver/remote/remote_connection_tests.py (cached) PASSED in 3.0s
//py:unit-test/unit/selenium/webdriver/remote/remote_server_tests.py (cached) PASSED in 4.0s
//py:unit-test/unit/selenium/webdriver/remote/subtyping_tests.py (cached) PASSED in 2.9s
//py:unit-test/unit/selenium/webdriver/safari/safari_options_tests.py (cached) PASSED in 3.2s
//py:unit-test/unit/selenium/webdriver/support/color_tests.py   (cached) PASSED in 2.4s
//py:unit-test/unit/selenium/webdriver/virtual_authenticator/credentials_tests.py (cached) PASSED in 2.6s
//py:unit-test/unit/selenium/webdriver/virtual_authenticator/virtual_authenticator_options_tests.py (cached) PASSED in 2.2s
//py:unit-test/unit/selenium/webdriver/webkitgtk/webkitgtk_options_tests.py (cached) PASSED in 1.7s
//py:unit-test/unit/selenium/webdriver/wpewebkit/wpewebkit_options_tests.py (cached) PASSED in 2.7s

Executed 0 out of 19 tests: 19 tests pass.

Chrome Tests

bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:common-chrome-bidi

INFO: Found 57 test targets...
INFO: Elapsed time: 870.454s, Critical Path: 266.09s
INFO: 286 processes: 170 internal, 116 local.
INFO: Build completed successfully, 286 total actions
//py:common-chrome-bidi-test/selenium/webdriver/common/alerts_tests.py   PASSED in 20.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/api_example_tests.py PASSED in 30.5s
//py:common-chrome-bidi-test/selenium/webdriver/common/bidi_browser_tests.py PASSED in 9.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/bidi_browsing_context_tests.py PASSED in 39.3s
//py:common-chrome-bidi-test/selenium/webdriver/common/bidi_network_tests.py PASSED in 7.7s
//py:common-chrome-bidi-test/selenium/webdriver/common/bidi_script_tests.py PASSED in 11.8s
//py:common-chrome-bidi-test/selenium/webdriver/common/bidi_session_tests.py PASSED in 3.9s
//py:common-chrome-bidi-test/selenium/webdriver/common/bidi_tests.py     PASSED in 6.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/children_finding_tests.py PASSED in 19.6s
//py:common-chrome-bidi-test/selenium/webdriver/common/clear_tests.py    PASSED in 5.5s
//py:common-chrome-bidi-test/selenium/webdriver/common/click_scrolling_tests.py PASSED in 13.7s
//py:common-chrome-bidi-test/selenium/webdriver/common/click_tests.py    PASSED in 3.2s
//py:common-chrome-bidi-test/selenium/webdriver/common/cookie_tests.py   PASSED in 12.3s
//py:common-chrome-bidi-test/selenium/webdriver/common/correct_event_firing_tests.py PASSED in 12.6s
//py:common-chrome-bidi-test/selenium/webdriver/common/devtools_tests.py PASSED in 2.7s
//py:common-chrome-bidi-test/selenium/webdriver/common/driver_element_finding_tests.py PASSED in 60.4s
//py:common-chrome-bidi-test/selenium/webdriver/common/driver_finder_tests.py PASSED in 1.3s
//py:common-chrome-bidi-test/selenium/webdriver/common/element_aria_label_tests.py PASSED in 2.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/element_aria_tests.py PASSED in 3.9s
//py:common-chrome-bidi-test/selenium/webdriver/common/element_attribute_tests.py PASSED in 24.0s
//py:common-chrome-bidi-test/selenium/webdriver/common/element_equality_tests.py PASSED in 4.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/executing_async_javascript_tests.py PASSED in 30.3s
//py:common-chrome-bidi-test/selenium/webdriver/common/executing_javascript_tests.py PASSED in 27.2s
//py:common-chrome-bidi-test/selenium/webdriver/common/fedcm_tests.py    PASSED in 27.9s
//py:common-chrome-bidi-test/selenium/webdriver/common/form_handling_tests.py PASSED in 22.8s
//py:common-chrome-bidi-test/selenium/webdriver/common/frame_switching_tests.py PASSED in 37.4s
//py:common-chrome-bidi-test/selenium/webdriver/common/implicit_waits_tests.py PASSED in 9.2s
//py:common-chrome-bidi-test/selenium/webdriver/common/interactions_with_device_tests.py PASSED in 24.6s
//py:common-chrome-bidi-test/selenium/webdriver/common/opacity_tests.py  PASSED in 3.0s
//py:common-chrome-bidi-test/selenium/webdriver/common/page_load_timeout_tests.py PASSED in 3.0s
//py:common-chrome-bidi-test/selenium/webdriver/common/page_loading_tests.py PASSED in 2.2s
//py:common-chrome-bidi-test/selenium/webdriver/common/position_and_size_tests.py PASSED in 8.8s
//py:common-chrome-bidi-test/selenium/webdriver/common/proxy_tests.py    PASSED in 1.4s
//py:common-chrome-bidi-test/selenium/webdriver/common/quit_tests.py     PASSED in 2.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/rendered_webelement_tests.py PASSED in 4.6s
//py:common-chrome-bidi-test/selenium/webdriver/common/repr_tests.py     PASSED in 3.6s
//py:common-chrome-bidi-test/selenium/webdriver/common/script_pinning_tests.py PASSED in 6.7s
//py:common-chrome-bidi-test/selenium/webdriver/common/select_class_tests.py PASSED in 30.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/select_element_handling_tests.py PASSED in 7.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/selenium_manager_tests.py PASSED in 1.2s
//py:common-chrome-bidi-test/selenium/webdriver/common/stale_reference_tests.py PASSED in 3.7s
//py:common-chrome-bidi-test/selenium/webdriver/common/takes_screenshots_tests.py PASSED in 4.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/text_handling_tests.py PASSED in 20.0s
//py:common-chrome-bidi-test/selenium/webdriver/common/timeout_tests.py  PASSED in 2.9s
//py:common-chrome-bidi-test/selenium/webdriver/common/typing_tests.py   PASSED in 27.9s
//py:common-chrome-bidi-test/selenium/webdriver/common/upload_tests.py   PASSED in 7.4s
//py:common-chrome-bidi-test/selenium/webdriver/common/virtual_authenticator_tests.py PASSED in 8.7s
//py:common-chrome-bidi-test/selenium/webdriver/common/visibility_tests.py PASSED in 13.8s
//py:common-chrome-bidi-test/selenium/webdriver/common/w3c_interaction_tests.py PASSED in 24.7s
//py:common-chrome-bidi-test/selenium/webdriver/common/web_components_tests.py PASSED in 5.9s
//py:common-chrome-bidi-test/selenium/webdriver/common/webdriverwait_tests.py PASSED in 72.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/window_switching_tests.py PASSED in 12.2s
//py:common-chrome-bidi-test/selenium/webdriver/common/window_tests.py   PASSED in 7.3s
//py:common-chrome-bidi-test/selenium/webdriver/support/event_firing_webdriver_tests.py PASSED in 12.2s
//py:common-chrome-bidi-test/selenium/webdriver/support/expected_conditions_tests.py PASSED in 12.3s
//py:common-chrome-bidi-test/selenium/webdriver/support/relative_by_tests.py PASSED in 24.1s
//py:common-chrome-bidi-test/selenium/webdriver/common/interactions_tests.py FLAKY, failed in 1 out of 2 in 31.0s
  Stats over 2 runs: max = 31.0s, min = 30.9s, avg = 30.9s, dev = 0.1s
  /private/var/tmp/_bazel_sandeepsuryaprasad/917761202eec54f1a5b4e28b8b80214b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/py/common-chrome-bidi-test/selenium/webdriver/common/interactions_tests.py/test_attempts/attempt_1.log

Executed 57 out of 57 tests: 57 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

Firefox Tests

bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:firefox-chrome-bidi
INFO: Found 57 test targets...
INFO: Elapsed time: 1427.739s, Critical Path: 552.19s
INFO: 286 processes: 170 internal, 116 local.
INFO: Build completed successfully, 286 total actions
//py:common-firefox-bidi-test/selenium/webdriver/common/alerts_tests.py  PASSED in 43.5s
//py:common-firefox-bidi-test/selenium/webdriver/common/api_example_tests.py PASSED in 53.6s
//py:common-firefox-bidi-test/selenium/webdriver/common/bidi_browser_tests.py PASSED in 14.4s
//py:common-firefox-bidi-test/selenium/webdriver/common/bidi_browsing_context_tests.py PASSED in 68.8s
//py:common-firefox-bidi-test/selenium/webdriver/common/bidi_script_tests.py PASSED in 15.4s
//py:common-firefox-bidi-test/selenium/webdriver/common/bidi_session_tests.py PASSED in 5.4s
//py:common-firefox-bidi-test/selenium/webdriver/common/bidi_tests.py    PASSED in 1.4s
//py:common-firefox-bidi-test/selenium/webdriver/common/children_finding_tests.py PASSED in 39.7s
//py:common-firefox-bidi-test/selenium/webdriver/common/clear_tests.py   PASSED in 14.0s
//py:common-firefox-bidi-test/selenium/webdriver/common/click_scrolling_tests.py PASSED in 21.6s
//py:common-firefox-bidi-test/selenium/webdriver/common/click_tests.py   PASSED in 5.0s
//py:common-firefox-bidi-test/selenium/webdriver/common/cookie_tests.py  PASSED in 26.2s
//py:common-firefox-bidi-test/selenium/webdriver/common/correct_event_firing_tests.py PASSED in 26.0s
//py:common-firefox-bidi-test/selenium/webdriver/common/devtools_tests.py PASSED in 1.3s
//py:common-firefox-bidi-test/selenium/webdriver/common/driver_element_finding_tests.py PASSED in 148.5s
//py:common-firefox-bidi-test/selenium/webdriver/common/driver_finder_tests.py PASSED in 1.6s
//py:common-firefox-bidi-test/selenium/webdriver/common/element_aria_label_tests.py PASSED in 1.3s
//py:common-firefox-bidi-test/selenium/webdriver/common/element_aria_tests.py PASSED in 1.3s
//py:common-firefox-bidi-test/selenium/webdriver/common/element_attribute_tests.py PASSED in 47.4s
//py:common-firefox-bidi-test/selenium/webdriver/common/element_equality_tests.py PASSED in 6.6s
//py:common-firefox-bidi-test/selenium/webdriver/common/executing_async_javascript_tests.py PASSED in 44.3s
//py:common-firefox-bidi-test/selenium/webdriver/common/executing_javascript_tests.py PASSED in 52.9s
//py:common-firefox-bidi-test/selenium/webdriver/common/fedcm_tests.py   PASSED in 1.3s
//py:common-firefox-bidi-test/selenium/webdriver/common/form_handling_tests.py PASSED in 47.7s
//py:common-firefox-bidi-test/selenium/webdriver/common/frame_switching_tests.py PASSED in 69.6s
//py:common-firefox-bidi-test/selenium/webdriver/common/implicit_waits_tests.py PASSED in 14.9s
//py:common-firefox-bidi-test/selenium/webdriver/common/interactions_tests.py PASSED in 37.0s
//py:common-firefox-bidi-test/selenium/webdriver/common/interactions_with_device_tests.py PASSED in 26.4s
//py:common-firefox-bidi-test/selenium/webdriver/common/opacity_tests.py PASSED in 5.3s
//py:common-firefox-bidi-test/selenium/webdriver/common/page_load_timeout_tests.py PASSED in 6.0s
//py:common-firefox-bidi-test/selenium/webdriver/common/page_loading_tests.py PASSED in 2.9s
//py:common-firefox-bidi-test/selenium/webdriver/common/position_and_size_tests.py PASSED in 16.9s
//py:common-firefox-bidi-test/selenium/webdriver/common/proxy_tests.py   PASSED in 1.2s
//py:common-firefox-bidi-test/selenium/webdriver/common/quit_tests.py    PASSED in 2.9s
//py:common-firefox-bidi-test/selenium/webdriver/common/rendered_webelement_tests.py PASSED in 8.1s
//py:common-firefox-bidi-test/selenium/webdriver/common/repr_tests.py    PASSED in 6.2s
//py:common-firefox-bidi-test/selenium/webdriver/common/script_pinning_tests.py PASSED in 12.0s
//py:common-firefox-bidi-test/selenium/webdriver/common/select_class_tests.py PASSED in 74.6s
//py:common-firefox-bidi-test/selenium/webdriver/common/select_element_handling_tests.py PASSED in 15.5s
//py:common-firefox-bidi-test/selenium/webdriver/common/selenium_manager_tests.py PASSED in 1.3s
//py:common-firefox-bidi-test/selenium/webdriver/common/stale_reference_tests.py PASSED in 6.8s
//py:common-firefox-bidi-test/selenium/webdriver/common/takes_screenshots_tests.py PASSED in 4.9s
//py:common-firefox-bidi-test/selenium/webdriver/common/text_handling_tests.py PASSED in 40.7s
//py:common-firefox-bidi-test/selenium/webdriver/common/timeout_tests.py PASSED in 4.5s
//py:common-firefox-bidi-test/selenium/webdriver/common/typing_tests.py  PASSED in 44.9s
//py:common-firefox-bidi-test/selenium/webdriver/common/upload_tests.py  PASSED in 5.4s
//py:common-firefox-bidi-test/selenium/webdriver/common/virtual_authenticator_tests.py PASSED in 1.3s
//py:common-firefox-bidi-test/selenium/webdriver/common/visibility_tests.py PASSED in 28.8s
//py:common-firefox-bidi-test/selenium/webdriver/common/w3c_interaction_tests.py PASSED in 20.6s
//py:common-firefox-bidi-test/selenium/webdriver/common/web_components_tests.py PASSED in 6.6s
//py:common-firefox-bidi-test/selenium/webdriver/common/webdriverwait_tests.py PASSED in 103.6s
//py:common-firefox-bidi-test/selenium/webdriver/common/window_switching_tests.py PASSED in 23.2s
//py:common-firefox-bidi-test/selenium/webdriver/common/window_tests.py  PASSED in 11.2s
//py:common-firefox-bidi-test/selenium/webdriver/support/event_firing_webdriver_tests.py PASSED in 23.0s
//py:common-firefox-bidi-test/selenium/webdriver/support/expected_conditions_tests.py PASSED in 21.5s
//py:common-firefox-bidi-test/selenium/webdriver/support/relative_by_tests.py PASSED in 45.9s
//py:common-firefox-bidi-test/selenium/webdriver/common/bidi_network_tests.py FLAKY, failed in 1 out of 2 in 23.1s
  Stats over 2 runs: max = 23.1s, min = 18.7s, avg = 20.9s, dev = 2.2s
  /private/var/tmp/_bazel_sandeepsuryaprasad/917761202eec54f1a5b4e28b8b80214b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/py/common-firefox-bidi-test/selenium/webdriver/common/bidi_network_tests.py/test_attempts/attempt_1.log

Executed 57 out of 57 tests: 57 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

Edge Tests

bazel test --local_test_jobs 1 --flaky_test_attempts 3 //py:common-edge-bidi

INFO: Found 57 test targets...
INFO: Elapsed time: 894.347s, Critical Path: 242.28s
INFO: 286 processes: 170 internal, 116 local.
INFO: Build completed successfully, 286 total actions
//py:common-edge-bidi-test/selenium/webdriver/common/alerts_tests.py     PASSED in 18.5s
//py:common-edge-bidi-test/selenium/webdriver/common/api_example_tests.py PASSED in 25.4s
//py:common-edge-bidi-test/selenium/webdriver/common/bidi_browser_tests.py PASSED in 8.5s
//py:common-edge-bidi-test/selenium/webdriver/common/bidi_browsing_context_tests.py PASSED in 38.6s
//py:common-edge-bidi-test/selenium/webdriver/common/bidi_network_tests.py PASSED in 7.2s
//py:common-edge-bidi-test/selenium/webdriver/common/bidi_script_tests.py PASSED in 11.6s
//py:common-edge-bidi-test/selenium/webdriver/common/bidi_session_tests.py PASSED in 3.5s
//py:common-edge-bidi-test/selenium/webdriver/common/bidi_tests.py       PASSED in 5.9s
//py:common-edge-bidi-test/selenium/webdriver/common/children_finding_tests.py PASSED in 18.1s
//py:common-edge-bidi-test/selenium/webdriver/common/clear_tests.py      PASSED in 5.2s
//py:common-edge-bidi-test/selenium/webdriver/common/click_scrolling_tests.py PASSED in 14.4s
//py:common-edge-bidi-test/selenium/webdriver/common/click_tests.py      PASSED in 3.0s
//py:common-edge-bidi-test/selenium/webdriver/common/cookie_tests.py     PASSED in 11.1s
//py:common-edge-bidi-test/selenium/webdriver/common/correct_event_firing_tests.py PASSED in 12.4s
//py:common-edge-bidi-test/selenium/webdriver/common/devtools_tests.py   PASSED in 2.8s
//py:common-edge-bidi-test/selenium/webdriver/common/driver_element_finding_tests.py PASSED in 65.4s
//py:common-edge-bidi-test/selenium/webdriver/common/driver_finder_tests.py PASSED in 1.2s
//py:common-edge-bidi-test/selenium/webdriver/common/element_aria_label_tests.py PASSED in 2.1s
//py:common-edge-bidi-test/selenium/webdriver/common/element_aria_tests.py PASSED in 3.4s
//py:common-edge-bidi-test/selenium/webdriver/common/element_attribute_tests.py PASSED in 23.6s
//py:common-edge-bidi-test/selenium/webdriver/common/element_equality_tests.py PASSED in 3.6s
//py:common-edge-bidi-test/selenium/webdriver/common/executing_async_javascript_tests.py PASSED in 29.9s
//py:common-edge-bidi-test/selenium/webdriver/common/executing_javascript_tests.py PASSED in 24.8s
//py:common-edge-bidi-test/selenium/webdriver/common/fedcm_tests.py      PASSED in 26.5s
//py:common-edge-bidi-test/selenium/webdriver/common/form_handling_tests.py PASSED in 20.2s
//py:common-edge-bidi-test/selenium/webdriver/common/frame_switching_tests.py PASSED in 34.1s
//py:common-edge-bidi-test/selenium/webdriver/common/implicit_waits_tests.py PASSED in 9.0s
//py:common-edge-bidi-test/selenium/webdriver/common/interactions_tests.py PASSED in 28.3s
//py:common-edge-bidi-test/selenium/webdriver/common/interactions_with_device_tests.py PASSED in 25.2s
//py:common-edge-bidi-test/selenium/webdriver/common/opacity_tests.py    PASSED in 2.8s
//py:common-edge-bidi-test/selenium/webdriver/common/page_loading_tests.py PASSED in 2.0s
//py:common-edge-bidi-test/selenium/webdriver/common/position_and_size_tests.py PASSED in 8.3s
//py:common-edge-bidi-test/selenium/webdriver/common/proxy_tests.py      PASSED in 1.2s
//py:common-edge-bidi-test/selenium/webdriver/common/quit_tests.py       PASSED in 1.9s
//py:common-edge-bidi-test/selenium/webdriver/common/rendered_webelement_tests.py PASSED in 4.5s
//py:common-edge-bidi-test/selenium/webdriver/common/repr_tests.py       PASSED in 3.6s
//py:common-edge-bidi-test/selenium/webdriver/common/script_pinning_tests.py PASSED in 5.7s
//py:common-edge-bidi-test/selenium/webdriver/common/select_class_tests.py PASSED in 27.7s
//py:common-edge-bidi-test/selenium/webdriver/common/select_element_handling_tests.py PASSED in 6.9s
//py:common-edge-bidi-test/selenium/webdriver/common/selenium_manager_tests.py PASSED in 1.3s
//py:common-edge-bidi-test/selenium/webdriver/common/stale_reference_tests.py PASSED in 3.7s
//py:common-edge-bidi-test/selenium/webdriver/common/takes_screenshots_tests.py PASSED in 3.8s
//py:common-edge-bidi-test/selenium/webdriver/common/text_handling_tests.py PASSED in 18.6s
//py:common-edge-bidi-test/selenium/webdriver/common/timeout_tests.py    PASSED in 2.8s
//py:common-edge-bidi-test/selenium/webdriver/common/typing_tests.py     PASSED in 26.4s
//py:common-edge-bidi-test/selenium/webdriver/common/upload_tests.py     PASSED in 104.1s
//py:common-edge-bidi-test/selenium/webdriver/common/virtual_authenticator_tests.py PASSED in 8.4s
//py:common-edge-bidi-test/selenium/webdriver/common/visibility_tests.py PASSED in 13.0s
//py:common-edge-bidi-test/selenium/webdriver/common/w3c_interaction_tests.py PASSED in 19.5s
//py:common-edge-bidi-test/selenium/webdriver/common/web_components_tests.py PASSED in 5.2s
//py:common-edge-bidi-test/selenium/webdriver/common/webdriverwait_tests.py PASSED in 69.6s
//py:common-edge-bidi-test/selenium/webdriver/common/window_switching_tests.py PASSED in 11.4s
//py:common-edge-bidi-test/selenium/webdriver/common/window_tests.py     PASSED in 5.4s
//py:common-edge-bidi-test/selenium/webdriver/support/event_firing_webdriver_tests.py PASSED in 11.1s
//py:common-edge-bidi-test/selenium/webdriver/support/expected_conditions_tests.py PASSED in 11.6s
//py:common-edge-bidi-test/selenium/webdriver/support/relative_by_tests.py PASSED in 21.9s
//py:common-edge-bidi-test/selenium/webdriver/common/page_load_timeout_tests.py FLAKY, failed in 1 out of 2 in 3.1s
  Stats over 2 runs: max = 3.1s, min = 2.2s, avg = 2.7s, dev = 0.4s
  /private/var/tmp/_bazel_sandeepsuryaprasad/917761202eec54f1a5b4e28b8b80214b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/py/common-edge-bidi-test/selenium/webdriver/common/page_load_timeout_tests.py/test_attempts/attempt_1.log

Executed 57 out of 57 tests: 57 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

Remote Tests

INFO: Found 59 test targets...
INFO: Elapsed time: 102.602s, Critical Path: 101.63s
INFO: 3 processes: 12 local.
INFO: Build completed, 2 tests FAILED, 3 total actions
//py:test-remote-test/selenium/webdriver/common/alerts_tests.py (cached) PASSED in 39.3s
//py:test-remote-test/selenium/webdriver/common/api_example_tests.py (cached) PASSED in 29.0s
//py:test-remote-test/selenium/webdriver/common/children_finding_tests.py (cached) PASSED in 33.6s
//py:test-remote-test/selenium/webdriver/common/clear_tests.py  (cached) PASSED in 20.4s
//py:test-remote-test/selenium/webdriver/common/click_scrolling_tests.py (cached) PASSED in 15.7s
//py:test-remote-test/selenium/webdriver/common/click_tests.py  (cached) PASSED in 27.9s
//py:test-remote-test/selenium/webdriver/common/cookie_tests.py (cached) PASSED in 26.6s
//py:test-remote-test/selenium/webdriver/common/correct_event_firing_tests.py (cached) PASSED in 35.7s
//py:test-remote-test/selenium/webdriver/common/devtools_tests.py (cached) PASSED in 6.6s
//py:test-remote-test/selenium/webdriver/common/driver_element_finding_tests.py (cached) PASSED in 20.9s
//py:test-remote-test/selenium/webdriver/common/driver_finder_tests.py (cached) PASSED in 11.1s
//py:test-remote-test/selenium/webdriver/common/element_aria_label_tests.py (cached) PASSED in 3.9s
//py:test-remote-test/selenium/webdriver/common/element_aria_tests.py (cached) PASSED in 3.6s
//py:test-remote-test/selenium/webdriver/common/element_attribute_tests.py (cached) PASSED in 20.1s
//py:test-remote-test/selenium/webdriver/common/element_equality_tests.py (cached) PASSED in 21.0s
//py:test-remote-test/selenium/webdriver/common/executing_async_javascript_tests.py (cached) PASSED in 33.1s
//py:test-remote-test/selenium/webdriver/common/executing_javascript_tests.py (cached) PASSED in 24.1s
//py:test-remote-test/selenium/webdriver/common/fedcm_tests.py  (cached) PASSED in 3.7s
//py:test-remote-test/selenium/webdriver/common/form_handling_tests.py (cached) PASSED in 33.2s
//py:test-remote-test/selenium/webdriver/common/frame_switching_tests.py (cached) PASSED in 20.9s
//py:test-remote-test/selenium/webdriver/common/implicit_waits_tests.py (cached) PASSED in 25.6s
//py:test-remote-test/selenium/webdriver/common/opacity_tests.py (cached) PASSED in 49.6s
//py:test-remote-test/selenium/webdriver/common/page_load_timeout_tests.py (cached) PASSED in 15.7s
//py:test-remote-test/selenium/webdriver/common/page_loading_tests.py (cached) PASSED in 28.1s
//py:test-remote-test/selenium/webdriver/common/position_and_size_tests.py (cached) PASSED in 17.3s
//py:test-remote-test/selenium/webdriver/common/print_pdf_tests.py (cached) PASSED in 28.6s
//py:test-remote-test/selenium/webdriver/common/proxy_tests.py  (cached) PASSED in 11.0s
//py:test-remote-test/selenium/webdriver/common/quit_tests.py   (cached) PASSED in 23.2s
//py:test-remote-test/selenium/webdriver/common/rendered_webelement_tests.py (cached) PASSED in 27.5s
//py:test-remote-test/selenium/webdriver/common/repr_tests.py   (cached) PASSED in 16.3s
//py:test-remote-test/selenium/webdriver/common/script_pinning_tests.py (cached) PASSED in 28.2s
//py:test-remote-test/selenium/webdriver/common/select_class_tests.py (cached) PASSED in 48.0s
//py:test-remote-test/selenium/webdriver/common/select_element_handling_tests.py (cached) PASSED in 38.8s
//py:test-remote-test/selenium/webdriver/common/selenium_manager_tests.py (cached) PASSED in 3.5s
//py:test-remote-test/selenium/webdriver/common/stale_reference_tests.py (cached) PASSED in 16.4s
//py:test-remote-test/selenium/webdriver/common/takes_screenshots_tests.py (cached) PASSED in 20.2s
//py:test-remote-test/selenium/webdriver/common/text_handling_tests.py (cached) PASSED in 32.2s
//py:test-remote-test/selenium/webdriver/common/timeout_tests.py (cached) PASSED in 16.3s
//py:test-remote-test/selenium/webdriver/common/typing_tests.py (cached) PASSED in 39.0s
//py:test-remote-test/selenium/webdriver/common/upload_tests.py (cached) PASSED in 16.3s
//py:test-remote-test/selenium/webdriver/common/virtual_authenticator_tests.py (cached) PASSED in 3.6s
//py:test-remote-test/selenium/webdriver/common/visibility_tests.py (cached) PASSED in 22.7s
//py:test-remote-test/selenium/webdriver/common/w3c_interaction_tests.py (cached) PASSED in 15.2s
//py:test-remote-test/selenium/webdriver/common/web_components_tests.py (cached) PASSED in 3.9s
//py:test-remote-test/selenium/webdriver/common/webdriverwait_tests.py (cached) PASSED in 52.8s
//py:test-remote-test/selenium/webdriver/common/window_switching_tests.py (cached) PASSED in 18.3s
//py:test-remote-test/selenium/webdriver/common/window_tests.py (cached) PASSED in 25.0s
//py:test-remote-test/selenium/webdriver/remote/remote_connection_tests.py (cached) PASSED in 29.1s
//py:test-remote-test/selenium/webdriver/remote/remote_custom_element_tests.py (cached) PASSED in 14.4s
//py:test-remote-test/selenium/webdriver/remote/remote_custom_locator_tests.py (cached) PASSED in 27.9s
//py:test-remote-test/selenium/webdriver/remote/remote_downloads_tests.py (cached) PASSED in 26.4s
//py:test-remote-test/selenium/webdriver/remote/remote_firefox_profile_tests.py (cached) PASSED in 11.0s
//py:test-remote-test/selenium/webdriver/remote/remote_server_tests.py (cached) PASSED in 12.0s
//py:test-remote-test/selenium/webdriver/remote/remote_tests.py (cached) PASSED in 2.9s
//py:test-remote-test/selenium/webdriver/support/event_firing_webdriver_tests.py (cached) PASSED in 16.2s
//py:test-remote-test/selenium/webdriver/support/expected_conditions_tests.py (cached) PASSED in 22.0s
//py:test-remote-test/selenium/webdriver/support/relative_by_tests.py (cached) PASSED in 29.2s
//py:test-remote-test/selenium/webdriver/common/interactions_tests.py    FAILED in 3 out of 3 in 26.6s
  Stats over 3 runs: max = 26.6s, min = 16.2s, avg = 19.7s, dev = 4.9s
  /private/var/tmp/_bazel_sandeepsuryaprasad/917761202eec54f1a5b4e28b8b80214b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/py/test-remote-test/selenium/webdriver/common/interactions_tests.py/test.log
  /private/var/tmp/_bazel_sandeepsuryaprasad/917761202eec54f1a5b4e28b8b80214b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/py/test-remote-test/selenium/webdriver/common/interactions_tests.py/test_attempts/attempt_1.log
  /private/var/tmp/_bazel_sandeepsuryaprasad/917761202eec54f1a5b4e28b8b80214b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/py/test-remote-test/selenium/webdriver/common/interactions_tests.py/test_attempts/attempt_2.log
//py:test-remote-test/selenium/webdriver/common/interactions_with_device_tests.py FAILED in 3 out of 3 in 14.4s
  Stats over 3 runs: max = 14.4s, min = 13.9s, avg = 14.1s, dev = 0.2s
  /private/var/tmp/_bazel_sandeepsuryaprasad/917761202eec54f1a5b4e28b8b80214b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/py/test-remote-test/selenium/webdriver/common/interactions_with_device_tests.py/test.log
  /private/var/tmp/_bazel_sandeepsuryaprasad/917761202eec54f1a5b4e28b8b80214b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/py/test-remote-test/selenium/webdriver/common/interactions_with_device_tests.py/test_attempts/attempt_1.log
  /private/var/tmp/_bazel_sandeepsuryaprasad/917761202eec54f1a5b4e28b8b80214b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/py/test-remote-test/selenium/webdriver/common/interactions_with_device_tests.py/test_attempts/attempt_2.log

Executed 2 out of 59 tests: 57 tests pass and 2 fail locally.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

Two remote tests fails with following exception
//py:test-remote-test/selenium/webdriver/common/interactions_tests.py

driver = <selenium.webdriver.remote.webdriver.WebDriver (session="787949de-a4ae-4857-a290-0dec06b94643")>
pages = <conftest.pages.<locals>.Pages object at 0x10739d9d0>

    @pytest.mark.xfail_firefox
    @pytest.mark.xfail_safari
    def test_selecting_multiple_items(driver, pages):
        """Copied from org.openqa.selenium.interactions.CombinedInputActionsTest."""
        pages.load("selectableItems.html")
        reportingElement = driver.find_element(By.ID, "infodiv")
        assert "no info" == reportingElement.text
    
        listItems = driver.find_elements(By.TAG_NAME, "li")
        selectThreeItems = (
            ActionChains(driver)
            .key_down(Keys.CONTROL)
            .click(listItems[1])
            .click(listItems[3])
            .click(listItems[5])
            .key_up(Keys.CONTROL)
        )
        selectThreeItems.perform()
    
>       assert "#item2 #item4 #item6" == reportingElement.text
E       AssertionError: assert '#item2 #item4 #item6' == 'no info'
E         - no info
E         + #item2 #item4 #item6

py/test/selenium/webdriver/common/interactions_tests.py:182: AssertionError

sandeepsuryaprasad · 2025-05-02T15:54:13Z

@cgoldberg @shbenzer @titusfortner I ran all Bazel tests both on trunk as well as on my branch and the results are same. The flakey //py:test-remote-test/selenium/webdriver/common/interactions_tests.py test fails on both branches.

titusfortner · 2025-05-02T18:17:39Z

All of the Python tests on GitHub are failing for:

selenium.common.exceptions.SessionNotCreatedException: Message: Expected browser binary location, but unable to find binary in default location, no 'moz:firefoxOptions.binary' capability provided, and no binary flag set on the command line

The capabilities being sent to start the driver in this code are missing the location to the browser, which on RBE is the pinned browser. Take a look at how the code in trunk is passing the value and make sure you haven't changed the logic with this code.

sandeepsuryaprasad · 2025-05-03T14:56:44Z

@cgoldberg , @shbenzer, @titusfortner thanks for helping me in finding out the issue. While raising this PR, I realised that I had completely missed the below part of the code, which was there on trunk but not on my branch. I have fixed it in my latest commit.

    if browser_path or browser_args:
        if not options:
            options = getattr(webdriver, f"{driver_class}Options")()
        if driver_class == "WebKitGTK":
            options.overlay_scrollbars_enabled = False
        if browser_path is not None:
            options.binary_location = browser_path.strip("'")
        if browser_args is not None:
            for arg in browser_args.split():
                options.add_argument(arg)

Also, I have made sure that the logic on trunk and logic on my branch is exactly same apart from structural changes. I have run all bazel tests and CI from my local (it passes). Can you please re-trigger the CI (especially RBE tests).

cgoldberg · 2025-05-03T14:58:48Z

It's running now

sandeepsuryaprasad · 2025-05-03T15:41:28Z

It's running now

Thanks!! Appreciate your quick response and support.

sandeepsuryaprasad · 2025-05-03T15:48:32Z

@cgoldberg , @shbenzer , @titusfortner finally all RBE tests are passing except one JS test.

//javascript/selenium-webdriver:test-firefox-contextSwitching-test.js-firefox FAILED in 2 out of 2 in 1.4s
  Stats over 2 runs: max = 1.4s, min = 1.0s, avg = 1.2s, dev = 0.2s
  /home/runner/.bazel/execroot/_main/bazel-out/k8-fastbuild/testlogs/javascript/selenium-webdriver/test-firefox-contextSwitching-test.js-firefox/test.log
  /home/runner/.bazel/execroot/_main/bazel-out/k8-fastbuild/testlogs/javascript/selenium-webdriver/test-firefox-contextSwitching-test.js-firefox/test_attempts/attempt_1.log

Executed 345 out of 2292 tests: 2291 tests pass and 1 fails remotely.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
(15:18:26) INFO: Streaming build results to: https://gypsum.cluster.engflow.com/invocation/7ebf92d9-7d5b-4a4c-b52e-4d69141923ca

qodo-merge-pro bot added the Review effort 3/5 label Mar 23, 2025

sandeepsuryaprasad changed the title ~~Refactored conftest.py in a more object oriented design approach~~ [py] Refactored conftest.py in a more object oriented design approach Mar 23, 2025

titusfortner requested a review from cgoldberg March 23, 2025 15:54

cgoldberg requested changes Mar 23, 2025

View reviewed changes

sandeepsuryaprasad force-pushed the refactor branch 2 times, most recently from 2c5d02f to 1e518ac Compare March 30, 2025 17:18

sandeepsuryaprasad force-pushed the refactor branch 3 times, most recently from 14fc4f6 to 3caaf8c Compare March 31, 2025 10:32

sandeepsuryaprasad added 12 commits May 3, 2025 09:08

refactored conftest.py

e045110

fixed driver_class bugs

b0b9e22

fixing teardown

00eca8e

fixed all review comments

5c4143a

fixed bugs

b521b6f

fixed bidi_network_tests

88f548f

added property to check for valid platform

a3124fc

fixted linting issue

7794eae

fixed lynting issue

abd640b

fixed linting issues

600c56e

fixed linting issues

b642575

fixed merge conflicts and cleaned up

87f5a6f

sandeepsuryaprasad force-pushed the refactor branch from c810e48 to b85d14b Compare May 3, 2025 07:37

fixed browser args bugs

2f66da6

sandeepsuryaprasad force-pushed the refactor branch from b85d14b to 2f66da6 Compare May 3, 2025 09:10

sandeepsuryaprasad added 4 commits May 3, 2025 15:10

fixed linting issue

95027a2

testing

96e8b41

testing

b7faad5

testing

c770c15

Merge branch 'trunk' into refactor

e004f07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[py] Refactored `conftest.py` in a more object oriented design approach #15495

[py] Refactored `conftest.py` in a more object oriented design approach #15495

sandeepsuryaprasad commented Mar 23, 2025 •

edited by qodo-merge-pro bot

Loading

qodo-merge-pro bot commented Mar 23, 2025

qodo-merge-pro bot commented Mar 23, 2025 •

edited

Loading

cgoldberg left a comment

cgoldberg commented Mar 23, 2025

sandeepsuryaprasad commented Mar 24, 2025

cgoldberg commented Mar 24, 2025

cgoldberg commented Mar 24, 2025 •

edited

Loading

sandeepsuryaprasad commented Mar 24, 2025

sandeepsuryaprasad commented Mar 24, 2025

cgoldberg commented Mar 24, 2025

sandeepsuryaprasad commented Mar 25, 2025

sandeepsuryaprasad commented Mar 25, 2025

sandeepsuryaprasad commented Mar 26, 2025

sandeepsuryaprasad commented Mar 30, 2025

cgoldberg commented Mar 30, 2025

sandeepsuryaprasad commented Mar 31, 2025 •

edited

Loading

cgoldberg commented Mar 31, 2025

sandeepsuryaprasad commented Mar 31, 2025

sandeepsuryaprasad commented Mar 31, 2025

cgoldberg commented Apr 1, 2025

sandeepsuryaprasad commented Apr 1, 2025

sandeepsuryaprasad commented Apr 1, 2025

shbenzer commented May 1, 2025 •

edited

Loading

sandeepsuryaprasad commented May 1, 2025 •

edited

Loading

shbenzer commented May 1, 2025

sandeepsuryaprasad commented May 1, 2025 •

edited

Loading

sandeepsuryaprasad commented May 2, 2025

titusfortner commented May 2, 2025

sandeepsuryaprasad commented May 3, 2025

cgoldberg commented May 3, 2025

sandeepsuryaprasad commented May 3, 2025

sandeepsuryaprasad commented May 3, 2025

[py] Refactored conftest.py in a more object oriented design approach #15495

Are you sure you want to change the base?

[py] Refactored conftest.py in a more object oriented design approach #15495

Conversation

sandeepsuryaprasad commented Mar 23, 2025 • edited by qodo-merge-pro bot Loading

User description

Motivation and Context

Types of changes

Checklist

PR Type

Description

Changes walkthrough 📝

qodo-merge-pro bot commented Mar 23, 2025

PR Reviewer Guide 🔍

qodo-merge-pro bot commented Mar 23, 2025 • edited Loading

PR Code Suggestions ✨

cgoldberg left a comment

Choose a reason for hiding this comment

cgoldberg commented Mar 23, 2025

sandeepsuryaprasad commented Mar 24, 2025

cgoldberg commented Mar 24, 2025

cgoldberg commented Mar 24, 2025 • edited Loading

sandeepsuryaprasad commented Mar 24, 2025

sandeepsuryaprasad commented Mar 24, 2025

cgoldberg commented Mar 24, 2025

sandeepsuryaprasad commented Mar 25, 2025

sandeepsuryaprasad commented Mar 25, 2025

sandeepsuryaprasad commented Mar 26, 2025

sandeepsuryaprasad commented Mar 30, 2025

cgoldberg commented Mar 30, 2025

sandeepsuryaprasad commented Mar 31, 2025 • edited Loading

cgoldberg commented Mar 31, 2025

sandeepsuryaprasad commented Mar 31, 2025

sandeepsuryaprasad commented Mar 31, 2025

cgoldberg commented Apr 1, 2025

sandeepsuryaprasad commented Apr 1, 2025

sandeepsuryaprasad commented Apr 1, 2025

shbenzer commented May 1, 2025 • edited Loading

sandeepsuryaprasad commented May 1, 2025 • edited Loading

shbenzer commented May 1, 2025

sandeepsuryaprasad commented May 1, 2025 • edited Loading

sandeepsuryaprasad commented May 2, 2025

titusfortner commented May 2, 2025

sandeepsuryaprasad commented May 3, 2025

cgoldberg commented May 3, 2025

sandeepsuryaprasad commented May 3, 2025

sandeepsuryaprasad commented May 3, 2025

[py] Refactored `conftest.py` in a more object oriented design approach #15495

[py] Refactored `conftest.py` in a more object oriented design approach #15495

sandeepsuryaprasad commented Mar 23, 2025 •

edited by qodo-merge-pro bot

Loading

qodo-merge-pro bot commented Mar 23, 2025 •

edited

Loading

cgoldberg commented Mar 24, 2025 •

edited

Loading

sandeepsuryaprasad commented Mar 31, 2025 •

edited

Loading

shbenzer commented May 1, 2025 •

edited

Loading

sandeepsuryaprasad commented May 1, 2025 •

edited

Loading

sandeepsuryaprasad commented May 1, 2025 •

edited

Loading