Skip to content

tests.md

Latest commit

 

History

History
70 lines (51 loc) · 3.53 KB

tests.md

File metadata and controls

70 lines (51 loc) · 3.53 KB
Raw

Travis CI

Every pull request runs the full suite of functional and unit tests on Travis CI. We test on latest stable Chrome and Firefox releases, as well as on Chrome Beta, Firefox Beta and Firefox ESR.

See .travis.yml for Travis configuration, scripts/setup_travis.sh for test setup, and scripts/run_travis.sh for test execution procedures.

We use ESLint to flag potential JavaScript errors and style issues. See .eslintrc.yml and .eslintignore for specifics.

Unit tests

We use QUnit for unit tests. To run them, click on the badger icon next to the URL bar to open the popup. Then in the popup, click on the gear icon (⚙) to open the options page. Your browser should navigate to an internal URL that starts with chrome-extension:// or moz-extension:// and ends with /skin/options.html. Replace /skin/options.html with /tests/index.html and hit enter. This will open the unit tests and run them.

Unit tests are located in /src/tests/tests. The unit test dependencies are in /src/tests/lib. Please add unit tests whenever possible.

Functional tests

Our functional tests are written in Python and driven by Selenium and pytest. To run them, you'll need to install chromedriver (link) for Chrome or geckodriver (link) for Firefox. You also need some python packages which can be installed by running:

$ pip install -r tests/requirements.txt

Now you should be able to run the Selenium tests! Try them out by running the code below. This should take several minutes.

$ BROWSER=chrome pytest -v

The BROWSER environment variable must be set. It must be one of:

  • BROWSER=/path/to/a/browser
  • the name of a browser executable that can be found like which $BROWSER
  • or simply BROWSER=chrome or BROWSER=firefox if you have them installed

Examples

Note that to use a debugger like pdb or ipdb you must pass the -s (--capture=no) flag to pytest.

# run qunit_test.py, with firefox, with verbose output (-v)
$ BROWSER=/usr/bin/firefox pytest -v tests/selenium/qunit_test.py

# run a specific test on a specific class in a specific module, on google-chrome-stable
$ BROWSER=google-chrome-stable pytest super_cookie_test.py::SuperCookieTest::test_should_detect_ls_of_third_party_frame

# run any tests whose name (including the module and class) matches the string cookie_test
# this is often useful as a less verbose way to run a single test
$ BROWSER=firefox pytest -k cookie_test

More pytest invocations can be found here (these are very useful).

If you are on Linux, you can also run the tests headlessly (without displaying a GUI). Install Xvfb with your system package manager, then set the ENABLE_XVFB=1 environment variable. Like this:

$ BROWSER=~/Downloads/firefox/firefox ENABLE_XVFB=1 pytest -s -v -k pbtest_org

Refer to the our Travis CI scripts for more information: scripts/setup_travis.sh and scripts/run_travis.sh.