Debugging GitLab CI tests

As the project moves to Drupal's infrastructure the code it moving to GitLab and tests will start using GitLab CI for running code checks and PHPUnit tests.

Setup

We're using Drupal's GitLab CI templates for running tests on Drupal modules, profiles and themes and suggest using the following configuration for the .gitlab-ci.yml file:

################
# GitLabCI template for Drupal projects.
################

include:
  - project: $_GITLAB_TEMPLATES_REPO
    ref: $_GITLAB_TEMPLATES_REF
    file:
      - '/includes/include.drupalci.main.yml'
      - '/includes/include.drupalci.variables.yml'
      - '/includes/include.drupalci.workflows.yml'
#
################
# Pipeline configuration variables are defined with default values and descriptions in the file
# https://git.drupalcode.org/project/gitlab_templates/-/blob/main/includes/include.drupalci.variables.yml
################
variables:
  SKIP_CSPELL: "1"
  OPT_IN_TEST_MAX_PHP: 1
  OPT_IN_TEST_PREVIOUS_MAJOR: 1
  OPT_IN_TEST_NEXT_MAJOR: 1

To only test against Drupal 10, the gitlab-ci.yml file would look like:

################
# GitLabCI template for Drupal projects.
################

include:
  - project: $_GITLAB_TEMPLATES_REPO
    ref: $_GITLAB_TEMPLATES_REF
    file:
      - '/includes/include.drupalci.main.yml'
      - '/includes/include.drupalci.variables.yml'
      - '/includes/include.drupalci.workflows.yml'
#
################
# Pipeline configuration variables are defined with default values and descriptions in the file
# https://git.drupalcode.org/project/gitlab_templates/-/blob/main/includes/include.drupalci.variables.yml
################
variables:
  SKIP_CSPELL: "1"
  OPT_IN_TEST_MAX_PHP: 1
  OPT_IN_TEST_CURRENT: 0
  OPT_IN_TEST_PREVIOUS_MAJOR: 1
  OPT_IN_TEST_NEXT_MAJOR: 0

Debugging tests

If the tests pass locally but fail in GitLab CI it may be because Drupal's GitLab CI templates run Drupal in a directory (web) rather than the root, so tests access Drupal at http://localhost/web.

If you're using DDev then one option is alter the setup to run Drupal in a directory. See https://github.com/localgovdrupal/localgov_project/pull/232

Another is to use gitlab-ci-local to run the GitLab CI config directly on your local computer.

gitlab-ci-local

There's some documentation on using gitlab-ci-local for running Drupal CI tests here: https://project.pages.drupalcode.org/gitlab_templates/info/test-locally/

I had to make some further tweaks to get them running. Notably the application doesn't like the OPT_IN_TEST_MAX_PHP variable we're setting so this needs to be commented out before running the tests. Also, we are testing against multiple versions of Drupal, which we probably don't want when running things locally, so commenting out the configuration for this can be good idea.

An example test run would look like:

git clone [email protected]:project/localgov_alert_banner.git
cd localgov_alert_banner
sed -i 's/OPT_IN_TEST_MAX_PHP/#OPT_IN_TEST_MAX_PHP/g' .gitlab-ci.yml

# To run everything
gitlab-ci-local --remote-variables [email protected]:project/gitlab_templates=includes/include.drupalci.variables.yml=main --variable="_GITLAB_TEMPLATES_REPO=project/gitlab_templates"

# Just to run PHPUnit tests
gitlab-ci-local --remote-variables [email protected]:project/gitlab_templates=includes/include.drupalci.variables.yml=main --variable="_GITLAB_TEMPLATES_REPO=project/gitlab_templates" phpunit

A few notes on this:

The chromedriver is a little flaky and so JS tests can randomly fail.