Add acceptance tests with Codeception, DDEV, and Selenium by Copilot · Pull Request #120 · maikschneider/bw_captcha

Copilot · 2026-03-08T11:46:06Z

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

…ures, and test cases Co-authored-by: maikschneider <696865+maikschneider@users.noreply.github.com>

Co-authored-by: maikschneider <696865+maikschneider@users.noreply.github.com>

Copilot

Pull request overview

This PR introduces a full acceptance testing infrastructure for the bw_captcha TYPO3 extension. It adds Codeception with WebDriver (Selenium/Chromium), DDEV integration, SQL/YAML fixtures for a minimal TYPO3 installation, and a GitHub Actions workflow to run the suite in CI with retry support.

Changes:

Added Codeception acceptance tests covering frontend captcha rendering/reload/submit/audio and backend form editor
Added DDEV infrastructure (Selenium container, init script, config) to support headless browser tests
Added GitHub Actions workflow for CI execution with artifact upload on failure

Reviewed changes

Copilot reviewed 19 out of 21 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
`composer.json`	Adds Codeception packages, `typo3/testing-framework`, autoload-dev mapping, and test scripts
`codeception.yml`	Root Codeception config with WebDriver, Asserts modules and Chrome/Selenium settings
`Tests/Acceptance.suite.yml`	Acceptance suite config with WebDriver, Asserts, Db modules and populator
`Tests/Acceptance/Support/AcceptanceTester.php`	Actor class with `loginAsAdmin` helper
`Tests/Acceptance/Frontend/CaptchaAppearanceCest.php`	Tests for captcha image visibility and attributes
`Tests/Acceptance/Frontend/CaptchaReloadCest.php`	Tests for reload button behavior
`Tests/Acceptance/Frontend/CaptchaSubmitCest.php`	Tests for form submission with wrong/empty/correct captcha
`Tests/Acceptance/Frontend/CaptchaAudioCest.php`	Tests for audio button visibility and behavior
`Tests/Acceptance/Backend/FormEditorCest.php`	Test for Captcha element availability in backend form editor
`Tests/Acceptance/Fixtures/*.sql`	SQL fixtures for DB bootstrapping
`Tests/Acceptance/Fixtures/config.yaml`	TYPO3 site configuration
`Tests/Acceptance/Fixtures/captcha_test.form.yaml`	Form definition with Captcha element
`.ddev/commands/web/init-typo3`	Shell script to reset DB, run TYPO3 setup, and copy fixtures
`.ddev/docker-compose.selenium-chrome.yaml`	Selenium standalone Chromium service definition
`.ddev/config.selenium-standalone-chrome.yaml`	Web environment vars for Selenium-related browser testing
`.github/workflows/tests.yml`	CI workflow with DDEV, retry, and artifact upload
`.gitignore`	Ignores generated Codeception actor file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-08T15:02:01Z

+        $I->click('button[type="submit"]');
+        $I->wait(2);
+
+        $I->see('not valid');


The wrongCaptchaInputShowsError test uses $I->see('not valid'), but the actual error message rendered by CaptchaValidator is "The entered captcha is not correct." (translation key validator.captcha.notvalid in locallang.xlf). The substring 'not valid' does not appear anywhere in the rendered error text, so this assertion will always fail. The assertion should check for 'not correct' or 'entered captcha' instead.

Copilot · 2026-03-08T15:02:02Z

@@ -0,0 +1,3 @@
+INSERT INTO `be_users` (`uid`, `pid`, `tstamp`, `crdate`, `deleted`, `disable`, `starttime`, `endtime`, `description`, `username`, `avatar`, `password`, `admin`, `usergroup`, `lang`, `email`, `db_mountpoints`, `options`, `realName`, `allowed_languages`, `uc`, `file_mountpoints`, `file_permissions`, `workspace_perms`, `TSconfig`, `workspace_id`, `mfa`, `category_perms`, `lastlogin`, `password_reset_token`)
+VALUES
+	(1, 0, 1744190674, 1744190647, 0, 0, 0, 0, '', 'admin', 0, '$argon2i$v=19$m=65536,t=16,p=1$dE9iNFdhb2pCQUp5SmRMVw$TmYfrOorB12ic1eh+Z51zuO28ymoBSZT7WyFYafnF08', 1, '', 'default', 'admin@example.com', '', 3, 'Admin User', '', NULL, '', 'readFolder,writeFolder,addFolder,renameFolder,moveFolder,deleteFolder,readFile,writeFile,addFile,renameFile,replaceFile,moveFile,copyFile,deleteFile', 1, '', 0, NULL, '0', 0, '');


The init-typo3 script first runs typo3 setup --no-interaction --force, which creates TYPO3 database schema and an admin backend user (likely with uid=1). Then be_users.sql runs INSERT INTO be_users ... VALUES (1, ...) with uid=1. If the admin user already exists from typo3 setup, this INSERT will fail with a duplicate key error and abort the script (since set -e is in effect). Either the be_users.sql fixture should use INSERT IGNORE or REPLACE INTO, or the typo3 setup admin creation should be suppressed using the appropriate environment variable (e.g., TYPO3_SETUP_ADMIN_USERNAME can be set and the user later replaced, or --no-admin flag if available).

Copilot · 2026-03-08T15:02:02Z

+        $captchaPhrases = [$lifetime => $knownPhrase];
+        $sessionData = serialize(['captchaPhrases' => $captchaPhrases]);
+
+        $I->updateInDatabase('fe_sessions', ['ses_data' => $sessionData], ['1' => '1']);


The updateInDatabase call uses ['1' => '1'] as the WHERE criteria, which in Codeception's Db module means it tries to match a column literally named 1 with value '1'. This does not construct a valid WHERE 1=1 clause; it will either throw an SQL error (referencing a non-existent column) or match nothing. Even if a WHERE 1=1 was somehow constructed (matching all rows), the test has no way to identify the browser's current session row in fe_sessions, since there is no fe_sessions fixture and the session is dynamically created. The correct approach is to grab the current session cookie from the browser (e.g., grabCookie('fe_typo_user')), use it to look up the correct session row, and update only that row.

Suggested change

$I->updateInDatabase('fe_sessions', ['ses_data' => $sessionData], ['1' => '1']);

// Use the current frontend session cookie to target the correct fe_sessions row

$sessionId = $I->grabCookie('fe_typo_user');

if ($sessionId === null) {

$I->fail('Session cookie "fe_typo_user" not found; cannot inject captcha phrase.');

}

$I->updateInDatabase('fe_sessions', ['ses_data' => $sessionData], ['ses_id' => $sessionId]);

Copilot · 2026-03-08T15:02:02Z

+export TYPO3_DB_PORT=3306
+export TYPO3_DB_HOST="$dbHost"
+export TYPO3_DB_DBNAME="$dbName"
+export TYPO3_SERVER_TYPE=apache


The init-typo3 script sets TYPO3_SERVER_TYPE=apache for the TYPO3 setup command (line 39), but .ddev/config.yaml configures webserver_type: nginx-fpm. While this setting only affects how TYPO3 generates .htaccess vs nginx configurations during typo3 setup, using the wrong value may cause TYPO3 to write .htaccess rules that have no effect under nginx, potentially breaking URL handling (e.g., _ realurl or extension-based routing). Consider setting TYPO3_SERVER_TYPE=other or nginx to align with the actual DDEV webserver.

Suggested change

export TYPO3_SERVER_TYPE=apache

export TYPO3_SERVER_TYPE=nginx

Copilot · 2026-03-08T15:02:02Z

+mysql -Nse "SHOW TABLES" $dbCredentials "$dbName" | while read -r table; do
+    mysql -e "DROP TABLE ${table}" $dbCredentials "$dbName"
+done


The database clearing loop drops tables one by one without first disabling foreign key checks. If any tables have foreign key constraints, DROP TABLE statements may fail with errors like "Cannot drop table 'x' referenced as a foreign key constraint". A more robust approach is to use SET FOREIGN_KEY_CHECKS=0; before dropping tables (or use a single DROP DATABASE/CREATE DATABASE approach). Consider wrapping the loop with mysql -e "SET FOREIGN_KEY_CHECKS=0; ... SET FOREIGN_KEY_CHECKS=1;" or using mysqldump --no-data followed by recreation.

Suggested change

mysql -Nse "SHOW TABLES" $dbCredentials "$dbName" | while read -r table; do

mysql -e "DROP TABLE ${table}" $dbCredentials "$dbName"

done

mysql -e "SET FOREIGN_KEY_CHECKS=0" $dbCredentials "$dbName"

mysql -Nse "SHOW TABLES" $dbCredentials "$dbName" | while read -r table; do

mysql -e "DROP TABLE \`${table}\`" $dbCredentials "$dbName"

done

mysql -e "SET FOREIGN_KEY_CHECKS=1" $dbCredentials "$dbName"

Copilot · 2026-03-08T15:02:03Z

+      user: 'db'
+      password: 'db'
+      populate: true
+      cleanup: true


Both populate: true and cleanup: true are enabled in the Db module configuration. populate: true will re-run the init-typo3 populator script before each test, and cleanup: true wraps each test in a database transaction for rollback. These settings are contradictory: the populator script drops and recreates all tables and runs typo3 setup, which cannot be safely wrapped in a transaction. This may cause failures or unexpected behavior. If the intent is to reset the database between tests (using the populator), cleanup should be set to false. If cleanup: true is needed for row-level isolation, the populator should not recreate the schema.

Suggested change

cleanup: true

cleanup: false

Copilot · 2026-03-08T15:02:03Z

+      dsn: 'mysql:host=db;dbname=db'
+      user: 'db'
+      password: 'db'
+      populate: true


With populate: true, the init-typo3 script (which runs typo3 setup, drops and recreates all tables, and imports fixtures) will be executed before every single test case. Since typo3 setup can take many seconds, this would make the test suite very slow. Consider using populate: false (run the populator only once before the suite) or restructuring the DB seeding to be incremental rather than a full TYPO3 setup on each test.

Suggested change

populate: true

populate: false

maikschneider · 2026-03-08T16:43:21Z

@copilot Fix the failing test workflow.
A path in the init command is wrong.

…po3 robustness, test assertions, and session handling Co-authored-by: maikschneider <696865+maikschneider@users.noreply.github.com>

Copilot · 2026-03-08T16:49:14Z

@copilot Fix the failing test workflow.
A path in the init command is wrong.

Fixed in 3708ff1. The CI failure was caused by config.selenium-standalone-chrome.yaml containing a MINK_DRIVER_ARGS_WEBDRIVER value with complex JSON ([, {, : characters) that broke YAML parsing when DDEV generated the docker-compose base file. Removed the Drupal-specific env vars that aren't needed for Codeception tests. Also addressed all review feedback: FK checks in DB cleanup, TYPO3_SERVER_TYPE=other for nginx, REPLACE INTO for be_users, populate/cleanup: false, correct error message assertions matching locallang.xlf, and proper session cookie targeting via grabCookie('fe_typo_user').

Initial plan

f106f3d

Copilot AI assigned Copilot and maikschneider Mar 8, 2026

Copilot started work on behalf of maikschneider March 8, 2026 11:46 View session

Copilot AI and others added 2 commits March 8, 2026 11:55

Add acceptance test infrastructure with codeception, DDEV setup, fixt…

bc1d687

…ures, and test cases Co-authored-by: maikschneider <696865+maikschneider@users.noreply.github.com>

Add explicit permissions to tests workflow for security

fb64de9

Co-authored-by: maikschneider <696865+maikschneider@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add acceptance tests for TYPO3 extension~~ Add acceptance tests with Codeception, DDEV, and Selenium Mar 8, 2026

Copilot finished work on behalf of maikschneider March 8, 2026 11:59

maikschneider requested a review from Copilot March 8, 2026 14:52

Copilot started reviewing on behalf of maikschneider March 8, 2026 14:53 View session

Copilot AI reviewed Mar 8, 2026

View reviewed changes

Copilot started work on behalf of maikschneider March 8, 2026 16:43 View session

Fix CI failure and address review comments: fix YAML parsing, init-ty…

3708ff1

…po3 robustness, test assertions, and session handling Co-authored-by: maikschneider <696865+maikschneider@users.noreply.github.com>

Copilot finished work on behalf of maikschneider March 8, 2026 16:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add acceptance tests with Codeception, DDEV, and Selenium#120

Add acceptance tests with Codeception, DDEV, and Selenium#120
Copilot wants to merge 4 commits into
mainfrom
copilot/add-acceptance-tests-typo3-extension

Copilot AI commented Mar 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

Copilot AI Mar 8, 2026

Uh oh!

maikschneider commented Mar 8, 2026

Uh oh!

Copilot AI commented Mar 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

-        $I->updateInDatabase('fe_sessions', ['ses_data' => $sessionData], ['1' => '1']);
+        // Use the current frontend session cookie to target the correct fe_sessions row
+        $sessionId = $I->grabCookie('fe_typo_user');
+        if ($sessionId === null) {
+            $I->fail('Session cookie "fe_typo_user" not found; cannot inject captcha phrase.');
+        }
+        $I->updateInDatabase('fe_sessions', ['ses_data' => $sessionData], ['ses_id' => $sessionId]);

	export TYPO3_SERVER_TYPE=apache
	export TYPO3_SERVER_TYPE=nginx

Conversation

Copilot AI commented Mar 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

maikschneider commented Mar 8, 2026

Uh oh!

Copilot AI commented Mar 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Mar 8, 2026 •

edited

Loading