Skip to content

Commit e74c3bf

Browse files
pablothedudepablothedude
committed
Run all Behat features in isolation
The initialization config for the tests are now using the same config as middleware is using when a new environment is bootstrapped. After the initial setup of the tests a mysqldump is created in order to run all test in isolation and prevent unwanted side effects. And the event stream is no only used to boostrap an SRAA with a token.
1 parent c0c59c7 commit e74c3bf

File tree

10 files changed

+323
-63
lines changed

10 files changed

+323
-63
lines changed

‎stepup/tests/behat/.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
1-
/vendor
1+
/vendor
2+
setup.sql

‎stepup/tests/behat/features/bootstrap/FeatureContext.php‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
use Behat\Behat\Hook\Scope\BeforeScenarioScope;
55
use Behat\MinkExtension\Context\MinkContext;
66
use Behat\Behat\Hook\Scope\BeforeFeatureScope;
7+
use Behat\Testwork\Hook\Scope\BeforeSuiteScope;
78
use Ramsey\Uuid\Uuid;
89
use Surfnet\StepupBehat\Factory\CommandPayloadFactory;
910
use Surfnet\StepupBehat\Repository\SecondFactorRepository;
@@ -65,9 +66,9 @@ private static function execCommand(string $command): void
6566
}
6667

6768
/**
68-
* @BeforeFeature
69+
* @BeforeSuite
6970
*/
70-
public static function setupDatabase(BeforeFeatureScope $scope)
71+
public static function setupDatabase(BeforeSuiteScope $scope)
7172
{
7273
// Generate test databases
7374
echo "Preparing test schemas\n";
@@ -83,8 +84,25 @@ public static function setupDatabase(BeforeFeatureScope $scope)
8384
// Perform an event replay
8485
echo "Replaying event stream\n";
8586
self::execCommand("docker exec -t stepup-middleware-1 bin/console middleware:event:replay --env=smoketest_event_replay --no-interaction -vvv");
87+
8688
// Push config
89+
echo "Push Middleware config\n";
8790
self::execCommand("./fixtures/middleware-push-config.sh");
91+
self::execCommand("./fixtures/middleware-push-whitelist.sh");
92+
self::execCommand("./fixtures/middleware-push-institution.sh");
93+
94+
// Write base setup for initializing features
95+
echo "Dump empty setup to mysql file\n";
96+
self::execCommand("mysqldump -h mariadb -u root -psecret --single-transaction --databases middleware_test gateway_test > setup.sql");
97+
}
98+
99+
/**
100+
* @BeforeFeature
101+
*/
102+
public static function load(BeforeFeatureScope $scope)
103+
{
104+
// restore base setup
105+
self::execCommand("mysql -h mariadb -u root -psecret < setup.sql");
88106
}
89107

90108
/**

‎stepup/tests/behat/features/identity.feature‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,11 @@ Feature: A (S)RA(A) user reads identities of StepUp users in the middleware API
33
As a (S)RA(A) user
44
I must be able to read from the middleware API
55

6+
Scenario: Provision the following users:
7+
Given a user "jane-a1" identified by "urn:collab:person:institution-a.example.com:jane-a-ra" from institution "institution-a.example.com" with UUID "00000000-0000-4000-8000-000000000001"
8+
And a user "joe-a1" identified by "urn:collab:person:institution-a.example.com:jane-a-ra" from institution "institution-a.example.com" with UUID "00000000-0000-4000-8000-000000000002"
9+
And a user "jill-a1" identified by "urn:collab:person:institution-a.example.com:jane-a-ra" from institution "institution-a.example.com" with UUID "00000000-0000-4000-8000-000000000003"
10+
611
Scenario: A (S)RA(A) user reads identities without additional authorization context
712
Given I authenticate with user "ra" and password "secret"
813
When I request "GET /identity?institution=institution-a.example.com"

‎stepup/tests/behat/features/ra_candidate.feature‎

Lines changed: 0 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,7 @@ Feature: A RAA manages ra candidates in the ra environment
2323
Then I should see the following candidates:
2424
| name | institution |
2525
| jane-a-ra | institution-a.example.com |
26-
| jane-b1 institution-b.example.com | institution-b.example.com |
27-
| user-b-ra institution-b.example.com | institution-b.example.com |
28-
| user-b5 institution-b.example.com | institution-b.example.com |
2926
| Admin | dev.openconext.local |
30-
| SRAA2 | dev.openconext.local |
3127

3228
Scenario: SRAA user checks if "Jane Toppan" is a candidate for all institutions (with filtering on institution-a)
3329
Given I am logged in into the ra portal as "admin" with a "yubikey" token
@@ -42,9 +38,6 @@ Feature: A RAA manages ra candidates in the ra environment
4238
When I visit the RA promotion page
4339
Then I should see the following candidates for "institution-b.example.com":
4440
| name | institution |
45-
| jane-b1 institution-b.example.com | institution-b.example.com |
46-
| user-b-ra institution-b.example.com | institution-b.example.com |
47-
| user-b5 institution-b.example.com | institution-b.example.com |
4841

4942
Scenario: SRAA user demotes "jane-a-ra" to no longer be an RAA for "institution-a"
5043
Given I am logged in into the ra portal as "admin" with a "yubikey" token
@@ -57,18 +50,12 @@ Feature: A RAA manages ra candidates in the ra environment
5750
Then I should see the following candidates for "institution-a.example.com":
5851
| name | institution |
5952
| jane-a-ra | institution-a.example.com |
60-
| jane-b1 institution-b.example.com | institution-b.example.com |
61-
| user-b-ra institution-b.example.com | institution-b.example.com |
62-
| user-b5 institution-b.example.com | institution-b.example.com |
6353

6454
Scenario: SRAA user checks if "Jane Toppan" is not a candidate for "institution-b"
6555
Given I am logged in into the ra portal as "admin" with a "yubikey" token
6656
When I visit the RA promotion page
6757
Then I should see the following candidates for "institution-b.example.com":
6858
| name | institution |
69-
| jane-b1 institution-b.example.com | institution-b.example.com |
70-
| user-b-ra institution-b.example.com | institution-b.example.com |
71-
| user-b5 institution-b.example.com | institution-b.example.com |
7259

7360
Scenario: SRAA user checks if "Jane Toppan" is not listed for "institution-a"
7461
Given I am logged in into the ra portal as "admin" with a "yubikey" token

‎stepup/tests/behat/features/ra_multiple_tokens.feature‎

Lines changed: 0 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -54,11 +54,7 @@ Feature: A RAA (jane a ra) has two loa 3 tokens which makes her a valid RA candi
5454
Then I should see the following candidates:
5555
| name | institution |
5656
| jane-a-ra | institution-a.example.com |
57-
| jane-b1 institution-b.example.com | institution-b.example.com |
58-
| user-b5 institution-b.example.com | institution-b.example.com |
59-
| user-b-ra institution-b.example.com | institution-b.example.com |
6057
| Admin | dev.openconext.local |
61-
| SRAA2 | dev.openconext.local |
6258

6359
Scenario: SRAA user checks if "jane-a-ra" is a candidate for institutions if relieved from the RAA role
6460
Given I am logged in into the ra portal as "admin" with a "yubikey" token
@@ -68,11 +64,7 @@ Feature: A RAA (jane a ra) has two loa 3 tokens which makes her a valid RA candi
6864
And I should see the following candidates:
6965
| name | institution |
7066
| jane-a-ra | institution-a.example.com |
71-
| jane-b1 institution-b.example.com | institution-b.example.com |
72-
| user-b5 institution-b.example.com | institution-b.example.com |
73-
| user-b-ra institution-b.example.com | institution-b.example.com |
7467
| Admin | dev.openconext.local |
75-
| SRAA2 | dev.openconext.local |
7668

7769
Scenario: Sraa revokes only one vetted token from "jane-a-ra" and that shouldn't remove her as candidate
7870
Given I am logged in into the ra portal as "admin" with a "yubikey" token
@@ -82,11 +74,7 @@ Feature: A RAA (jane a ra) has two loa 3 tokens which makes her a valid RA candi
8274
And I should see the following candidates:
8375
| name | institution |
8476
| jane-a-ra | institution-a.example.com |
85-
| jane-b1 institution-b.example.com | institution-b.example.com |
86-
| user-b5 institution-b.example.com | institution-b.example.com |
87-
| user-b-ra institution-b.example.com | institution-b.example.com |
8877
| Admin | dev.openconext.local |
89-
| SRAA2 | dev.openconext.local |
9078

9179
Scenario: Sraa revokes the last vetted token from "Jane Toppan" and that must remove her as candidate
9280
Given I am logged in into the ra portal as "admin" with a "yubikey" token
@@ -95,8 +83,4 @@ Feature: A RAA (jane a ra) has two loa 3 tokens which makes her a valid RA candi
9583
Then I visit the RA promotion page
9684
And I should see the following candidates:
9785
| name | institution |
98-
| jane-b1 institution-b.example.com | institution-b.example.com |
99-
| user-b5 institution-b.example.com | institution-b.example.com |
100-
| user-b-ra institution-b.example.com | institution-b.example.com |
10186
| Admin | dev.openconext.local |
102-
| SRAA2 | dev.openconext.local |

‎stepup/tests/behat/fixtures/events.sql‎

Lines changed: 47 additions & 31 deletions
Large diffs are not rendered by default.

‎stepup/tests/behat/fixtures/middleware-institution.json‎

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
{
2+
"dev.openconext.local": {
3+
"use_ra_locations": true,
4+
"show_raa_contact_information": true,
5+
"verify_email": false,
6+
"allowed_second_factors": [],
7+
"number_of_tokens_per_identity": 5,
8+
"self_vet": true,
9+
"allow_self_asserted_tokens": true
10+
},
11+
"institution-a.example.com": {
12+
"use_ra_locations": true,
13+
"show_raa_contact_information": true,
14+
"verify_email": true,
15+
"allowed_second_factors": [],
16+
"number_of_tokens_per_identity": 2,
17+
"self_vet": true,
18+
"sso_on_2fa": true,
19+
"allow_self_asserted_tokens": true
20+
},
21+
"institution-b.example.com": {
22+
"use_ra_locations": false,
23+
"show_raa_contact_information": true,
24+
"verify_email": false,
25+
"allowed_second_factors": [],
26+
"number_of_tokens_per_identity": 2,
27+
"self_vet": true,
28+
"sso_on_2fa": true,
29+
"allow_self_asserted_tokens": true
30+
},
31+
"institution-d.example.com": {
32+
"use_ra_locations": true,
33+
"show_raa_contact_information": false,
34+
"verify_email": true,
35+
"allowed_second_factors": [
36+
"yubikey",
37+
"tiqr"
38+
],
39+
"number_of_tokens_per_identity": 1,
40+
"self_vet": false,
41+
"allow_self_asserted_tokens": false
42+
},
43+
"institution-f.example.com": {
44+
"use_ra_locations": true,
45+
"show_raa_contact_information": false,
46+
"verify_email": false,
47+
"allowed_second_factors": [],
48+
"number_of_tokens_per_identity": 2,
49+
"use_ra": [
50+
"institution-f.example.com",
51+
"institution-a.example.com",
52+
"institution-b.example.com"
53+
],
54+
"use_raa": [
55+
"institution-f.example.com",
56+
"institution-a.example.com"
57+
],
58+
"select_raa": [
59+
"institution-f.example.com"
60+
],
61+
"self_vet": true,
62+
"allow_self_asserted_tokens": false
63+
},
64+
"institution-g.example.com": {
65+
"use_ra_locations": true,
66+
"show_raa_contact_information": true,
67+
"verify_email": false,
68+
"allowed_second_factors": [],
69+
"number_of_tokens_per_identity": 2,
70+
"use_ra": [
71+
"institution-g.example.com"
72+
],
73+
"use_raa": [
74+
"institution-g.example.com"
75+
],
76+
"select_raa": [
77+
"institution-h.example.com",
78+
"institution-g.example.com"
79+
],
80+
"self_vet": true,
81+
"allow_self_asserted_tokens": false
82+
},
83+
"institution-h.example.com": {
84+
"use_ra_locations": true,
85+
"show_raa_contact_information": true,
86+
"verify_email": false,
87+
"allowed_second_factors": [],
88+
"number_of_tokens_per_identity": 2,
89+
"use_ra": [
90+
"institution-h.example.com"
91+
],
92+
"use_raa": [
93+
"institution-h.example.com"
94+
],
95+
"select_raa": [
96+
"institution-h.example.com",
97+
"institution-g.example.com"
98+
],
99+
"self_vet": true,
100+
"allow_self_asserted_tokens": true
101+
},
102+
"institution-v.example.com": {
103+
"use_ra_locations": true,
104+
"show_raa_contact_information": true,
105+
"verify_email": false,
106+
"allowed_second_factors": [],
107+
"number_of_tokens_per_identity": 2,
108+
"use_ra": [],
109+
"use_raa": [],
110+
"select_raa": [
111+
"institution-a.example.com",
112+
"institution-b.example.com"
113+
],
114+
"self_vet": true,
115+
"allow_self_asserted_tokens": false
116+
},
117+
"institution-i.example.com": {
118+
"use_ra_locations": true,
119+
"show_raa_contact_information": true,
120+
"verify_email": false,
121+
"allowed_second_factors": [],
122+
"number_of_tokens_per_identity": 2,
123+
"use_ra": [
124+
"institution-v.example.com"
125+
],
126+
"use_raa": [
127+
"institution-v.example.com"
128+
],
129+
"select_raa": [],
130+
"self_vet": true,
131+
"allow_self_asserted_tokens": false
132+
}
133+
}

‎stepup/tests/behat/fixtures/middleware-push-institution.sh‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
#!/bin/bash
2+
3+
CWD=$(pwd)
4+
5+
function error_exit {
6+
echo "${1}"
7+
if [ -n "${TMP_FILE}" -a -d "${TMP_FILE}" ]; then
8+
rm "${TMP_FILE}"
9+
fi
10+
cd "${CWD}"
11+
exit 1
12+
}
13+
14+
# Script to write the middleware institution config
15+
16+
TMP_FILE=$(mktemp -t midcfg.XXXXXX)
17+
if [ $? -ne "0" ]; then
18+
error_exit "Could not create temp file"
19+
fi
20+
21+
echo "Pushing new institution configuration to: https://middleware.dev.openconext.local/management/institution-configuration"
22+
23+
http_response=$(curl -k --write-out %\{http_code\} --output "${TMP_FILE}" -XPOST -s \
24+
-u management:secret \
25+
-H "Accept: application/json" \
26+
-H "Content-type: application/json" \
27+
-d @./fixtures/middleware-institution.json \
28+
https://middleware.dev.openconext.local/management/institution-configuration)
29+
30+
output=$(cat ${TMP_FILE})
31+
rm ${TMP_FILE}
32+
echo $output
33+
34+
res=$?
35+
if [ $res -ne "0" ]; then
36+
error_exit "Curl failed with code $res"
37+
fi
38+
39+
# Check for HTTP 200
40+
if [ "${http_response}" -ne "200" ]; then
41+
error_exit "Unexpected HTTP response: ${http_response}"
42+
fi
43+
44+
# On success JSON output should start with: {"status":"OK"
45+
ok_count=$(echo "${output}" | grep -c "status")
46+
if [ $ok_count -ne "1" ]; then
47+
error_exit "Expected one JSON \"status: OK\" in response, found $ok_count"
48+
fi
49+
50+
echo "OK. New config pushed"

‎stepup/tests/behat/fixtures/middleware-push-whitelist.sh‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
#!/bin/bash
2+
3+
CWD=$(pwd)
4+
5+
function error_exit {
6+
echo "${1}"
7+
if [ -n "${TMP_FILE}" -a -d "${TMP_FILE}" ]; then
8+
rm "${TMP_FILE}"
9+
fi
10+
cd ${CWD}
11+
exit 1
12+
}
13+
14+
# Script to write the middleware institution whitelist
15+
16+
TMP_FILE=$(mktemp -t midcfg.XXXXXX)
17+
if [ $? -ne "0" ]; then
18+
error_exit "Could not create temp file"
19+
fi
20+
21+
echo "Pushing new institution whitelist to: http://middleware.dev.openconext.local/management/whitelist/replace"
22+
23+
http_response=$(curl -k --write-out %{http_code} --output ${TMP_FILE} -XPOST -s \
24+
-u management:secret \
25+
-H "Accept: application/json" \
26+
-H "Content-type: application/json" \
27+
-d @./fixtures/middleware-whitelist.json \
28+
https://middleware.dev.openconext.local/management/whitelist/replace)
29+
30+
res=$?
31+
32+
output=$(cat "${TMP_FILE}")
33+
rm "${TMP_FILE}"
34+
echo "$output"
35+
36+
if [ $res -ne "0" ]; then
37+
error_exit "Curl failed with code $res"
38+
fi
39+
40+
# Check for HTTP 200
41+
if [ "${http_response}" -ne "200" ]; then
42+
error_exit "Unexpected HTTP response: ${http_response}"
43+
fi
44+
45+
# On success JSON output should start with: {"status":"OK"
46+
ok_count=$(echo "${output}" | grep -c "status")
47+
if [ $ok_count -ne "1" ]; then
48+
error_exit "Expected one JSON \"status: OK\" in response, found $ok_count"
49+
fi
50+
51+
echo "OK. New config pushed"

0 commit comments

Comments
 (0)