Skip to content

Commit 4232781

Browse files
yduartepxnyo
yduartep
and
xnyo
authored
feat(ci): add new gha for running frontend e2e tests against specific stack (#360)
Co-authored-by: Giuseppe Guerra <[email protected]> Co-authored-by: Giuseppe Guerra <[email protected]>
1 parent cac5bc0 commit 4232781

File tree

8 files changed

+951
-0
lines changed

8 files changed

+951
-0
lines changed

‎actions/internal/plugins/frontend-e2e-against-stack/.gitignore‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Logs
2+
logs
3+
*.log
4+
npm-debug.log*
5+
yarn-debug.log*
6+
yarn-error.log*
7+
.pnpm-debug.log*
8+
9+
node_modules/
10+
11+
coverage/
12+
dist/
13+
14+
.*.bun-build
15+
16+
# Editor
17+
.idea

‎actions/internal/plugins/frontend-e2e-against-stack/CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Changelog
2+
3+
## 1.0.0
4+
5+
- Create new action frontend-e2e-against-stack to run e2e playwright tests against pre-selected stack.

‎actions/internal/plugins/frontend-e2e-against-stack/README.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Run e2e tests from frontend plugins against specific stack
2+
3+
This is a GitHub Action that help the execution of e2e tests on any frontend plugin that is using [Playwright](https://playwright.dev/) against specific selected stack.
4+
You need to define in which region the selected stack belong, the plugin from where are executed the tests and optionally which other plugins and datasources you want to provision when starting a Grafana instance.
5+
Also, you need to have the **playwright** configuration and the test specifications in the plugin that run the tests and the action will do the rest.
6+
This action use the following input parameters to run:
7+
8+
| Name | Description | Default | Required |
9+
| --------------------- |--------------------------------------------------------------------------------------------------------------------|-------------------|----------|
10+
| `plugin-directory` | Directory of the plugin, if not in the root of the repository. If provided, package-manager must also be provided. | . | No |
11+
| `package-manager` | The package manager to use for building the plugin | | No |
12+
| `npm-registry-auth` | Whether to authenticate to the npm registry in Google Artifact Registry | false | No |
13+
| `stack-slug` | Name of the stack where you want to run the tests | | Yes |
14+
| `env` | Region of the stack where you want to run the tests | | Yes |
15+
| `other-plugins` | List of other plugins that you want to enable separated by comma | | No |
16+
| `datasource-ids` | List of data sources that you want to enable separated by comma | | No |
17+
| `upload-report-path` | Name of the folder where you want to store the test report | playwright-report | No |
18+
| `upload-videos-path` | Name of the folder where you want to store the test videos | playwright-videos | No |
19+
| `plugin-secrets` | A JSON string containing key-value pairs of specific plugin secrets necessary to run the tests. | | No |
20+
| `grafana-ini-path` | Path to a custom grafana.ini file to configure the Grafana instance | | No |
21+
22+
## Example workflows
23+
24+
This is an example of how you could use this action.
25+
26+
```yml
27+
name: Build and Test PR
28+
29+
on:
30+
pull_request:
31+
32+
jobs:
33+
e2e-tests:
34+
permissions:
35+
contents: write
36+
id-token: write
37+
runs-on: ubuntu-latest
38+
steps:
39+
- uses: actions/checkout@v4
40+
with:
41+
persist-credentials: false
42+
43+
- name: Get plugin specific secrets
44+
id: create-plugin-secrets
45+
shell: bash
46+
run: |
47+
echo 'plugin-json-secrets={"MY_SECRET1": "value_secrete_1", "MY_SECRET2": "value_secret_2"}' >> "$GITHUB_OUTPUT"
48+
49+
- name: Run e2e cross app tests
50+
id: e2e-cross-apps-tests
51+
uses: grafana/plugin-ci-workflows/actions/internal/plugins/frontend-e2e-against-stack@main
52+
with:
53+
npm-registry-auth: "true"
54+
stack-slug: "mygrafanastack"
55+
env: "dev-central"
56+
other-plugins: "grafana-plugin1-app,grafana-plugin2-app"
57+
datasource-ids: "grafanacloud-mygrafanastack-prom,grafanacloud-mygrafanastack-logs"
58+
upload-report-path: "playwright-cross-apps-report"
59+
upload-videos-path: "playwright-cross-apps-videos"
60+
plugin-secrets: ${{ steps.create-plugin-secrets.outputs.plugin-json-secrets }}
61+
grafana-ini-path: "provisioning/custom-grafana.ini" # Optional
62+
```

‎actions/internal/plugins/frontend-e2e-against-stack/action.yml‎

Lines changed: 297 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,297 @@
1+
name: Run e2e tests
2+
description: Run e2e tests against specific stack and environment
3+
inputs:
4+
plugin-directory:
5+
description: Directory of the plugin, if not in the root of the repository. If provided, package-manager must also be provided.
6+
required: false
7+
default: .
8+
9+
package-manager:
10+
description: The package manager to use.
11+
required: false
12+
default: ""
13+
14+
npm-registry-auth:
15+
description: |
16+
Whether to authenticate to the npm registry in Google Artifact Registry.
17+
If true, the root of the plugin repository must contain a `.npmrc` file.
18+
required: false
19+
default: "false"
20+
21+
stack-slug:
22+
description: "Name of the stack where you want to run the tests"
23+
required: true
24+
25+
env:
26+
description: "Region of the stack where you want to run the tests"
27+
required: true
28+
29+
other-plugins:
30+
description: "List of other plugins that you want to enable separated by comma"
31+
required: false
32+
33+
datasource-ids:
34+
description: "List of data sources that you want to enable separated by comma"
35+
required: false
36+
37+
upload-report-path:
38+
description: "Name of the artifact where you want to store the test report"
39+
required: false
40+
default: "playwright-report"
41+
42+
upload-videos-path:
43+
description: "Name of the artifact where you want to store the test videos"
44+
required: false
45+
default: "playwright-videos"
46+
47+
plugin-secrets:
48+
description: "A JSON string containing key-value pairs of specific plugin secrets necessary to run the tests."
49+
required: false
50+
51+
grafana-ini-path:
52+
description: "Path to a custom grafana.ini file to configure the Grafana instance. Path should be relative to the plugin directory."
53+
required: false
54+
default: ""
55+
56+
runs:
57+
using: "composite"
58+
steps:
59+
- name: Checkout repository
60+
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
61+
with:
62+
persist-credentials: false
63+
64+
- name: Setup Node.js environment
65+
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
66+
with:
67+
node-version: "20"
68+
cache: "yarn"
69+
70+
- name: Install e2e action dependencies
71+
run: yarn install
72+
shell: bash
73+
working-directory: ${{ github.action_path }}
74+
75+
- name: Login to Google Cloud
76+
id: gcloud-auth
77+
if: inputs.npm-registry-auth == 'true'
78+
uses: google-github-actions/auth@8254fb75a33b976a221574d287e93919e6a36f70 # v2.1.6
79+
with:
80+
token_format: access_token
81+
workload_identity_provider: "projects/304398677251/locations/global/workloadIdentityPools/github/providers/github-provider"
82+
service_account: "github-cloud-npm-dev-pkgs@grafanalabs-workload-identity.iam.gserviceaccount.com"
83+
84+
- name: NPM registry auth
85+
if: inputs.npm-registry-auth == 'true'
86+
shell: bash
87+
working-directory: ${{ inputs.plugin-directory }}
88+
env:
89+
GOOGLE_APPLICATION_CREDENTIALS: ${{ env.GOOGLE_APPLICATION_CREDENTIALS }}
90+
run: npx google-artifactregistry-auth --credential-config ./.npmrc
91+
92+
- name: Install dependencies
93+
shell: bash
94+
working-directory: ${{ inputs.plugin-directory }}
95+
run: ${{ github.action_path }}/pm.sh install
96+
env:
97+
PACKAGE_MANAGER: ${{ inputs.package-manager }}
98+
99+
- name: Build
100+
shell: bash
101+
working-directory: ${{ inputs.plugin-directory }}
102+
run: ${{ github.action_path }}/pm.sh build
103+
env:
104+
PACKAGE_MANAGER: ${{ inputs.package-manager }}
105+
106+
# The action should end up with a dist/ folder, but if the working directory is not the root of the repo,
107+
# we need to copy the dist/ folder to the root of the repo.
108+
- name: Copy dist if needed
109+
run: |
110+
if [ "$PLUGIN_DIRECTORY" != "." ]; then
111+
mkdir -p dist
112+
cp -r $PLUGIN_DIRECTORY/dist/* dist/
113+
fi
114+
shell: bash
115+
if: inputs.plugin-directory != '.'
116+
env:
117+
PLUGIN_DIRECTORY: ${{ inputs.plugin-directory }}
118+
119+
- name: Extract plugin ID from plugin.json
120+
id: extract-plugin-id
121+
shell: bash
122+
working-directory: ${{ inputs.plugin-directory }}
123+
run: |
124+
if [ ! -f "src/plugin.json" ]; then
125+
echo "Error: plugin.json not found at src/plugin.json"
126+
exit 1
127+
fi
128+
129+
if ! jq empty src/plugin.json 2>/dev/null; then
130+
echo "Error: plugin.json contains invalid JSON"
131+
exit 1
132+
fi
133+
134+
PLUGIN_ID=$(jq -r '.id' src/plugin.json)
135+
136+
if [ "$PLUGIN_ID" = "null" ] || [ -z "$PLUGIN_ID" ]; then
137+
echo "Error: 'id' field not found or empty in plugin.json"
138+
exit 1
139+
fi
140+
141+
echo "plugin-id=$PLUGIN_ID" >> $GITHUB_OUTPUT
142+
echo "Extracted plugin ID: $PLUGIN_ID"
143+
144+
- name: Get common secrets
145+
id: get-common-secrets
146+
uses: grafana/shared-workflows/actions/get-vault-secrets@5d7e361bc7e0a183cde8afe9899fb7b596d2659b # v1.2.0
147+
with:
148+
common_secrets: |
149+
HG_TOKEN=hg-ci:token
150+
export_env: false
151+
152+
- name: Set plugin secrets as environment variables
153+
id: set-env-vars
154+
if: ${{ inputs.plugin-secrets != '' }}
155+
shell: bash
156+
env:
157+
SECRETS_JSON: "${{ inputs.plugin-secrets }}"
158+
run: |
159+
echo "Parsing and setting plugin environment variables..."
160+
echo "$SECRETS_JSON" | jq -r 'to_entries[] | "echo \"\(.key)=\(.value)\" >> $GITHUB_ENV"' | bash
161+
echo "Plugin environment variables set."
162+
163+
- name: Generate provisioning
164+
shell: bash
165+
run: npx plop --plopfile ${{ github.action_path }}/plopfile.mjs --dest . e2e-testing-provisioning
166+
working-directory: ${{ inputs.plugin-directory }}
167+
env:
168+
E2E_STACK_SLUG: ${{ inputs.stack-slug }}
169+
E2E_ENV: ${{ inputs.env }}
170+
HG_TOKEN: ${{ fromJSON(steps.get-common-secrets.outputs.secrets).HG_TOKEN }}
171+
E2E_PLUGIN_ID: ${{ steps.extract-plugin-id.outputs.plugin-id }}
172+
E2E_OTHER_PLUGINS: ${{ inputs.other-plugins }}
173+
E2E_DATASOURCE_IDS: ${{ inputs.datasource-ids }}
174+
E2E_GRAFANA_INI_PATH: ${{ inputs.grafana-ini-path }}
175+
176+
- name: Start server
177+
run: docker compose -f docker-compose.e2e.yaml up -d --build --quiet-pull --timestamps
178+
working-directory: ${{ inputs.plugin-directory }}
179+
shell: bash
180+
181+
- name: Install Playwright Browsers
182+
run: npx playwright install chromium --with-deps
183+
working-directory: ${{ inputs.plugin-directory }}
184+
shell: bash
185+
186+
- name: Wait for Grafana to be ready
187+
shell: bash
188+
env:
189+
MAIN_PLUGIN_ID: ${{ steps.extract-plugin-id.outputs.plugin-id }}
190+
OTHER_PLUGINS: ${{ inputs.other-plugins }}
191+
run: |
192+
echo "Waiting for Grafana to be available..."
193+
timeout=300 # 5 minutes timeout
194+
start_time=$(date +%s)
195+
196+
# Parse expected plugins from environment variables (safely)
197+
expected_plugins="$MAIN_PLUGIN_ID"
198+
if [ -n "$OTHER_PLUGINS" ]; then
199+
expected_plugins="$expected_plugins,$OTHER_PLUGINS"
200+
fi
201+
202+
# Convert comma-separated list to array for easier processing
203+
IFS=',' read -ra PLUGIN_ARRAY <<< "$expected_plugins"
204+
205+
echo "Expected plugins: $expected_plugins"
206+
207+
while true; do
208+
current_time=$(date +%s)
209+
elapsed=$((current_time - start_time))
210+
211+
if [ $elapsed -ge $timeout ]; then
212+
echo "Timeout reached: Grafana did not become ready within 5 minutes."
213+
echo "Container logs:"
214+
docker logs "$MAIN_PLUGIN_ID" --tail 50
215+
echo "Installed plugins:"
216+
curl -s http://localhost:3000/api/plugins 2>/dev/null | jq -r '.[].id' 2>/dev/null || echo "Could not fetch plugins list"
217+
exit 1
218+
fi
219+
220+
# Check if Grafana is responding to health checks
221+
if curl -f -s http://localhost:3000/api/health > /dev/null 2>&1; then
222+
echo "Grafana health endpoint is responding! (${elapsed}s elapsed)"
223+
224+
# Check if API is ready
225+
if curl -f -s http://localhost:3000/api/org > /dev/null 2>&1; then
226+
echo "Grafana API is ready! Now checking plugins..."
227+
228+
# Get list of installed plugins
229+
installed_plugins=$(curl -s http://localhost:3000/api/plugins 2>/dev/null | jq -r '.[].id' 2>/dev/null)
230+
231+
if [ $? -eq 0 ] && [ -n "$installed_plugins" ]; then
232+
# Check if all expected plugins are installed
233+
all_plugins_ready=true
234+
missing_plugins=""
235+
236+
for expected_plugin in "${PLUGIN_ARRAY[@]}"; do
237+
# Trim whitespace
238+
expected_plugin=$(echo "$expected_plugin" | xargs)
239+
if [ -n "$expected_plugin" ]; then
240+
if ! echo "$installed_plugins" | grep -q "^${expected_plugin}$"; then
241+
all_plugins_ready=false
242+
missing_plugins="$missing_plugins $expected_plugin"
243+
fi
244+
fi
245+
done
246+
247+
if [ "$all_plugins_ready" = true ]; then
248+
echo "All expected plugins are installed and ready!"
249+
break
250+
else
251+
echo "Still waiting for plugins:$missing_plugins (${elapsed}s elapsed)"
252+
fi
253+
else
254+
echo "Could not fetch plugins list, retrying... (${elapsed}s elapsed)"
255+
fi
256+
else
257+
echo "Grafana health OK, but API not fully ready yet... (${elapsed}s elapsed)"
258+
fi
259+
else
260+
echo "Waiting for Grafana health endpoint... (${elapsed}s elapsed)"
261+
fi
262+
263+
sleep 10
264+
done
265+
working-directory: ${{ inputs.plugin-directory }}
266+
267+
- name: Run Playwright tests
268+
shell: bash
269+
env:
270+
NODE_ENV: production
271+
run: npx playwright test
272+
working-directory: ${{ inputs.plugin-directory }}
273+
274+
- name: Stop grafana docker
275+
run: docker compose -f docker-compose.e2e.yaml down
276+
working-directory: ${{ inputs.plugin-directory }}
277+
shell: bash
278+
279+
- name: Upload E2E report
280+
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
281+
if: always()
282+
with:
283+
name: ${{ inputs.upload-report-path }}
284+
path: playwright-report/
285+
retention-days: 30
286+
287+
- name: Upload E2E videos
288+
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
289+
if: always()
290+
with:
291+
name: ${{ inputs.upload-videos-path }}
292+
path: test-results/
293+
retention-days: 30
294+
295+
branding:
296+
icon: "shield"
297+
color: "green"

0 commit comments

Comments
 (0)