Skip to content

Commit a7f2b5e

Browse files
Ma11hewThomasMa11hewThomas
authored
Merge pull request #123 from ctrf-io/docs/integrations
feat: add integrations
2 parents b124017 + 5b82745 commit a7f2b5e

File tree

7 files changed

+230
-10
lines changed

7 files changed

+230
-10
lines changed

‎README.md

Lines changed: 25 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -72,10 +72,11 @@ Checkout the built-in reports [here](docs/report-showcase.md)
7272
6. [GitHub Token](#github-token)
7373
7. [Storing Artifacts](#storing-artifacts)
7474
8. [Filtering](#filtering)
75-
9. [Generating an AI Report](#generating-an-ai-report)
76-
10. [Run With NPX](#run-with-npx)
77-
11. [Report Showcase](#report-showcase)
78-
12. [What is CTRF?](#what-is-ctrf)
75+
9. [Integrations](#integrations)
76+
10. [Generating an AI Report](#generating-an-ai-report)
77+
11. [Run With NPX](#run-with-npx)
78+
12. [Report Showcase](#report-showcase)
79+
13. [What is CTRF?](#what-is-ctrf)
7980

8081
## Usage
8182

@@ -160,6 +161,7 @@ There are several inputs available
160161
fetch-previous-results: false # Always fetch previous workflow runs when using custom templates. Default is false
161162
group-by: 'filePath' # Specify grouping for applicable reports (e.g., suite or file path). Default is filePath
162163
always-group-by: false # Force grouping by suite or file path for all reports. Default is false
164+
integrations-config: '{}' # JSON configuration for integrations with other developer tools
163165
if: always()
164166
```
165167

@@ -213,14 +215,15 @@ input and providing the issue number ():
213215
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
214216
if: always()
215217
```
218+
216219
**Note:** Special considerations apply to pull requests from forks. See [Fork Pull Requests](docs/fork-pull-requests.md) for details.
217220

218221
### Comment Management Inputs
219222

220223
`--update-comment` An existing tagged comment is found, the new report is
221224
appended to it. Otherwise, a new comment is created.
222225

223-
`--overwrite-comment` An existing tagged comment is found, that comment’s entire
226+
`--overwrite-comment` An existing tagged comment is found, that comment's entire
224227
content is replaced with the new report. Otherwise, a new comment is created.
225228

226229
`--comment-tag` A unique identifier for comments posted. Used to find and
@@ -332,11 +335,25 @@ previous results as follows:
332335
This ensures that you only see workflow runs that are related to your current
333336
branch or pull request
334337

338+
## Integrations
339+
340+
CTRF tooling offers seamless developer tool integration, allowing you to combine the GitHub Test Reporter with the following tools:
341+
342+
| Integration | Description | Repository |
343+
|------------|-------------|------------|
344+
| Slack Test Reporter | Send test results and notifications to Slack channels | [ctrf-io/slack-test-reporter](https://github.com/ctrf-io/slack-test-reporter) |
345+
| Microsoft Teams Test Reporter | Post test results and alerts to Teams channels | [ctrf-io/teams-test-reporter](https://github.com/ctrf-io/teams-test-reporter) |
346+
| AI Test Reporter | Intelligent test analysis using leading AI models | [ctrf-io/ai-test-reporter](https://github.com/ctrf-io/ai-test-reporter) |
347+
348+
For detailed information about configuring and using these integrations, see our [Integrations Documentation](docs/integrations.md).
349+
350+
Integrations are currently in beta. Please report any issues to the [GitHub Test Reporter repository](https://github.com/ctrf-io/github-test-reporter/issues).
351+
335352
## Generating an AI Report
336353

337354
You can generate human-readable AI report for your failed tests using models
338-
from the leading AI providers by using the
339-
[AI Test Reporter](https://github.com/ctrf-io/ai-test-reporter)
355+
from the leading AI providers by using the AI Test Reporter integration or the
356+
[AI Test Reporter](https://github.com/ctrf-io/ai-test-reporter) directly.
340357

341358
## Run With NPX
342359

@@ -360,4 +377,4 @@ analyzing test outcomes across multiple platforms becomes more straightforward.
360377
## Support Us
361378

362379
If you find this project useful, consider giving it a GitHub star ⭐ It means a
363-
lot to us.
380+
lot to us.

‎action.yml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -197,6 +197,13 @@ inputs:
197197
required: false
198198
default: false
199199

200+
# Integration Configuration
201+
integrations-config:
202+
description:
203+
'JSON configuration for integrations with other developer tools'
204+
required: false
205+
default: '{}'
206+
200207
outputs:
201208
summary:
202209
description: 'The markdown summary generated from the test report.'

‎badges/coverage.svg

Lines changed: 1 addition & 1 deletion
Loading

‎dist/index.js

Lines changed: 9 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎dist/index.js.map

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/integrations.md

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
# Integrations
2+
3+
GitHub Test Reporter supports various integrations to enhance your test reporting workflow. These integrations allow you to seamlessly combine the GitHub Test Reporter with popular developer tools.
4+
5+
You can use multiple integrations in the same workflow.
6+
7+
Integrations are currently in beta. Please report any issues to the [GitHub Test Reporter repository](https://github.com/ctrf-io/github-test-reporter/issues).
8+
9+
| Integration | Description | Repository |
10+
|------------|-------------|------------|
11+
| Slack Test Reporter | Send test results and notifications to Slack channels | [ctrf-io/slack-test-reporter](https://github.com/ctrf-io/slack-test-reporter) |
12+
| Microsoft Teams Test Reporter | Post test results and alerts to Teams channels | [ctrf-io/teams-test-reporter](https://github.com/ctrf-io/teams-test-reporter) |
13+
| AI Test Reporter | Intelligent test analysis using leading AI models | [ctrf-io/ai-test-reporter](https://github.com/ctrf-io/ai-test-reporter) |
14+
15+
To suggest a new integration, please open a discussion on the [CTRF Discussions](https://github.com/ctrf-io/ctrf/discussions).
16+
17+
## Configuration
18+
19+
Integrations are configured using the `integrations-config` input in your GitHub Actions workflow. The configuration is passed as a JSON string:
20+
21+
```yaml
22+
- name: Publish Test Report
23+
uses: ctrf-io/github-test-reporter@v1
24+
with:
25+
report-path: './ctrf/*.json'
26+
integrations-config: |
27+
{
28+
"slack": {
29+
"enabled": true,
30+
"action": "results"
31+
},
32+
"teams": {
33+
"enabled": true,
34+
"action": "results"
35+
},
36+
"ai": {
37+
"enabled": true,
38+
"action": "openai"
39+
}
40+
}
41+
env:
42+
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
43+
TEAMS_WEBHOOK_URL: ${{ secrets.TEAMS_WEBHOOK_URL }}
44+
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
45+
if: always()
46+
```
47+
48+
## Available Integrations
49+
50+
### Slack Integration
51+
52+
The Slack integration allows you to send test results directly to your Slack channels.
53+
54+
The Slack integration is powered by the [Slack Test Reporter](https://github.com/ctrf-io/slack-test-reporter), see the latest documentation for more information.
55+
56+
Actions available:
57+
58+
- `results`
59+
- `failed`
60+
- `flaky`
61+
- `ai`
62+
63+
Requires the `SLACK_WEBHOOK_URL` environment variable to be set.
64+
65+
```yaml
66+
- name: Publish Test Report
67+
uses: ctrf-io/github-test-reporter@v1
68+
with:
69+
report-path: './ctrf/*.json'
70+
integrations-config: |
71+
{
72+
"slack": {
73+
"enabled": true,
74+
"action": "results",
75+
"options": {
76+
"title": "Test Results",
77+
"prefix": "Custom prefix",
78+
"suffix": "Custom suffix",
79+
"consolidated": false,
80+
"onFailOnly": false
81+
}
82+
}
83+
}
84+
env:
85+
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
86+
if: always()
87+
```
88+
89+
See the [Slack Test Reporter](https://github.com/ctrf-io/slack-test-reporter) for more information.
90+
91+
### Microsoft Teams Integration
92+
93+
The Microsoft Teams integration allows you to send test results directly to your Microsoft Teams channels.
94+
95+
The Microsoft Teams integration is powered by the [Microsoft Teams Test Reporter](https://github.com/ctrf-io/teams-test-reporter), see the documentation for more information.
96+
97+
Actions available:
98+
99+
- `results`
100+
- `failed`
101+
- `flaky`
102+
- `ai`
103+
104+
Requires the `TEAMS_WEBHOOK_URL` environment variable to be set.
105+
106+
```yaml
107+
- name: Publish Test Report
108+
uses: ctrf-io/github-test-reporter@v1
109+
with:
110+
report-path: './ctrf/*.json'
111+
integrations-config: |
112+
{
113+
"teams": {
114+
"enabled": true,
115+
"action": "results",
116+
"options": {
117+
"title": "Test Results",
118+
"onFailOnly": false
119+
}
120+
}
121+
}
122+
env:
123+
TEAMS_WEBHOOK_URL: ${{ secrets.TEAMS_WEBHOOK_URL }}
124+
if: always()
125+
```
126+
127+
See the [Microsoft Teams Test Reporter](https://github.com/ctrf-io/teams-test-reporter) for more information.
128+
129+
### AI Integration
130+
131+
The AI integration provides intelligent analysis of your test results using advanced AI models.
132+
133+
The AI integration is powered by the [AI Test Reporter](https://github.com/ctrf-io/ai-test-reporter), see the latestdocumentation for more information.
134+
135+
Actions available:
136+
137+
- `openai`
138+
- `claude`
139+
- `azure-openai`
140+
- `grok`
141+
- `deepseek`
142+
- `mistral`
143+
- `gemini`
144+
- `perplexity`
145+
- `openrouter`
146+
147+
Requires the environment variable to be set for the AI provider you are using.
148+
149+
```yaml
150+
- name: Publish Test Report
151+
uses: ctrf-io/github-test-reporter@v1
152+
with:
153+
report-path: './ctrf/*.json'
154+
integrations-config: |
155+
{
156+
"ai": {
157+
"enabled": true,
158+
"action": "openai",
159+
"options": {
160+
"model": "gpt-4",
161+
"systemPrompt": "Custom system prompt",
162+
"frequencyPenalty": 0,
163+
"maxTokens": 1000,
164+
"presencePenalty": 0,
165+
"temperature": 0.7,
166+
"topP": 1,
167+
"log": false,
168+
"maxMessages": 10,
169+
"consolidate": false,
170+
"deploymentId": "your-azure-deployment-id"
171+
}
172+
}
173+
}
174+
env:
175+
OPENAI_TOKEN: ${{ secrets.OPENAI_TOKEN }}
176+
if: always()
177+
```

‎src/ctrf/enrichers.ts

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -542,6 +542,16 @@ export function enrichCurrentReportWithRunDetails(
542542
extendedReport.results.environment.extra =
543543
extendedReport.results.environment.extra ?? {}
544544

545+
if (!extendedReport.results.environment.buildName) {
546+
extendedReport.results.environment.buildName = run.job
547+
}
548+
if (!extendedReport.results.environment.buildNumber) {
549+
extendedReport.results.environment.buildNumber = run.run_number.toString()
550+
}
551+
if (!extendedReport.results.environment.buildUrl) {
552+
extendedReport.results.environment.buildUrl = run.build_url
553+
}
554+
545555
extendedReport.results.environment.extra.runId = run.run_id
546556
extendedReport.results.environment.extra.runNumber = run.run_number
547557
extendedReport.results.environment.extra.buildUrl = run.build_url

0 commit comments

Comments
 (0)