Skip to content

Commit a48edc8

Browse files
authored
Merge pull request #1127 from reportportal/develop
Publish new section with Agentic test results description
2 parents 06b502d + 9bcd38a commit a48edc8

9 files changed

Lines changed: 97 additions & 11 deletions

File tree

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
---
2+
sidebar_position: 4
3+
sidebar_label: Agentic test results
4+
description: Learn how to report AI agentic test results to ReportPortal to get a unified view of your testing pipeline, verify AI agent work with evidence, and make informed release decisions.
5+
---
6+
7+
# Agentic test results
8+
9+
As AI-driven testing becomes a core part of modern quality engineering, ReportPortal provides first-class support for **agentic test results** — outcomes produced by autonomous AI agents rather than traditional scripted test automation.
10+
11+
Starting from version [26.0.3](https://github.com/reportportal/reportportal/releases/tag/26.0.3), ReportPortal can receive and display results from AI agentic testing pipelines alongside classic automation results. This gives teams a **unified view** of their entire testing landscape — human-authored tests, automated regression suites, and AI agent sessions — in one place.
12+
13+
## Why report agentic results to ReportPortal?
14+
15+
- **Full testing picture.** Consolidate results from AI agents and traditional automation into a single dashboard to support confident release decisions.
16+
- **Evidence-based verification.** Capture logs, attachments, and step-level details from agent sessions so you can review exactly what each AI agent tested and why it passed or failed.
17+
- **Traceability.** Distinguish agentic runs from classic launches at a glance in the ReportPortal UI, making it easy to track coverage and trends over time.
18+
19+
## How to report agentic results
20+
21+
To integrate your AI testing workflows with ReportPortal, choose one of the following approaches.
22+
23+
### Programmatically
24+
25+
Use ReportPortal [integration guides](./ImplementOwnIntegration.md) and [reporting SDKs](./ReportingSDK.md) for various programming languages to send results from your agent pipeline via the ReportPortal API.
26+
27+
### Using agent skills
28+
29+
Coming soon.
30+
31+
In the meantime, you can explore and experiment with the [goRP CLI skill](https://github.com/reportportal/goRP/blob/master/.cursor/skills/reportportal/SKILL.md) to report results directly from an AI agent session.
32+
33+
:::tip Mark your launches as agentic
34+
To [distinguish](/work-with-reports/ViewLaunches#launches-tab) agentic launches from classic automation in the ReportPortal UI, include the following system attribute when starting a launch:
35+
36+
```json
37+
{
38+
"key": "isAgentic",
39+
"value": true,
40+
"system": true
41+
}
42+
```
43+
44+
This attribute is hidden from end users but is used by ReportPortal to apply the agentic visual indicator on the Launches page along with extended filtering options.
45+
:::
46+
47+
## Integration showcase
48+
49+
The [AgenticQA](https://test.io/ai-in-qa/agentic-qa) testing platform integrates with ReportPortal out of the box, streaming evidence from AI agent test sessions in real time.

docs/log-data-in-reportportal/HowToRunYourTests.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: Learn how to run your tests with ReportPortal test automation repor
55

66
# How to run your tests
77

8-
Automated tests can be executed in the any CI tool you are using on the project (Jenkins, Bamboo, Azure DevOps, GitHub Actions, CircleCI, TeamCity, GitLab, Travis CI, Codeship, Buddy, GoCD, Wercker and more).
8+
Automated tests can be executed in any CI tool you are using on the project (Jenkins, Bamboo, Azure DevOps, GitHub Actions, CircleCI, TeamCity, GitLab, Travis CI, Codeship, Buddy, GoCD, Wercker and more).
99
It doesn't require any settings from ReportPortal.
1010

1111
So that the test results get into the ReportPortal, you need to perform [test framework integration](/log-data-in-reportportal/test-framework-integration/) or [implement own integration](/log-data-in-reportportal/ImplementOwnIntegration).
Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,19 @@
11
---
2-
sidebar_position: 2
2+
sidebar_position: 1
33
sidebar_label: Implement own integration
44
description: Learn how to implement custom integration with ReportPortal using the developer guide.
55
---
66

77
# Implement own integration
88

9+
Willing to build your own integration with ReportPortal?
10+
911
You can find a detailed description of how report results to ReportPortal in our [developers` guide](/developers-guides/ReportingDevelopersGuide).
1012

1113
Based on this information you can create your integration with a test framework.
1214

15+
Consider using our [Reporting SDKs](./ReportingSDK.md) for different programming languages.
16+
1317
:::note
1418
We can propose a paid support for integration with a new framework or supporting an existing one. If you need more details, please drop a letter on support@reportportal.io.
1519
:::

docs/log-data-in-reportportal/ImportDataToReportPortal.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
sidebar_position: 4
2+
sidebar_position: 5
33
sidebar_label: Import data to ReportPortal
44
description: Import test logs into ReportPortal using the UI or API, supporting JUnit and RobotFramework formats for streamlined test data integration.
55
---
Lines changed: 29 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,33 @@
11
---
2-
sidebar_position: 3
2+
sidebar_position: 2
33
sidebar_label: Reporting SDK
4-
description: Integrate your test framework with ReportPortal using the Reporting SDK to log detailed test data, including nested steps and test case IDs, for enhanced reporting.
4+
description: Integrate your test framework with ReportPortal using the Reporting SDKs to log detailed test data, including nested steps and test case IDs, for enhanced reporting.
55
---
66

77
# Reporting SDK
88

9+
There are reporting SDKs for different programming languages covering the essentials to build the integration with ReportPortal from scratch.
10+
11+
## Java
12+
13+
[client-java](https://github.com/reportportal/client-java)
14+
15+
## C# (.NET)
16+
17+
[client-dotnet](https://github.com/reportportal/client-dotnet)
18+
19+
## JavaScript
20+
21+
[client-javascript](https://github.com/reportportal/client-javascript)
22+
23+
## Python
24+
25+
[client-python](https://github.com/reportportal/client-Python)
26+
27+
## Feature guides
28+
29+
### Nested steps
30+
931
How to report tests with nested steps?
1032

1133
**JAVA**
@@ -15,7 +37,11 @@ How to report tests with nested steps?
1537
**.NET**
1638
* https://github.com/reportportal/commons-net/blob/master/docs/Logging.md
1739

18-
How to report tests with Test case ID. How to add TestID from your Test Management System to reported execution.
40+
### Test case id
41+
42+
How to report tests with Test case ID.
43+
How to add TestID from your Test Management System to reported execution.
1944

45+
**JAVA**
2046
* https://github.com/reportportal/client-java/wiki/Test-case-ID
2147
* https://github.com/reportportal/client-java/wiki/Test-case-ID-Key

docs/log-data-in-reportportal/test-framework-integration/index.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
---
22
sidebar_label: Test Framework Integration
3+
sidebar_position: 3
34
description: Integrate your test framework with ReportPortal. Configure agents, send results, and enrich logs, attachments, and attributes for a clear automation test report.
45
---
56

docs/work-with-reports/ViewLaunches.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,12 +28,12 @@ A typical Launch structure comprises the following elements: **Suite** \> **Test
2828

2929
Launches are categorized into two types:
3030

31-
* AGENTIC – launches reported via AI agent (by adding system attribute 'isAgentic':true on Agent side on launch start request)
32-
* AUTOMATION - launches reported via Test frameworks agents
31+
* AGENTIC – launches reported by [AI agents](/log-data-in-reportportal/AgenticTestResults).
32+
* AUTOMATION - launches reported via [Test frameworks integrations](/log-data-in-reportportal/test-framework-integration/).
3333

3434
Agentic launches are visually marked on the Launches page with an **Agentic** badge.
3535

36-
<MediaViewer src={require('./img/view-launches/FilterLatestRun.png')} alt="Agentic launches in our test automation dashboard" />
36+
<MediaViewer src={require('./img/view-launches/AgenticLaunches.png')} alt="Agentic launches in our test automation dashboard" />
3737

3838
:::note
3939
The maximum page size is 300 items per page for all collection endpoints, including Launches / Suites / Tests / Steps / Logs.

static/ai-sitemap.json

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -640,7 +640,12 @@
640640
{
641641
"title": "Reporting SDK",
642642
"url": "https://reportportal.io/docs/log-data-in-reportportal/ReportingSDK/",
643-
"description": "Integrate your test framework with ReportPortal using the Reporting SDK to log detailed test data, including nested steps and test case IDs, for enhanced reporting."
643+
"description": "Integrate your test framework with ReportPortal using the Reporting SDKs to log detailed test data, including nested steps and test case IDs, for enhanced reporting."
644+
},
645+
{
646+
"title": "Agentic test results",
647+
"url": "https://reportportal.io/docs/log-data-in-reportportal/AgenticTestResults/",
648+
"description": "Learn how to report AI agentic test results to ReportPortal to get a unified view of your testing pipeline, verify AI agent work with evidence, and make informed release decisions."
644649
},
645650
{
646651
"title": "Import data to ReportPortal",

static/llms.txt

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,8 @@
144144
- [PHP](https://reportportal.io/docs/log-data-in-reportportal/test-framework-integration/PHP/): ReportPortal integrates with popular PHP frameworks, helping teams manage test results, visualize trends, and optimize testing efficiently.
145145
- [Other](https://reportportal.io/docs/log-data-in-reportportal/test-framework-integration/Other/): Integrate unsupported or custom test frameworks by implementing your own integration with ReportPortal, enabling centralized test result visualization.
146146
- [Implement own integration](https://reportportal.io/docs/log-data-in-reportportal/ImplementOwnIntegration/): Learn how to implement custom integration with ReportPortal using the developer guide.
147-
- [Reporting SDK](https://reportportal.io/docs/log-data-in-reportportal/ReportingSDK/): Integrate your test framework with ReportPortal using the Reporting SDK to log detailed test data, including nested steps and test case IDs, for enhanced reporting.
147+
- [Reporting SDK](https://reportportal.io/docs/log-data-in-reportportal/ReportingSDK/): Integrate your test framework with ReportPortal using the Reporting SDKs to log detailed test data, including nested steps and test case IDs, for enhanced reporting.
148+
- [Agentic test results](https://reportportal.io/docs/log-data-in-reportportal/AgenticTestResults/): Learn how to report AI agentic test results to ReportPortal to get a unified view of your testing pipeline, verify AI agent work with evidence, and make informed release decisions.
148149
- [Import data to ReportPortal](https://reportportal.io/docs/log-data-in-reportportal/ImportDataToReportPortal/): Import test logs into ReportPortal using the UI or API, supporting JUnit and RobotFramework formats for streamlined test data integration.
149150
- [How to get an access token in ReportPortal](https://reportportal.io/docs/log-data-in-reportportal/HowToGetAnAccessTokenInReportPortal/): Learn how to obtain an access token in ReportPortal using your login credentials or API key, enabling secure interactions with the ReportPortal REST API.
150151
- [How to report attributes to ReportPortal](https://reportportal.io/docs/log-data-in-reportportal/HowToReportAttributesToReportPortal/): Learn how to report attributes to ReportPortal to categorize, filter, and analyze test data effectively, enhancing your test management process.

0 commit comments

Comments
 (0)