| description | A guide for generating Trunk-compatible test reports with other test frameworks |
|---|
Trunk Flaky Tests is designed to be test framework agnostic. If you don't see a guide for your test framework, you can still use Flaky Tests. To use Flaky Tests, you will need to report test results in a format that Trunk understands and upload them to Trunk in CI. This guide will explain how to integrate your test framework with Trunk.
Trunk detects flaky tests by analyzing each test case's results over time. Trunk currently supports the JUnit XML and XCResult report formats. You will need to configure your test runner to report in one of these formats using a plugin or your own test result reporter.
Make sure your test reports accurately report the file name, test name, and stack trace of each test result. Make sure the test names are not randomized. These details help Trunk better detect and display your test cases' health status.
You'll need to validate and upload the generated JUnit files to Trunk later during the setup process. Make sure the reports are generated with a consistent name and aren't cached or committed to Git.
Since you'll be generating JUnit reports using a new plugin or custom reporter, you should use the Trunk Analytics CLI to validate your results and fix any warnings or errors.
You can install the Trunk Analytics CLI locally like this:
{% tabs %} {% tab title="Linux (x64)" %}
SKU="trunk-analytics-cli-x86_64-unknown-linux.tar.gz"
curl -fL --retry 3 \
"https://github.com/trunk-io/analytics-cli/releases/latest/download/${SKU}" \
| tar -xz
chmod +x trunk-analytics-cli{% endtab %}
{% tab title="Linux (arm64)" %}
SKU="trunk-analytics-cli-aarch64-unknown-linux.tar.gz"
curl -fL --retry 3 \
"https://github.com/trunk-io/analytics-cli/releases/latest/download/${SKU}" \
| tar -xz
chmod +x trunk-analytics-cli{% endtab %}
{% tab title="macOS (arm64)" %}
SKU="trunk-analytics-cli-aarch64-apple-darwin.tar.gz"
curl -fL --retry 3 \
"https://github.com/trunk-io/analytics-cli/releases/latest/download/${SKU}" \
| tar -xz
chmod +x trunk-analytics-cli{% endtab %}
{% tab title="macOS (x64)" %}
SKU="trunk-analytics-cli-x86_64-apple-darwin.tar.gz"
curl -fL --retry 3 \
"https://github.com/trunk-io/analytics-cli/releases/latest/download/${SKU}" \
| tar -xz
chmod +x trunk-analytics-cli{% endtab %} {% endtabs %}
Then, you can validate the results using the trunk-analytics-cli validate command like this:
./trunk-analytics-cli validate --junit-paths <PATH_TO_REPORTS>You'll need to upload the JUnit reports generated by your CI jobs to Trunk so Trunk can detect flaky tests and report them to the dashboard. See CI Providers for a guide on how to upload test results to Trunk.
| Azure DevOps Pipelines | azure-devops-pipelines.md | azure.png | |
| BitBucket Pipelines | bitbucket-pipelines.md | bitbucket.png | |
| BuildKite | buildkite.md | buildkite.png | |
| CircleCI | circleci.md | circle-ci.png | |
| Drone CI | droneci.md | drone.png | |
| GitHub Actions | github-actions.md | github.png | |
| GitLab | gitlab.md | gitlab.png | |
| Jenkins | jenkins.md | jenkins.png | |
| Semaphore | semaphoreci.md | semaphore.png | |
| TeamCity | broken-reference | teamcity.png | |
| Travis CI | travisci.md | travis.png | |
| Other CI Providers | otherci.md | other.png |
