Skip to content
This repository was archived by the owner on Sep 11, 2025. It is now read-only.
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 144 additions & 2 deletions DEVELOPING.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,149 @@
# Developing `trustify-tests`

This document describes how to setup your environment to run the `trustify-tests`
on your local environment.
This document describes:

- the layout of the `trustify-tests` repository

- how to contribute a test

- how to setup your environment to run the `trustify-tests` on your local
environment

## Repository Layout

The layout of the `trustify-tests` repository looks like follows:

```
.
├── package.json
├── playwright.config.ts
├── config
├── etc
└── tests
├── api
│   ├── fixtures.ts
│   ├── client
│   ├── dependencies
│   ├── features
│   └── helpers
├── common
│   ├── constants.ts
│   └── assets
│   ├── csaf
│   └── sbom
└── ui
├── dependencies
├── features
│   ├── *.feature
│   ├── @sbom-explorer
│   └── @vulnerability-explorer
├── helpers
└── steps
```

- `package.json` - project configuration the `npm` ([Node.js Package Manager](https://docs.npmjs.com/))
understands; you can define your scripts (commands) here that you can then
execute by `npm run <your command>`

- `playwright.config.ts` - a configuration for [Playwright](https://playwright.dev/docs/intro)
and [Playwright-BDD](https://vitalets.github.io/playwright-bdd/#/)

- `config` contains configuration files that are common for the repository;
currently it contains

- `openapi.yaml` - a file with the [Trustify](https://github.com/trustification/trustify)
API definition; every time the file changes on the [Trustify](https://github.com/trustification/trustify)
side it should be also updated here
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As explained on elsewhere, atm we are not going to use the hey-api generated Sdk/Client/Types, due to overall preference of directly seeing the url/headers/params covered by api test inside the test code, instead of intermediate helpers.

As such, the is no need to keep this file updated ...
... depending on if we later find good use case for it, then it should be kept up to date (automatically), and if not then these parts can be dropped (for now since it was already implemented it can stay around until there is bigger progress and better understanding of maintaining the api tests).

Same would apply to following parts and sections too.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case you guys find this useful https://github.com/trustification/trustify/blob/main/.github/workflows/openapi.yaml

That is the Gh Action that automatically pushes the openapi.yaml file to the UI repository to automatically. Just sharing in case you would like to have a similar thing here.


- `openapi-ts.config.ts` - a configuration for `@hey-api/openapi-ts` telling
it how to generate the content of `tests/api/client`; whenever this or
`openapi.yaml` file changes `npm run openapi` should be executed to update
the content of `tests/api/client`

- `etc` contains auxiliary files such as Podman/Docker compose files to start
a Playwright container

- `tests/api` contains API tests organized as follows

- `fixtures.ts` - API tests fixtures written in TypeScript

- `client` contains a TypeScript interface to [Trustify](https://github.com/trustification/trustify)
API generated from `config/openapi.yaml` by `npm run openapi`

- `dependencies` contains setup and tear down routines which are run before
the start and after the end of the API test suite, respectively

- `features` contains API tests itself; `_openapi_client_examples.ts` shows
how to use generated TypeScript interface to [Trustify](https://github.com/trustification/trustify)
in API tests

- `helpers` contains auxiliary utilities used by API tests

- `tests/common` contains data and definitions shared by both API and UI tests

- `constants.ts` - constant definitions used by both API and UI tests

- `assets/csaf` contains compressed (`bz2`) samples of CSAF files

- `assets/sboms` contains compressed (`bz2`) samples of SBOM files

- `tests/ui` contains UI tests; UI tests are developed following BDD (Behavior
Driven Development) methodology; the directory is organized as follows

- `dependencies` contains setup and tear down routines which are run before
the start and after the end of the UI test suite, respectively

- `features` contains the UI tests itself; the content of the directory is
further organized as follows

- `*.feature` files are test scenarios described in [Gherkin](https://cucumber.io/docs/gherkin/);
`*.feature` files on the top level of the `tests/ui/features` directory
describe scenarios that need to be implemented first in the [front end](https://github.com/trustification/trustify-ui);
that is, they describe the expected front end behavior

- `@*` directories contain `*.feature` files and `*.step.ts` files used to
test the so far implemented [front end](https://github.com/trustification/trustify-ui)
features; see also [Tags from path](https://vitalets.github.io/playwright-bdd/#/writing-steps/scoped?id=tags-from-path)
documentation

- `helpers` contains auxiliary utilities used by UI tests

- `steps` contains implementation of common BDD steps used in `tests/ui/features`

## Contributing a Test

To contribute an API test, put your code under the `tests/api/features` directory.
If the test also contains a generic code that could be shared by more API tests,
put that code under the `tests/api/helpers` directory. In a case that code is
also intended to be shared by UI tests, put it under the `tests/common` directory
instead. If you have also some assets that need to be contributed together with
the test, put them under the `tests/common/assets` directory.

To contribute a UI (front end) test, put your code under the `tests/ui/features`
directory. Depending on the status of a feature your test is trying to cover,
there are two ways of how to proceed:

1. **A test is covering an implemented UI feature.** Put your test under
a `tests/ui/features/@*` directory. You can choose from the existing or create
your own depending on your use case.

1. **A test is covering a use case (scenario) not yet implemented.** Describe
your use case in [Gherkin](https://cucumber.io/docs/gherkin/) and put it
inside a `tests/ui/features/*.feature` file. The use case should be
communicated with the [upstream](https://github.com/trustification/trustify-ui)
before. Once the upstream implements the requested features covering your use
case, the next step is to put your `*.feature` file(s) under a `tests/ui/features/@*`
directory and implement missing steps to make it work under the Playwright
BDD framework.

Other directories you should be interested in when contributing a UI test:

- `tests/common` (described earlier)

- `tests/ui/helpers` follows the same rules as `tests/api/helpers`

- `tests/ui/steps` is intended to be a right place for steps that are common
across many use cases (scenarios)

## System Requirements

Expand Down
Loading