Skip to content

Commit 8a744b8

Browse files
simonmegglesmcmk
andauthored
docs: Updated README, DEVGUIDE, CONTRIBUTION (#23)
Co-authored-by: Simon Meggle <simon.meggle@checkmk.com>
1 parent 681ab40 commit 8a744b8

7 files changed

Lines changed: 524 additions & 165 deletions

File tree

CONTRIBUTION.md

Whitespace-only changes.

DEVGUIDE.md

Lines changed: 52 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,55 @@
1-
## Oxygen developer guide
1+
# Oxygen developer guide
2+
3+
WIP - the following section was moved from the README:
4+
5+
---
6+
7+
## Handlers & Configuration
8+
9+
Handlers are registered in `config.yml` inside the package. Inspect or modify the file via the CLI commands above. A typical entry looks like:
10+
11+
```yaml
12+
rmkbridge.junit:
13+
handler: JUnitHandler
14+
keyword: run_junit
15+
tags:
16+
- rmkbridge-junit
17+
```
18+
19+
- The dictionary key (`rmkbridge.junit`) must be importable (`import rmkbridge.junit`).
20+
- `handler` names the class instantiated from that module.
21+
- `keyword` becomes the Robot Framework keyword (`Run Junit`, `run_junit`, etc.).
22+
- `tags` are applied to every injected test case.
23+
- Handlers may define extra settings (for example, the ZAP handler ships `accepted_risk_level`).
24+
25+
### Creating Custom Handlers
26+
27+
1. Extend `rmkbridge.BaseHandler` and implement `parse_results` plus your trigger keyword.
28+
2. Make your module discoverable (PYTHONPATH or installable package).
29+
3. Append your configuration with `python -m rmkbridge --add-config path/to/handler_config.yml`.
30+
4. Conform to the [handler result specification](handler_result_specification.md) when returning parsed suites.
31+
32+
The [developer guide](DEVGUIDE.md) walks through a complete example.
33+
34+
## Developing
35+
36+
Clone the repository and install the development dependencies:
37+
38+
```bash
39+
pip install -r requirements.txt
40+
pip install -e .
41+
```
42+
43+
Robotmk Bridge uses [`invoke`](https://www.pyinvoke.org/) for common tasks:
44+
45+
```bash
46+
invoke --list
47+
invoke test
48+
```
49+
50+
You can also run Robot Framework acceptance suites under `tests/` to validate changes end-to-end.
51+
---
52+
253

354
This is a developer guide for Oxygen. We will write a handler for [https://locust.io/](https://locust.io/), which is a performance testing tool.
455

README.md

Lines changed: 91 additions & 163 deletions
Original file line numberDiff line numberDiff line change
@@ -1,223 +1,151 @@
1-
# Oxygen
1+
# Robotmk Bridge
22

3+
<<<<<<< Updated upstream
34
Version: 0.1.0 <!-- x-release-please-version -->
5+
=======
6+
_**The bridge between automation islands and your monitoring**_
7+
>>>>>>> Stashed changes
48
5-
Oxygen is a [Robot Framework](https://robotframework.org/) tool that empowers the user to convert the results of any testing tool or framework to [Robot Framework's reporting](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#screenshots). This consolidates all test reporting together regardless of tools used.
9+
Robotmk Bridge is a [Robot Framework](https://robotframework.org/) library, listener, and CLI that for external test tools to convert their results into Robot Framework results.
610

7-
Oxygen has built-in support for three testing frameworks: [JUnit](https://junit.org/junit5/), [Gatling](https://gatling.io/), and [Zed Attack Proxy (ZAP)](https://www.zaproxy.org/).
11+
It is used in two modes:
812

9-
Oxygen is designed to be extensible. Users can create their own *handlers* for other testing framework or tools to transform their reporting into the Robot Framework's `log.html` and `report.html`.
13+
- In the [Robotmk Bridge Agent Plugin](https://github.com/elabit/robotmk-bridge-plugin) which periodically writes converted results into the [Robotmk](https://robotmk.org) results spool folder.
14+
- As a **Robot Framework Library** to run external test tools from Robot Framework tests.
1015

11-
# Table of Contents
12-
1. [Installation](#installation)
13-
1. [Keyword documentation](#keyword-documentation)
14-
1. [Usage](#usage)
15-
1. [Developing Oxygen](#developing-oxygen)
16-
1. [License](#license)
17-
1. [Acknowledgements](#acknowledgments)
16+
In this way, **any test data** can be imported into [Checkmk](https://checkmk.com) monitoring with the help of Robotmk.
1817

19-
# Installation
2018

21-
To install Oxygen, run the following:
22-
```
23-
$ pip install robotframework-oxygen
24-
```
25-
26-
## Pre-requisites
27-
28-
- Oxygen is supported on Windows, Linux and MacOS
29-
- [Python 3.10](http://python.org) or above
30-
- [pip](https://pypi.python.org/pypi/pip) for easy installation
31-
- [Robot Framework](http://robotframework.org)
32-
- [additional dependencies](requirements.txt)
33-
34-
To check the Python version on the command line, run:
35-
```
36-
$ python --version
37-
```
38-
39-
# Keyword documentation
40-
41-
[Keyword Documentation](http://eficode.github.io/robotframework-oxygen/)
42-
43-
# Usage
44-
45-
## Example: Robot Framework running other test tools
46-
47-
Main usage scenario for Oxygen is the ability to write acceptance test cases that run your tests in other test tools and integrate the resulting test report as part of Robot Framework's. This means you are able to run all of your testing from Robot Framework and thus having all test reporting consolidated together.
4819

49-
After installing Oxygen, it can be used in the Robot Framework suite to write test cases. For example, to build acceptance tests that run different sets of JUnit tests:
20+
![](img/architecture.png)
5021

51-
``` RobotFramework
52-
*** Settings ***
53-
Library rmkbridge.RobotmkBridgeLibrary
54-
55-
*** Test cases ***
22+
## Features
5623

57-
JUnit unit tests should pass
58-
[Tags] testset-1
59-
Run JUnit path/to/mydir/results.xml java -jar junit.jar --reports-dir=path/to/mydir
60-
61-
JUnit integration tests should pass
62-
[Tags] testset-2
63-
Run JUnit path/to/anotherdir/results.xml java -jar junit.jar --reports-dir=path/to/anotherdir
64-
```
24+
- Unifies third-party test results inside Robot Framework log.html/report.html and the Robotmk Checkmk plug-in.
25+
- Ships ready-made **handlers**, currently there is support for
26+
- [JUnit](https://junit.org/junit5/)
27+
- [Gatling](https://gatling.io/)
28+
- [OWASP ZAP](https://www.zaproxy.org/)
29+
- Provides the dynamic library `rmkbridge.RobotmkBridgeLibrary` plus the listener `rmkbridge.listener` for Robot test suites.
30+
- Offers a CLI (`python -m rmkbridge`) to transform standalone result files into Robot Framework output.xml artifacts.
31+
- Lets you implement custom handlers by extending `rmkbridge.BaseHandler`.
6532

66-
Then, run the suite by providing Oxygen as [a listener](http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#listener-interface):
33+
## Installation
6734

35+
```bash
36+
pip install robotframework-robotmk-bridge
6837
```
69-
$ robot --listener oxygen.listener my_tests.robot
70-
```
71-
72-
Opening the Robot Framework `log.html` and `report.html`, you should see that test case `JUnit unt tests should pass` has been replaced by Oxygen with test cases matching with what is in the `path/to/mydir/results.xml` JUnit report file. Similarly, test case `JUnit integration tests should pass` has been replaced with results from `path/to/anotherdir/results.xml`; each JUnit test case with its relevant information has a counterpart in the `log.html`. Each JUnit test case is also tagged with the tags from the original Robot Framework test case.
7338

74-
The example above, for the brevity, shows incomplete commands to run JUnit tool from command line. Please refer to [keyword documentation](#keyword-documentation) for more detailed documentation about keyword's arguments, as well as documentation for [Gatling](https://gatling.io/) and [ZAP](https://www.zaproxy.org/) related keywords. And, of course, refer to the particular tool documentation as well.
39+
## Prerequisites
7540

76-
## Using from command line
41+
- Windows, Linux, or macOS
42+
- [Python 3.10+](https://www.python.org/downloads/)
43+
- [Robot Framework 6.x](https://robotframework.org) (Robot Framework 7+ support is planned)
44+
- [pip](https://pip.pypa.io/) and any extra [requirements](requirements.txt) your handlers need
7745

78-
In case where you want to run your other testing tools separately, but yet combine results into unified Robot Framework `log.html` and `report.html`, you can use Oxygen's command line interface to convert single result file to single corresponding Robot Framework `output.xml`:
46+
To verify the installation:
7947

80-
```
81-
$ python -m oxygen oxygen.junit my_junit_results.xml
48+
```bash
49+
python -m rmkbridge --version
8250
```
8351

84-
As a convention, the resulting Robot Framework xml file will be named by adding a suffix to the end. In the example above, the resulting Robot Framework xml file would be named `my_junit_results_robot_output.xml`.
8552

86-
**Note** that resulting xml file will also be created at the same location as the original result file. Therefore, when original result files are in another directory:
53+
## Quickstart
8754

88-
```
89-
$ python -m oxygen oxygen.gatling path/to/results.log
90-
```
55+
### Option 1: Use as a Library to execute Test tools from Robot Framework
56+
57+
This mode consists of two steps:
9158

92-
Then `results_robot_output.xml` will be created under `path/to/`.
59+
1. Running the tool in Robot Framework
60+
2. Running the Bridge-Listener
9361

94-
## Extending Oxygen: writing your own handler
62+
#### Step 1: Running the tool in Robot Framework
9563

96-
### [Read the developer guide on how to write your own handler](DEVGUIDE.md)
64+
Each supported external test tool comes with a special **trigger keyword** `Run <tool>` to run the tool from inside Robot Framework.
65+
Depending on the Handler, the keywords support individual arguments.
9766

98-
You might also want to look at [specification for handler results](handler_result_specification.md)
67+
```robotframework
68+
*** Settings ***
69+
Library rmkbridge.RobotmkBridgeLibrary
9970
100-
### Configuring your handler to Oxygen
71+
*** Test Cases ***
72+
JUnit unit tests should pass
73+
Custom Keyword 1
74+
Run JUnit path/to/results.xml java -jar junit.jar --reports-dir path/to
75+
Custom Keyword 2
10176
102-
Oxygen knows about different handlers based on the [`config.yml`](https://github.com/eficode/robotframework-oxygen/blob/master/config.yml) file. This configuration file can be interacted with through Oxygen's command line.
77+
Gatling regression should stay green
78+
Custom Keyword 1
79+
Run Gatling path/to/gatling.log ${GATLING_HOME}/bin/gatling.sh --simulation MySimulation
80+
Custom Keyword 2
10381
104-
The configuration has the following parts:
105-
```yml
106-
oxygen.junit: # Python module. Oxygen will use this key to try to import the handler
107-
handler: JUnitHandler # Class that Oxygen will initiate after the handler is imported
108-
keyword: run_junit # Keyword that should be used to run the other test tool
109-
tags: # List of tags that by default should be added to the test cases converted with this handler
110-
- oxygen-junit
111-
oxygen.zap:
112-
handler: ZAProxyHandler
113-
keyword: run_zap
114-
tags: oxygen-zap
115-
accepted_risk_level: 2 # Handlers can have their own command line arguments
116-
required_confidence_level: 1 # See https://github.com/eficode/robotframework-oxygen/blob/master/DEVGUIDE.md for more information
82+
ZAP scan finds no blockers
83+
Custom Keyword 1
84+
Run Zap path/to/zap.json python zap_scan.py
85+
Custom Keyword 2
11786
```
11887

119-
#### `--add-config`
88+
#### Step 2: Running the Listener
12089

121-
This argument is used to add new handler configuration to Oxygen:
90+
Now execute the suite with the Robotmk Bridge listener so the external reports are injected into the output:
12291

12392
```bash
124-
$ python -m oxygen --add-config path/to/your_handler_config.yml
93+
robot --listener rmkbridge.listener tests/my_suite.robot
12594
```
12695

127-
This file is read and appended to the Oxygen's `config.yml`. Based on the key, Oxygen will try to import you handler.
128-
129-
### `--reset-config`
96+
Robotmk Bridge creates Test Results using the following rules:
13097

131-
This argument is used to return Oxygen's `config.yml` back to the state it was when the tool was installed:
98+
- **trigger keywords** (which run the tools) become **Test Cases**.
99+
- Keywords _before_ the trigger keyword are wrapped into a **Test Setup keyword**.
100+
- Keywords _before_ the trigger keyword are wrapped into a **Test Teardown keyword**.
132101

133-
```bash
134-
$ python -m oxygen --reset-config
135-
```
136-
137-
The command **does not** verify the operation from the user, so be careful.
102+
## Option 2: Command Line Usage to convert existing results
138103

139-
### `--print-config`
104+
Use the CLI when you need to convert tool reports without running Robot Framework suites:
140105

141-
This argument prints the current configuration of Oxygen:
142106
```bash
143-
$ python -m oxygen --print-config
144-
Using config file: /path/to/oxygen/src/oxygen/config.yml
145-
oxygen.gatling:
146-
handler: GatlingHandler
147-
keyword: run_gatling
148-
tags: oxygen-gatling
149-
oxygen.junit:
150-
handler: JUnitHandler
151-
keyword: run_junit
152-
tags:
153-
- oxygen-junit
154-
oxygen.zap:
155-
accepted_risk_level: 2
156-
handler: ZAProxyHandler
157-
keyword: run_zap
158-
required_confidence_level: 1
159-
tags: oxygen-zap
160-
161-
$
107+
python -m rmkbridge rmkbridge.junit --result-file path/to/results.xml
162108
```
163-
Because you can add the configuration to the same handler multiple times, note that only the last entry is in effect.
164-
165-
## `utils` module
166-
167-
In [utils module](https://github.com/eficode/robotframework-oxygen/blob/master/src/oxygen/utils.py), you will find assortment of functionalities that you might want to leverage when writing your own handler.
168-
169-
### `run_command_line()`
170109

171-
Most of the time, handlers want to run the other test tool through command line. For this, `utils` provides `run_command_line()` that wraps Python's [`subprocess`](https://docs.python.org/3/library/subprocess.html) module for more easier to use when writing your handler.
110+
- The converted file gets created next to the source as `*_robot_output.xml`.
111+
- Similar to trigger keywords, each handler also exposes its own CLI flags. List them with `python -m rmkbridge rmkbridge.junit --help`.
112+
- Global switches:
113+
- `python -m rmkbridge --print-config`
114+
- `python -m rmkbridge --add-config path/to/custom_handler.yml`
115+
- `python -m rmkbridge --reset-config`
172116

173-
`run_command_line()` takes following arguments:
174-
- `cmd`: the command to be executed in a subprocess
175-
- `check_return_code`: if set to `True`, will raise an exception if the `cmd` fails in the subprocess. **Note** that this fails the keyword and, thus, the execution of the test case is stopped. If you want to enable test case to continue even after `run_command_line()` has failed, you should disable it by setting `False`. It is often a good idea to allow user using your handler's keyword to decide how they want the command line execution to affect the test case
176-
- `env`: a dictionary of environment variables that should be passed to the subprocess. By default, `run_command_line()` inherits the environment from the current Python process as well as from modifications done by the Robot Framework command line arguments (ie. `--pythonpath`)
177117

178-
# Developing Oxygen
179118

180-
## Setup
119+
## Keyword Documentation
181120

182-
Clone the Oxygen repository to the environment where you want to the run the tool.
121+
- [Open the generated keyword reference](docs/index.html)
122+
- Regenerate locally when you add handlers:
183123

184-
Oxygen requires a set of dependencies to be installed. Dependencies are listed in the `requirements.txt` file:
185-
```
186-
$ pip install -r requirements.txt
187-
```
188-
189-
Also install the package itself as editable:
190-
191-
```
192-
$ pip install -e .
193-
```
124+
```bash
125+
python -m robot.libdoc rmkbridge.RobotmkBridgeLibrary docs/RobotmkBridgeLibrary-$(python -c "import rmkbridge; print(rmkbridge.VERSION)").html
126+
```
194127

195-
## Tasks
128+
## 🤝 Contribute Your Own Handlers!
196129

197-
Oxygen uses task runner tool [`invoke`](http://www.pyinvoke.org/) to run tests, build the project, etc.
130+
The Robotmk Bridge is an open-source project — and we’d love to see it grow with the help of the community!
131+
Our goal is to make Robotmk a truly multi-purpose integration layer for all kinds of test results.
198132

199-
Please refer to the available tasks for the project:
200-
```
201-
$ invoke --list
202-
```
203-
204-
and the task file [`tasks.py`](https://github.com/eficode/robotframework-oxygen/blob/master/tasks.py).
133+
If you’re working with a testing tool that isn’t supported yet, consider developing your own Bridge Handler and sharing it with others.
134+
Every new handler expands what Robotmk can do and helps bring monitoring and test automation even closer together.
205135

206-
### Tests
207-
208-
```
209-
$ invoke test
210-
```
136+
Pull requests, discussions, and ideas are always welcome!
211137

212-
(Metadata test in Test explorer will stay red)
138+
Read more:
213139

140+
- [How to write your own Handler in Python](./DEVGUIDE.md)
141+
- [How to contribute ot the project](./CONTRIBUTION.md)
214142

215143

216-
# License
144+
## License & Acknowledgements
217145

218-
Details of project licensing can be found in the [LICENSE](LICENSE) file in the project repository.
219146

220-
# Acknowledgments
147+
Special thanks to Eficdoe Oy, Finland. RobotmkBridge is based on their project [robotframework-oxygen](https://github.com/eficode/robotframework-oxygen?tab=readme-ov-file#developing-oxygen).
221148

222-
See [ACKNOWLEDGEMENTS](./ACKNOWLEDGEMENTS.md) for more information.
149+
See [ACKNOWLEDGEMENTS.md](ACKNOWLEDGEMENTS.md) for the roots of the project and credits.
150+
Released under the [MIT License](LICENSE).
223151

0 commit comments

Comments
 (0)