|
1 | | -# Oxygen |
| 1 | +# Robotmk Bridge |
2 | 2 |
|
| 3 | +<<<<<<< Updated upstream |
3 | 4 | Version: 0.1.0 <!-- x-release-please-version --> |
| 5 | +======= |
| 6 | +_**The bridge between automation islands and your monitoring**_ |
| 7 | +>>>>>>> Stashed changes |
4 | 8 |
|
5 | | -Oxygen is a [Robot Framework](https://robotframework.org/) tool that empowers the user to convert the results of any testing tool or framework to [Robot Framework's reporting](https://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#screenshots). This consolidates all test reporting together regardless of tools used. |
| 9 | +Robotmk Bridge is a [Robot Framework](https://robotframework.org/) library, listener, and CLI that for external test tools to convert their results into Robot Framework results. |
6 | 10 |
|
7 | | -Oxygen has built-in support for three testing frameworks: [JUnit](https://junit.org/junit5/), [Gatling](https://gatling.io/), and [Zed Attack Proxy (ZAP)](https://www.zaproxy.org/). |
| 11 | +It is used in two modes: |
8 | 12 |
|
9 | | -Oxygen is designed to be extensible. Users can create their own *handlers* for other testing framework or tools to transform their reporting into the Robot Framework's `log.html` and `report.html`. |
| 13 | +- In the [Robotmk Bridge Agent Plugin](https://github.com/elabit/robotmk-bridge-plugin) which periodically writes converted results into the [Robotmk](https://robotmk.org) results spool folder. |
| 14 | +- As a **Robot Framework Library** to run external test tools from Robot Framework tests. |
10 | 15 |
|
11 | | -# Table of Contents |
12 | | -1. [Installation](#installation) |
13 | | -1. [Keyword documentation](#keyword-documentation) |
14 | | -1. [Usage](#usage) |
15 | | -1. [Developing Oxygen](#developing-oxygen) |
16 | | -1. [License](#license) |
17 | | -1. [Acknowledgements](#acknowledgments) |
| 16 | +In this way, **any test data** can be imported into [Checkmk](https://checkmk.com) monitoring with the help of Robotmk. |
18 | 17 |
|
19 | | -# Installation |
20 | 18 |
|
21 | | -To install Oxygen, run the following: |
22 | | -``` |
23 | | -$ pip install robotframework-oxygen |
24 | | -``` |
25 | | - |
26 | | -## Pre-requisites |
27 | | - |
28 | | -- Oxygen is supported on Windows, Linux and MacOS |
29 | | -- [Python 3.10](http://python.org) or above |
30 | | -- [pip](https://pypi.python.org/pypi/pip) for easy installation |
31 | | -- [Robot Framework](http://robotframework.org) |
32 | | -- [additional dependencies](requirements.txt) |
33 | | - |
34 | | -To check the Python version on the command line, run: |
35 | | -``` |
36 | | -$ python --version |
37 | | -``` |
38 | | - |
39 | | -# Keyword documentation |
40 | | - |
41 | | -[Keyword Documentation](http://eficode.github.io/robotframework-oxygen/) |
42 | | - |
43 | | -# Usage |
44 | | - |
45 | | -## Example: Robot Framework running other test tools |
46 | | - |
47 | | -Main usage scenario for Oxygen is the ability to write acceptance test cases that run your tests in other test tools and integrate the resulting test report as part of Robot Framework's. This means you are able to run all of your testing from Robot Framework and thus having all test reporting consolidated together. |
48 | 19 |
|
49 | | -After installing Oxygen, it can be used in the Robot Framework suite to write test cases. For example, to build acceptance tests that run different sets of JUnit tests: |
| 20 | + |
50 | 21 |
|
51 | | -``` RobotFramework |
52 | | -*** Settings *** |
53 | | -Library rmkbridge.RobotmkBridgeLibrary |
54 | | -
|
55 | | -*** Test cases *** |
| 22 | +## Features |
56 | 23 |
|
57 | | -JUnit unit tests should pass |
58 | | - [Tags] testset-1 |
59 | | - Run JUnit path/to/mydir/results.xml java -jar junit.jar --reports-dir=path/to/mydir |
60 | | -
|
61 | | -JUnit integration tests should pass |
62 | | - [Tags] testset-2 |
63 | | - Run JUnit path/to/anotherdir/results.xml java -jar junit.jar --reports-dir=path/to/anotherdir |
64 | | -``` |
| 24 | +- Unifies third-party test results inside Robot Framework log.html/report.html and the Robotmk Checkmk plug-in. |
| 25 | +- Ships ready-made **handlers**, currently there is support for |
| 26 | + - [JUnit](https://junit.org/junit5/) |
| 27 | + - [Gatling](https://gatling.io/) |
| 28 | + - [OWASP ZAP](https://www.zaproxy.org/) |
| 29 | +- Provides the dynamic library `rmkbridge.RobotmkBridgeLibrary` plus the listener `rmkbridge.listener` for Robot test suites. |
| 30 | +- Offers a CLI (`python -m rmkbridge`) to transform standalone result files into Robot Framework output.xml artifacts. |
| 31 | +- Lets you implement custom handlers by extending `rmkbridge.BaseHandler`. |
65 | 32 |
|
66 | | -Then, run the suite by providing Oxygen as [a listener](http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#listener-interface): |
| 33 | +## Installation |
67 | 34 |
|
| 35 | +```bash |
| 36 | +pip install robotframework-robotmk-bridge |
68 | 37 | ``` |
69 | | -$ robot --listener oxygen.listener my_tests.robot |
70 | | -``` |
71 | | - |
72 | | -Opening the Robot Framework `log.html` and `report.html`, you should see that test case `JUnit unt tests should pass` has been replaced by Oxygen with test cases matching with what is in the `path/to/mydir/results.xml` JUnit report file. Similarly, test case `JUnit integration tests should pass` has been replaced with results from `path/to/anotherdir/results.xml`; each JUnit test case with its relevant information has a counterpart in the `log.html`. Each JUnit test case is also tagged with the tags from the original Robot Framework test case. |
73 | 38 |
|
74 | | -The example above, for the brevity, shows incomplete commands to run JUnit tool from command line. Please refer to [keyword documentation](#keyword-documentation) for more detailed documentation about keyword's arguments, as well as documentation for [Gatling](https://gatling.io/) and [ZAP](https://www.zaproxy.org/) related keywords. And, of course, refer to the particular tool documentation as well. |
| 39 | +## Prerequisites |
75 | 40 |
|
76 | | -## Using from command line |
| 41 | +- Windows, Linux, or macOS |
| 42 | +- [Python 3.10+](https://www.python.org/downloads/) |
| 43 | +- [Robot Framework 6.x](https://robotframework.org) (Robot Framework 7+ support is planned) |
| 44 | +- [pip](https://pip.pypa.io/) and any extra [requirements](requirements.txt) your handlers need |
77 | 45 |
|
78 | | -In case where you want to run your other testing tools separately, but yet combine results into unified Robot Framework `log.html` and `report.html`, you can use Oxygen's command line interface to convert single result file to single corresponding Robot Framework `output.xml`: |
| 46 | +To verify the installation: |
79 | 47 |
|
80 | | -``` |
81 | | -$ python -m oxygen oxygen.junit my_junit_results.xml |
| 48 | +```bash |
| 49 | +python -m rmkbridge --version |
82 | 50 | ``` |
83 | 51 |
|
84 | | -As a convention, the resulting Robot Framework xml file will be named by adding a suffix to the end. In the example above, the resulting Robot Framework xml file would be named `my_junit_results_robot_output.xml`. |
85 | 52 |
|
86 | | -**Note** that resulting xml file will also be created at the same location as the original result file. Therefore, when original result files are in another directory: |
| 53 | +## Quickstart |
87 | 54 |
|
88 | | -``` |
89 | | -$ python -m oxygen oxygen.gatling path/to/results.log |
90 | | -``` |
| 55 | +### Option 1: Use as a Library to execute Test tools from Robot Framework |
| 56 | + |
| 57 | +This mode consists of two steps: |
91 | 58 |
|
92 | | -Then `results_robot_output.xml` will be created under `path/to/`. |
| 59 | +1. Running the tool in Robot Framework |
| 60 | +2. Running the Bridge-Listener |
93 | 61 |
|
94 | | -## Extending Oxygen: writing your own handler |
| 62 | +#### Step 1: Running the tool in Robot Framework |
95 | 63 |
|
96 | | -### [Read the developer guide on how to write your own handler](DEVGUIDE.md) |
| 64 | +Each supported external test tool comes with a special **trigger keyword** `Run <tool>` to run the tool from inside Robot Framework. |
| 65 | +Depending on the Handler, the keywords support individual arguments. |
97 | 66 |
|
98 | | -You might also want to look at [specification for handler results](handler_result_specification.md) |
| 67 | +```robotframework |
| 68 | +*** Settings *** |
| 69 | +Library rmkbridge.RobotmkBridgeLibrary |
99 | 70 |
|
100 | | -### Configuring your handler to Oxygen |
| 71 | +*** Test Cases *** |
| 72 | +JUnit unit tests should pass |
| 73 | + Custom Keyword 1 |
| 74 | + Run JUnit path/to/results.xml java -jar junit.jar --reports-dir path/to |
| 75 | + Custom Keyword 2 |
101 | 76 |
|
102 | | -Oxygen knows about different handlers based on the [`config.yml`](https://github.com/eficode/robotframework-oxygen/blob/master/config.yml) file. This configuration file can be interacted with through Oxygen's command line. |
| 77 | +Gatling regression should stay green |
| 78 | + Custom Keyword 1 |
| 79 | + Run Gatling path/to/gatling.log ${GATLING_HOME}/bin/gatling.sh --simulation MySimulation |
| 80 | + Custom Keyword 2 |
103 | 81 |
|
104 | | -The configuration has the following parts: |
105 | | -```yml |
106 | | -oxygen.junit: # Python module. Oxygen will use this key to try to import the handler |
107 | | - handler: JUnitHandler # Class that Oxygen will initiate after the handler is imported |
108 | | - keyword: run_junit # Keyword that should be used to run the other test tool |
109 | | - tags: # List of tags that by default should be added to the test cases converted with this handler |
110 | | - - oxygen-junit |
111 | | -oxygen.zap: |
112 | | - handler: ZAProxyHandler |
113 | | - keyword: run_zap |
114 | | - tags: oxygen-zap |
115 | | - accepted_risk_level: 2 # Handlers can have their own command line arguments |
116 | | - required_confidence_level: 1 # See https://github.com/eficode/robotframework-oxygen/blob/master/DEVGUIDE.md for more information |
| 82 | +ZAP scan finds no blockers |
| 83 | + Custom Keyword 1 |
| 84 | + Run Zap path/to/zap.json python zap_scan.py |
| 85 | + Custom Keyword 2 |
117 | 86 | ``` |
118 | 87 |
|
119 | | -#### `--add-config` |
| 88 | +#### Step 2: Running the Listener |
120 | 89 |
|
121 | | -This argument is used to add new handler configuration to Oxygen: |
| 90 | +Now execute the suite with the Robotmk Bridge listener so the external reports are injected into the output: |
122 | 91 |
|
123 | 92 | ```bash |
124 | | -$ python -m oxygen --add-config path/to/your_handler_config.yml |
| 93 | +robot --listener rmkbridge.listener tests/my_suite.robot |
125 | 94 | ``` |
126 | 95 |
|
127 | | -This file is read and appended to the Oxygen's `config.yml`. Based on the key, Oxygen will try to import you handler. |
128 | | - |
129 | | -### `--reset-config` |
| 96 | +Robotmk Bridge creates Test Results using the following rules: |
130 | 97 |
|
131 | | -This argument is used to return Oxygen's `config.yml` back to the state it was when the tool was installed: |
| 98 | +- **trigger keywords** (which run the tools) become **Test Cases**. |
| 99 | +- Keywords _before_ the trigger keyword are wrapped into a **Test Setup keyword**. |
| 100 | +- Keywords _before_ the trigger keyword are wrapped into a **Test Teardown keyword**. |
132 | 101 |
|
133 | | -```bash |
134 | | -$ python -m oxygen --reset-config |
135 | | -``` |
136 | | - |
137 | | -The command **does not** verify the operation from the user, so be careful. |
| 102 | +## Option 2: Command Line Usage to convert existing results |
138 | 103 |
|
139 | | -### `--print-config` |
| 104 | +Use the CLI when you need to convert tool reports without running Robot Framework suites: |
140 | 105 |
|
141 | | -This argument prints the current configuration of Oxygen: |
142 | 106 | ```bash |
143 | | -$ python -m oxygen --print-config |
144 | | -Using config file: /path/to/oxygen/src/oxygen/config.yml |
145 | | -oxygen.gatling: |
146 | | - handler: GatlingHandler |
147 | | - keyword: run_gatling |
148 | | - tags: oxygen-gatling |
149 | | -oxygen.junit: |
150 | | - handler: JUnitHandler |
151 | | - keyword: run_junit |
152 | | - tags: |
153 | | - - oxygen-junit |
154 | | -oxygen.zap: |
155 | | - accepted_risk_level: 2 |
156 | | - handler: ZAProxyHandler |
157 | | - keyword: run_zap |
158 | | - required_confidence_level: 1 |
159 | | - tags: oxygen-zap |
160 | | -
|
161 | | -$ |
| 107 | +python -m rmkbridge rmkbridge.junit --result-file path/to/results.xml |
162 | 108 | ``` |
163 | | -Because you can add the configuration to the same handler multiple times, note that only the last entry is in effect. |
164 | | - |
165 | | -## `utils` module |
166 | | - |
167 | | -In [utils module](https://github.com/eficode/robotframework-oxygen/blob/master/src/oxygen/utils.py), you will find assortment of functionalities that you might want to leverage when writing your own handler. |
168 | | - |
169 | | -### `run_command_line()` |
170 | 109 |
|
171 | | -Most of the time, handlers want to run the other test tool through command line. For this, `utils` provides `run_command_line()` that wraps Python's [`subprocess`](https://docs.python.org/3/library/subprocess.html) module for more easier to use when writing your handler. |
| 110 | +- The converted file gets created next to the source as `*_robot_output.xml`. |
| 111 | +- Similar to trigger keywords, each handler also exposes its own CLI flags. List them with `python -m rmkbridge rmkbridge.junit --help`. |
| 112 | +- Global switches: |
| 113 | + - `python -m rmkbridge --print-config` |
| 114 | + - `python -m rmkbridge --add-config path/to/custom_handler.yml` |
| 115 | + - `python -m rmkbridge --reset-config` |
172 | 116 |
|
173 | | -`run_command_line()` takes following arguments: |
174 | | -- `cmd`: the command to be executed in a subprocess |
175 | | -- `check_return_code`: if set to `True`, will raise an exception if the `cmd` fails in the subprocess. **Note** that this fails the keyword and, thus, the execution of the test case is stopped. If you want to enable test case to continue even after `run_command_line()` has failed, you should disable it by setting `False`. It is often a good idea to allow user using your handler's keyword to decide how they want the command line execution to affect the test case |
176 | | -- `env`: a dictionary of environment variables that should be passed to the subprocess. By default, `run_command_line()` inherits the environment from the current Python process as well as from modifications done by the Robot Framework command line arguments (ie. `--pythonpath`) |
177 | 117 |
|
178 | | -# Developing Oxygen |
179 | 118 |
|
180 | | -## Setup |
| 119 | +## Keyword Documentation |
181 | 120 |
|
182 | | -Clone the Oxygen repository to the environment where you want to the run the tool. |
| 121 | +- [Open the generated keyword reference](docs/index.html) |
| 122 | +- Regenerate locally when you add handlers: |
183 | 123 |
|
184 | | -Oxygen requires a set of dependencies to be installed. Dependencies are listed in the `requirements.txt` file: |
185 | | -``` |
186 | | -$ pip install -r requirements.txt |
187 | | -``` |
188 | | - |
189 | | -Also install the package itself as editable: |
190 | | - |
191 | | -``` |
192 | | -$ pip install -e . |
193 | | -``` |
| 124 | + ```bash |
| 125 | + python -m robot.libdoc rmkbridge.RobotmkBridgeLibrary docs/RobotmkBridgeLibrary-$(python -c "import rmkbridge; print(rmkbridge.VERSION)").html |
| 126 | + ``` |
194 | 127 |
|
195 | | -## Tasks |
| 128 | +## 🤝 Contribute Your Own Handlers! |
196 | 129 |
|
197 | | -Oxygen uses task runner tool [`invoke`](http://www.pyinvoke.org/) to run tests, build the project, etc. |
| 130 | +The Robotmk Bridge is an open-source project — and we’d love to see it grow with the help of the community! |
| 131 | +Our goal is to make Robotmk a truly multi-purpose integration layer for all kinds of test results. |
198 | 132 |
|
199 | | -Please refer to the available tasks for the project: |
200 | | -``` |
201 | | -$ invoke --list |
202 | | -``` |
203 | | - |
204 | | -and the task file [`tasks.py`](https://github.com/eficode/robotframework-oxygen/blob/master/tasks.py). |
| 133 | +If you’re working with a testing tool that isn’t supported yet, consider developing your own Bridge Handler and sharing it with others. |
| 134 | +Every new handler expands what Robotmk can do and helps bring monitoring and test automation even closer together. |
205 | 135 |
|
206 | | -### Tests |
207 | | - |
208 | | -``` |
209 | | -$ invoke test |
210 | | -``` |
| 136 | +Pull requests, discussions, and ideas are always welcome! |
211 | 137 |
|
212 | | -(Metadata test in Test explorer will stay red) |
| 138 | +Read more: |
213 | 139 |
|
| 140 | +- [How to write your own Handler in Python](./DEVGUIDE.md) |
| 141 | +- [How to contribute ot the project](./CONTRIBUTION.md) |
214 | 142 |
|
215 | 143 |
|
216 | | -# License |
| 144 | +## License & Acknowledgements |
217 | 145 |
|
218 | | -Details of project licensing can be found in the [LICENSE](LICENSE) file in the project repository. |
219 | 146 |
|
220 | | -# Acknowledgments |
| 147 | +Special thanks to Eficdoe Oy, Finland. RobotmkBridge is based on their project [robotframework-oxygen](https://github.com/eficode/robotframework-oxygen?tab=readme-ov-file#developing-oxygen). |
221 | 148 |
|
222 | | -See [ACKNOWLEDGEMENTS](./ACKNOWLEDGEMENTS.md) for more information. |
| 149 | +See [ACKNOWLEDGEMENTS.md](ACKNOWLEDGEMENTS.md) for the roots of the project and credits. |
| 150 | +Released under the [MIT License](LICENSE). |
223 | 151 |
|
0 commit comments