Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,15 @@ branches: # this prevents undesired branch builds for PRs
os: linux

python:
- "3.7"
- "3.8"
- "3.9"
- "3.10"
- "3.7"
- "3.8"
- "3.9"
- "3.10"

install:
- pip install -r requirements.txt
- pip install codecov nose-exclude nose-timer "pluggy>=1.0"
- pip install -r requirements.txt
- pip install -r requirements_dev.txt
- pip install codecov nose-exclude nose-timer "pluggy>=1.0"
script: coverage run --source=apiritif -m nose2 -s tests/unit -v
after_success:
- codecov
16 changes: 16 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"python.testing.unittestEnabled": true,
"python.testing.pytestEnabled": false,
"python.testing.nosetestsEnabled": true,
"python.unitTest.nosetestsEnabled": true,
"python.linting.mypyEnabled": true,
"python.linting.flake8Enabled": true,
"python.testing.unittestArgs": [
"-v",
"-s",
"./tests",
"-p",
"test_*.py"
],
"python.formatting.provider": "black"
}
45 changes: 45 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
####################
# Setup tasks #
####################
install:
pip install -r requirement.txt

setup_venv:
python3.9 -m venv venv


setup: setup_venv
(\
. venv/bin/activate;\
pip install -r requirements.txt;\
)



####################
# Testing #
####################
test:
pytest -vvv -x tests

lint:
flake8 async.py boy.py app
mypy --install-types --non-interactive async.py boy.py app

cover:
coverage run --source=app,betonyou -m pytest --ignore=tests/integration --ignore=tests/legacy_integration -xv tests

coverage-report: cover
coverage report -m --skip-empty

coverage-gutter: cover
coverage html --skip-empty -d coverage
coverage xml --skip-empty

bandit:
bandit -r app boy.py

bandit-ci:
bandit -r -ll -ii app boy.py

test-all: lint test bandit
30 changes: 24 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# Apiritif

Apiritif is a number of utilities aimed to simplify the process of maintaining API tests.
Apiritif is a number of utilities aimed to simplify the process of maintaining API tests.
Apiritif tests fully based on python nose tests. This library can help you to develop and run your existing tests.
In order to create any valid tests for Apiritif you can read [nose test documentation](https://nose.readthedocs.io/en/latest/testing.html).

Check Apiritif version with the following command:

```
python -m apiritif -- version
```
Expand All @@ -25,6 +26,7 @@ response.assert_ok() # will raise AssertionError if request wasn't successful
```

`http` object provides the following methods:

```python
from apiritif import http

Expand All @@ -37,6 +39,7 @@ http.head("http://api.example.com/posts")
```

All methods (`get`, `post`, `put`, `patch`, `delete`, `head`) support the following arguments:

```python
def get(address, # URL for the request
params=None, # URL params dict
Expand All @@ -50,19 +53,21 @@ def get(address, # URL for the request
```

##### Certificate usage
Currently `http` supports `pem` and `pkcs12` certificates.

Currently `http` supports `pem` and `pkcs12` certificates.
Here is an example of certificate usage:

```python
http.get("http://api.example.com/posts", encrypted_cert=('./cert.pem', 'passphrase'))
```

First parameter is path to certificate, second is the passphrase certificate encrypted with.

## HTTP Targets

Target is an object that captures resource name of the URL (protocol, domain, port)
and allows to set some settings applied to all requests made for a target.


```python
from apiritif import http

Expand All @@ -72,6 +77,7 @@ qa_env.get("/api/v4/user")
```

Target constructor supports the following options:

```python
target = apiritif.http.target(
address, # target base address
Expand All @@ -83,12 +89,12 @@ target = apiritif.http.target(
)
```


## Assertions

Apiritif responses provide a lot of useful assertions that can be used on responses.

Here's the list of assertions that can be used:

```python
response = http.get("http://example.com/")

Expand Down Expand Up @@ -147,6 +153,7 @@ response.assert_not_cssselect(selector, expected_value=None, attribute=None)
```

Note that assertions can be chained, so the following construction is entirely valid:

```python

response = http.get("http://example.com/")
Expand Down Expand Up @@ -181,6 +188,7 @@ def test_with_login():

http.get("https://blazedemo.com/users/all").assert_ok()
```

At the same time requests to `users/all` page will be executed outside of transaction even if something inside transaction fails.

Transaction defines the name for the block of code. This name with execution results of this particular block will be displayed in the output report.
Expand All @@ -203,6 +211,7 @@ class Tests(TestCase):
http.get("https://blazedemo.com/contactUs").assert_ok()
http.get("https://blazedemo.com/copyright").assert_ok()
```

In this case we have multiple requests divided into blocks. I do not want to test pages under `users` space if it is not available.
For this purpose we can use `smart_transaction`.

Expand All @@ -225,15 +234,17 @@ class Tests(TestCase):
http.get("https://blazedemo.com/contactUs").assert_ok()
http.get("https://blazedemo.com/copyright").assert_ok()
```

Now this two blocks are wrapped into `smart_transaction` which would help with error test flow handling and logging.

Also each transaction defines the name for the block of code and will be displayed in the output report.

Now about `apiritif.put_into_thread_store(func_mode=True)`, this is test execution mode for apiritif.
We can execute all of the transactions in test no matter what or stop after first failed transaction.
This flag tells to apiritif "Stop execution if some transaction failed". `False` says "Run till the end in any case".

##### Nose Flow Control

It's one more feature based on smart transactions. It changes `func_mode` if necessary to execute whole teardown block,
intended to finalize all necessary things.

Expand All @@ -247,17 +258,21 @@ def test_flow-control(self):
self._teardown1()
self._teardown2()
```

If this test will be interrupted in `_method_with_exception`, both of teardown methods will be executed even if them raise exception.
Please note two differences with usage of `tearDown` method of nose:

1. all parts of teardown stage will be executed as mentioned above (will be interrupted in regular nose execution)
2. results of teardown steps will be written by apiritif SampleWriter into output file (nose lost them as tearDown isn't recognised as test).

##### Graceful shutdown

Somethimes waiting of end of test isn't necessary and we prefer to break it but save all current results and handle all teardown steps. (see above)
It's possible with GRACEFUL flag. To use it you can run apiritif with GRACEFUL environment variable pointed to any file name.
Apiritif will be interrupted as soon as the file is created.

## CSV Reader

In order to use data from csv file as test parameters Apiritif provides two different csv readers.
Simple `CSVReader` helps you to read data from file line by line and use this data wherever you need:

Expand All @@ -271,7 +286,7 @@ class Tests(TestCase):
```

In case of multithreading testing you may need to deviate data between threads and ysu uniq lines for each thread.
`CSVReaderPerThread` helps to solve this problem:
`CSVReaderPerThread` helps to solve this problem:

```python
data_per_thread_reader = apiritif.CSVReaderPerThread('---path to required file---')
Expand All @@ -288,16 +303,19 @@ class Tests(TestCase):

Apiritif writes output data from tests in `apiritif.#.csv` files by default. Here `#` is number of executing process.
The output file is similar to this:

```csv
timeStamp,elapsed,Latency,label,responseCode,responseMessage,success,allThreads,bytes
1602759519185,0,0,Correct test,,,true,0,2
1602759519186,0,0,Correct transaction,,,true,0,2
1602759519187,0,0,Test with exception,,Exception: Horrible error,false,0,2
```

It contains test and transaction results for executed tests by one process.

### Environment Variables

There are environment variables to control length of response/request body to be written into traces and logs:

* `APIRITIF_TRACE_BODY_EXCLIMIT` - limit of body part to include into exception messages, default is 1024
* `APIRITIF_TRACE_BODY_HARDLIMIT` - limit of body length to include into JSON trace records, default is unlimited
13 changes: 8 additions & 5 deletions apiritif/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,12 @@
"""

from .csv import CSVReaderPerThread
from .thread import put_into_thread_store, get_from_thread_store, external_handler, get_stage, set_stage
from .thread import get_transaction_handlers, set_transaction_handlers, get_iteration
from .http import http, transaction, transaction_logged, smart_transaction, recorder
from .http import Event, TransactionStarted, TransactionEnded, Request, Assertion, AssertionFailure
from .http import (HTTP, Assertion, AssertionFailure, Event, Request,
TransactionEnded, TransactionStarted, http, recorder,
smart_transaction, transaction, transaction_logged)
from .thread import (external_handler, get_from_thread_store, get_iteration,
get_stage, get_transaction_handlers,
put_into_thread_store, set_stage,
set_transaction_handlers)
from .utilities import *
from .utils import headers_as_text, assert_regexp, assert_not_regexp, log
from .utils import assert_not_regexp, assert_regexp, headers_as_text, log
Loading