|
| 1 | +# Definition of standards for testing |
| 2 | + |
| 3 | +## Context and Problem Statement |
| 4 | + |
| 5 | +As any application, we want IroCO2 to be reliable throughout time. To that end, tests should be written all along code writing, but those tests should not depend on the person who write code. Thus, a definition of standards appears to be mandatory for two reasons : |
| 6 | + - this will facilitate test writing, as well as test reviewing |
| 7 | + - this will ensure the application is resilient, and prevent any regression to be silently implemented |
| 8 | + |
| 9 | +The frame we drew with standards to be respected is settled upon two lines of conduct : code coverage as an indicator, and test strategy as a mindset. |
| 10 | + |
| 11 | +--- |
| 12 | + |
| 13 | +## 1. Code coverage |
| 14 | + |
| 15 | +The definition of standards around coverage was mainly based on experience and knowledge of the Ippon Technology team in place at the moment of said definition. Nevertheless, online resources can easily be found if you wish to deepen your comprehension of these rules definition (see for example the article ["What is code coverage"](https://www.atlassian.com/continuous-delivery/software-testing/code-coverage)). |
| 16 | + |
| 17 | +### Objectives for code coverage by perimeter |
| 18 | + |
| 19 | +Since each side of IroCO2 application has its specificities, minimal coverage rate was established project by project. |
| 20 | + |
| 21 | +Please keep in mind, in particular for Back-end and Front-end projects, that coverage philosophy of IroCO2 team go beyond quantity of tests, thinking also about **quality** and **pertinence** of each test. |
| 22 | + |
| 23 | +| Perimeter | Objective | |
| 24 | +| -------------- | --------- | |
| 25 | +| Infrastructure | 50%* | |
| 26 | +| Back-end | 90% | |
| 27 | +| Front-end | 80%** | |
| 28 | + |
| 29 | +> \* : *indicative value which reflect the ambition to test main components with Terraform Test* |
| 30 | +> |
| 31 | +> \*\* : *defined without clear idea of necessary time to catch up on that objective, so it can be considered as a minimal* |
| 32 | +
|
| 33 | +--- |
| 34 | + |
| 35 | +## 2. Test strategy |
| 36 | + |
| 37 | +The main idea behind our test strategy is the Test Pyramid, whose base is made of Unit tests, End-to-end tests at the top, with Integration tests between (see this [pragmatic article](https://martinfowler.com/articles/practical-test-pyramid.html) of Martin Fowler). |
| 38 | + |
| 39 | +Still, the concrete strategy must match the reality of the project IroCO2, and moreover it seems necessary to define what we mean with unit tests, integration tests or end-to-end tests, for each perimeter. |
| 40 | + |
| 41 | +--- |
| 42 | + |
| 43 | +### 2.a. Infrastructure |
| 44 | + |
| 45 | +#### UNIT testing |
| 46 | + |
| 47 | +- Definition |
| 48 | +> Testing infrastructure elements |
| 49 | +- Strategy |
| 50 | +> Testing of main elements using a combination of Terraform Test, variable validation, pre/post conditions as well as check blocks. |
| 51 | +
|
| 52 | +#### INTEGRATION testing |
| 53 | + |
| 54 | +- Definition |
| 55 | +> Testing infrastructure elements |
| 56 | +- Strategy |
| 57 | +> Testing of main elements using a combination of Terraform Test, variable validation, pre/post conditions as well as check blocks. |
| 58 | +
|
| 59 | +#### END-TO-END testing |
| 60 | +- Definition |
| 61 | +> Testing the whole application. |
| 62 | +- Strategy |
| 63 | +> A few tests of use cases, to be executed on an AWS sandbox account (e.g. Ippon's one) to run a `terraform apply` on the whole stack. Automation can be implemented for Continuous Delivery on non production environment, based on the fork of the github repo. |
| 64 | +
|
| 65 | +--- |
| 66 | + |
| 67 | +### 2.b. Back-end |
| 68 | + |
| 69 | +#### UNIT testing |
| 70 | +- Definition |
| 71 | +> Testing the classes individually, mocking anything which could be outside its scope. |
| 72 | +- Strategy |
| 73 | +> Our ambition is to test every possible output of every method of every classe. **The coverage with unit tests should never decrease due to a new development**. When possible, it should rise with new unit tests for existing classes with missing test cases. |
| 74 | +> Also, an ambition of IroCO2 team is to produce back-end code with TDD. |
| 75 | +
|
| 76 | +#### INTEGRATION testing |
| 77 | +- Definition |
| 78 | +> Testing the components in a more complex way. Some (but not all) external behavior are mocked. |
| 79 | +- Strategy |
| 80 | +> For a given use case, several scenarii should be tested, including errors handling. |
| 81 | +
|
| 82 | +#### END-TO-END testing |
| 83 | +- Definition |
| 84 | +> Testing back-end from endpoints to databases and cloud services without mocking anything. |
| 85 | +- Strategy |
| 86 | +> Each endpoint should be tested through two test cases at least : one testing expected behavior, and one testing error handling |
| 87 | +
|
| 88 | +--- |
| 89 | + |
| 90 | +### 2.c. Front-end |
| 91 | + |
| 92 | +#### UNIT testing |
| 93 | +- Definition |
| 94 | +> Testing the components individually, mocking what does not depend on its scope. |
| 95 | +- Strategy |
| 96 | +> Our ambition is to test every UI component of the application. **The coverage with unit tests should never decrease due to a new development**. When possible, it should rise with new unit tests for existing components with missing test cases. |
| 97 | +> Also, an ambition of IroCO2 team is to produce front-end code with TDD. |
| 98 | +
|
| 99 | +#### INTEGRATION testing |
| 100 | +- Definition |
| 101 | +> Testing the components in a more complex way, navigating between components for example |
| 102 | +- Strategy |
| 103 | +> At the moment of the first definition of test strategy, this floor of the Front-end test pyramid is not a priority for IroCO2 team. Anyone is free to init the subject at any moment, beginning with replacing content of this field with the strategy for integration tests. |
| 104 | +
|
| 105 | +#### END-TO-END testing |
| 106 | +- Definition |
| 107 | +> Testing the whole application. |
| 108 | +- Strategy |
| 109 | +> A few tests of use cases, to be executed on an AWS sandbox account (e.g. Ippon's one) to run a `terraform apply` on the whole stack. Automation can be implemented for Continuous Delivery on non production environment, based on the fork of the github repo. |
0 commit comments