|
| 1 | +# I. Why we need to setup an automated testing framework? |
| 2 | +In many cases, we have observed that unit test isn't enough to guarantee that the whole system will not break. We simply got lucky when Ed found a problem through his manual system testing that may not always happen. |
| 3 | + |
| 4 | +In some cases, not even the unit test is correct due to writer misunderstanding of required logic. |
| 5 | + |
| 6 | +# II. Framework |
| 7 | +1. Standarize writing unit test (we can even go further with a ci to check). A PR should include: |
| 8 | +* feature specification (in written markdown file in a folder called feature-request) (Gherkin language) |
| 9 | +* unit test write |
| 10 | + |
| 11 | +2. a full - fledged e2e test flow through ci that does following flow |
| 12 | +* upgrade testing: |
| 13 | + * for chain (if exists, else use current terrad): do an upgrade gov test to newer chain binary |
| 14 | + * for wasm (if there are changes to wasm, else use current wasm files): change to newer wasm files |
| 15 | +* regression testing (change to system should not break existing one) |
| 16 | + * backend: |
| 17 | + * chain should run normal |
| 18 | + * ibc should work normal |
| 19 | + * wasm should run normal |
| 20 | + * state sync should run normal |
| 21 | + * frontend: api testing of most important transaction types and query |
| 22 | +* fuzz testing: |
| 23 | + * some nice background reading: https://github.com/osmosis-labs/osmosis/blob/main/simulation/ADR.md |
| 24 | + * we need to care only about events that invoke a state change, we then create randomized request to these events: |
| 25 | + * transaction: create randomized request and broadcast multiple times |
| 26 | + * begin block: randomized environment condition that invokes a begin block logic (Ex: system time, number of validators, ...) |
| 27 | + * end block: randomized environment condition that invokes an end block logic (Ex: system time, number of validators, ...) |
| 28 | + * init genesis: create randomized genesis state |
| 29 | + |
| 30 | +It is hard to invoke a begin, end block logic without first understanding what environment condition it relies on. |
0 commit comments