Skip to content

Commit b5967cb

Browse files
docs: add testing framework document (#142)
1 parent 649f7c4 commit b5967cb

File tree

1 file changed

+30
-0
lines changed

1 file changed

+30
-0
lines changed

tests/README.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# I. Why we need to setup an automated testing framework?
2+
In many cases, we have observed that unit test isn't enough to guarantee that the whole system will not break. We simply got lucky when Ed found a problem through his manual system testing that may not always happen.
3+
4+
In some cases, not even the unit test is correct due to writer misunderstanding of required logic.
5+
6+
# II. Framework
7+
1. Standarize writing unit test (we can even go further with a ci to check). A PR should include:
8+
* feature specification (in written markdown file in a folder called feature-request) (Gherkin language)
9+
* unit test write
10+
11+
2. a full - fledged e2e test flow through ci that does following flow
12+
* upgrade testing:
13+
* for chain (if exists, else use current terrad): do an upgrade gov test to newer chain binary
14+
* for wasm (if there are changes to wasm, else use current wasm files): change to newer wasm files
15+
* regression testing (change to system should not break existing one)
16+
* backend:
17+
* chain should run normal
18+
* ibc should work normal
19+
* wasm should run normal
20+
* state sync should run normal
21+
* frontend: api testing of most important transaction types and query
22+
* fuzz testing:
23+
* some nice background reading: https://github.com/osmosis-labs/osmosis/blob/main/simulation/ADR.md
24+
* we need to care only about events that invoke a state change, we then create randomized request to these events:
25+
* transaction: create randomized request and broadcast multiple times
26+
* begin block: randomized environment condition that invokes a begin block logic (Ex: system time, number of validators, ...)
27+
* end block: randomized environment condition that invokes an end block logic (Ex: system time, number of validators, ...)
28+
* init genesis: create randomized genesis state
29+
30+
It is hard to invoke a begin, end block logic without first understanding what environment condition it relies on.

0 commit comments

Comments
 (0)