You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
*ĐArcher* is an automated testing framework aiming to test on-chain-off-chain synchronization bugs in [decentralized applications (DApps)](https://ethereum.org/en/dapps/).
6
6
7
-
# Introduction
7
+
##Introduction
8
8
9
9
A DApp usually consists of an on-chain layer (smart contracts on blockchain) and an off-chain layer (programs running outside blockchain).
10
10
The on-chain layer provides adequate security to the DApp, while the off-chain layer focuses on the performance and user experience.
@@ -21,57 +21,63 @@ The inconsistencies may result in significant consequences, such as financial lo
21
21
On-chain-off-chain synchronization bugs refer to those bugs that result in the off-chain state to be inconsistent with the on-chain state.
22
22
*ĐArcher* aims to detect these bugs on top of fuzzing with some oracles, as proposed in the paper.
23
23
24
-
The workflow of *ĐArcher* is shown as follows:
24
+
Using *ĐArcher* to test DApps involves two phases. The workflow of *ĐArcher* is shown as follows:
25
25
26
26
27
+
### Phase 1 - Explore functionalities and collect transaction data
28
+
27
29
*ĐArcher* leverages [Crawljax](https://github.com/crawljax/crawljax) to automatically explore the functionalities of DApps and initiate transactions.
28
30
For each transaction, *ĐArcher* tries to check whether the on-chain and off-chain states are inconsistent during the lifecycle of the transaction.
29
31
30
32
Each transaction will be driven to traverse its lifecycle in the order: `Created` -> `Pending` -> `Executed` -> `Reversed` -> `Executed` -> `Finalized`.
31
33
At each lifecycle state, the off-chain state of the DApp will be fetched and saved for later analysis.
32
34
33
35
By the end of functionality exploration using Crawljax, a set of transaction executed will be recorded along with the off-chain states at each transaction lifecycle state.
34
-
An analyzer will be used to apply oracles proposed in the paper on each transaction to check whether the inconsistencies between on-chain and off-chain states are induced during its lifecycle.
36
+
37
+
### Phase 2 - Analyze transaction data and identify on-chain-off-chain synchronization bugs
38
+
39
+
An analyzer will be used to apply oracles proposed in the paper on each transaction recorded in Phase 1 to check whether the inconsistencies between on-chain and off-chain states are induced during its lifecycle.
35
40
In fact, here the oracles only use off-chain states to check inconsistencies based on the heuristics explained in the paper.
36
41
This is one of our contributions.
37
42
38
43
In the end, a report will be generated, stating during the lifecycle of which transactions the inconsistencies between on-chain and off-chain states occur.
39
-
This indicates the on-chain-off-chain synchronization bugs in the DApp.
44
+
This indicates the on-chain-off-chain synchronization bugs (the bugs have two types: Type-I and Type-II, respectively) in the DApp.
40
45
41
46
Note that *ĐArcher* does not locate bugs in the source code.
42
47
*ĐArcher* only identifies transactions, of which the handling program may have bugs to keep on-chain and off-chain states consistent.
43
48
49
+
For detail explanations, please refer to the preview version of our paper below.
50
+
44
51
## Publication
45
52
46
53
See the FSE'21 submission #98[preview version](./darcher-preview.pdf).
47
54
48
-
## Requirements
55
+
## Installation
56
+
57
+
### Pre-configured Virtual Machine
49
58
50
-
See [REQUIREMENTS](./REQUIREMENTS.md)
59
+
Alternatively, we provide a VirtualBox image [here on Zenodo](https://zenodo.org/record/4883114) (please always use the latest version on Zenodo), which contains pre-configured environment of *ĐArcher*.
60
+
Note that the size of the image is huge (approximate 23.2 GB) due to the fact that *ĐArcher* works on integrated testing of Apps, which requires a GUI environment.
51
61
52
-
## Installation
62
+
The default user of the virtual machine is `darcher` with password `darcher` (in case you need root privilege).
63
+
Inside the virtual machine, the *ĐArcher* project is located at `$HOME/darcher`.
64
+
All commands below are executed at the root directory of *ĐArcher* project.
53
65
54
66
### Build from Source
67
+
68
+
Please check the system requirements: [REQUIREMENTS](./REQUIREMENTS.md)
**Note**: This project contains several submodules. Be sure to add `--recurse-submodules` when git cloning.
60
75
61
-
Proceed at [INSTALL](./INSTALL.md) for installation of dependencies and compilation.
62
-
63
-
### Pre-configured Virtual Machine
64
-
65
-
Alternatively, we provide a virtual box image [here](https://zenodo.org/record/4812350), which contains pre-configured environment of *ĐArcher*.
66
-
Note that the size of the image is huge (approximate 23.2 GB) due to the fact that *ĐArcher* works on integrated testing of Apps, which requires a GUI environment.
67
-
68
-
The default user of the virtual machine is `darcher` with password `darcher` (in case you need root privilege).
69
-
Inside the virtual machine, the *ĐArcher* project is located at `$HOME/darcher`.
70
-
All commands below are executed at the root directory of *ĐArcher* project.
76
+
Proceed at [INSTALL](./INSTALL.md) for installation of dependencies and compilation.
@@ -87,36 +93,87 @@ All commands below are executed at the root directory of *ĐArcher* project.
87
93
### Experiment Results
88
94
89
95
The results of experiments in the paper are located in the folder
96
+
`experiment-results`.
97
+
```bash
98
+
experiment-results
99
+
├── AgroChain
100
+
│ ├── AgroChain.report.json
101
+
│ └── rounds
102
+
│ ├── AgroChain0
103
+
│ └── transactions
104
+
│ ├── {txhash}.json
105
+
│ ├── ...
106
+
│ ├── AgroChain1
107
+
│ ├── ...
108
+
├── Augur
109
+
├── ...
110
+
```
111
+
The structure of the `experiment-result` folder is shown above.
112
+
Each subject has one folder, containing a `.report.json` file and 10 rounds of experiment results (we repeat experiment for each subject for 10 times).
90
113
91
-
The transactions initiated and processed during the experiments are [`experiment-results`](./experiment-results).
92
-
The results are grouped by each DApp and each round of experiment.
93
-
The experiment for each DApp is set to be 1 hour.
94
-
We repeated experiments for each DApp 10 times (10 rounds).
114
+
As mentioned previously, the testing using *ĐArcher* has two phases:
115
+
1.**Phase 1**: Collect transaction data
116
+
2.**Phase 2**: Analyze using proposed oracles
95
117
96
-
Every transaction executed in each round of experiments is stored in a `.json` file, which contains:
118
+
#### Result of Phase 1
119
+
120
+
Results of Phase 1 are located in folder `experiment-result/<subject>/rounds/<subject>[0-9]/transactions`.
121
+
The ten sub-folders for each subject represents 10 repeated experiments.
122
+
During the experiments, each transaction executed are recorded in a `.json` file in this folder. The `.json` file contains the following things:
97
123
- the stack trace of DApp where the transaction is initiated
98
124
- the off-chain state of DApp at each transaction lifecycle state
99
125
- the console error during the handling of the transaction (used as Baseline-II in the paper)
100
126
- the contract vulnerability revealed by the transaction (used as Baseline-I in the paper)
101
127
102
-
Note that to facilitate programming, the transaction lifecycle states are encoded as integers:
103
-
- 0: `Created`
104
-
- 1: `Pending`
105
-
- 2: `Executed`
106
-
- 3: `Dropped`
107
-
- 4: `Finalized`
108
-
- 5: `Reversed`
109
-
- 6: `Re-executed` (the second time the transaction reach `Executed` state)
110
-
111
-
The analysis report of all transactions of a DApp are put in a `.report.json` file.
112
-
The report groups the transactions by the stack traces that indicate the corresponding functionalities in the DApp.
128
+
The `.json` file has the following structure:
129
+
```
130
+
{
131
+
"hash": transaction hash
132
+
"stack": the stack trace of the DApp where the transaction is initiated
133
+
"states": {
134
+
0: DApp off-chain state when transaction is at `Created` state
135
+
1: DApp off-chain state when transaction is at `Pending` state
136
+
2: DApp off-chain state when transaction is at `Executed` state
137
+
3: DApp off-chain state when transaction is at `Dropped` state
138
+
4: DApp off-chain state when transaction is at `Finalized` state
139
+
5: DApp off-chain state when transaction is at `Reversed` state
140
+
6: DApp off-chain state when transaction is at `Re-executed` state (the second time the transaction reach `Executed` state)
141
+
}
142
+
}
143
+
```
144
+
145
+
#### Results of Phase 2
146
+
147
+
After analyzing the transaction data in Phase 1, an analysis report of all transactions for each DApp subject is put in a `.report.json` file under the folder `experiment-result/<subject>/<subject>.report.json`.
148
+
149
+
The report groups the transactions by the stack traces that indicate the corresponding functionalities in the DApp.
113
150
In each group, the following things are listed:
114
151
- A list of all transactions with this stack trace.
115
-
- A list of transactions that induce console errors in the DApp.
152
+
- A list of transactions that induce runtime errors in the DApp.
116
153
- A list of transactions that reveal vulnerabilities in the smart contract on blockchain.
117
154
- A list of transactions that are reported by oracles of *ĐArcher* to reveal Type-I or Type-II bugs (see the definitions in the paper) in the DApp.
118
155
119
-
### Evaluation Reproduction
156
+
The structure of the `.report.json` file is shown as follows:
157
+
```
158
+
{
159
+
"totalRuntimeError": total number of runtime errors in the experiment,
160
+
"totalTransactions": total number of transactions executed in the experiment,
161
+
"totalFunctionalities": total number of distinct stack traces (considered as distinct functionalities of the DApp),
162
+
"functionalities": [
163
+
{
164
+
"stack": the stack trace of this functionality
165
+
"txHashes": the list of transactions invoking this functionality
166
+
"runtimeErrorTxHashes": the list of transactions that induce runtime errors in the DApp
167
+
"vulnerabilityTxHashes": the list of transactions that reveal vulnerabilities in the smart contract
168
+
"BugTypeOneTxHashes": the list of transactions that are reported by ĐArcher as Type-I bugs
169
+
"BugTypeTwoTxHashes": the list of transactions that are reported by ĐArcher as Type-II bugs
To facilitate the experiments on DApps, we have built docker images (publicly available on Docker Hub [here](https://hub.docker.com/u/darcherframework)) for each DApp subject along with the controlled blockchain that traverses the transaction lifecycles.
122
179
@@ -134,7 +191,7 @@ After that, the experiment will start.
134
191
135
192
**Note**: Some DApps may take some time to initialize, so it may wait for a while after docker compose services are started. (You may see `Waiting for services in docker ready...` in the console).
136
193
137
-
**To interrupt the experiment**, press `CTRL-C` to send `SIGINT` signal.
194
+
Each experiment will take one hour, but you can interrupt the experiment and see partial results. **To interrupt the experiment**, press `CTRL-C` to send `SIGINT` signal.
138
195
If the experiment is interrupted, you need to manually stop the docker-compose services using `yarn workspace @darcher/examples cleanup:<subject>`.
139
196
140
197
#### Special Case - MetaMask
@@ -149,8 +206,8 @@ Disabling and enabling extensions need to be done manually.
149
206
150
207
### Experiment Output
151
208
152
-
Each experiment will take 1 hour, and the transactions executed during the experiment will be placed under `./packages/darcher-examples/<subject>/results` directory.
153
-
A `.json` file will be generated for each transaction, as stated [above](#Experiment Results), with the off-chain states at each lifecycle states.
209
+
Each experiment will take 1 hour, and the transactions executed during the experiment will be placed under `./packages/darcher-examples/<subject>/results/<date>` directory.
210
+
A `.json` file will be generated for each transaction, as stated in the previous section [Experiment Results](#Experiment Results).
154
211
155
212
Note that the functionality exploration using Crawljax is a random process.
156
213
The output may not be the same across different runs.
@@ -160,24 +217,26 @@ After the experiment finishes, if the docker-compose services are not stopped au
At this moment, the results output are only transactions with raw data of off-chain states.
220
+
At this moment, the results output are only transactions with raw data of off-chain states, i.e., results of **Phase 1**.
164
221
We need to further analyze the transactions using the oracles proposed in the paper.
165
222
See below.
166
223
167
-
### Analysis with _ĐArcher_ Oracles
224
+
### Analysis with _ĐArcher_ Oracles - Getting Phase 2 Results
168
225
169
226
The transactions `.json` files generated in the experiment do not reveal the on-chain-off-chain synchronization bugs in the DApp until the oracle is applied to the off-chain states.
170
227
171
228
We provided scripts to analyze the transactions in the `scripts` folder of the root directory.
172
-
The script `<subject>.analyze.ts` are TypeScript scripts for each subject.
229
+
The script `./scripts/<subject>.analyze.ts` are TypeScript scripts for each subject.
173
230
174
231
Previously when off-chain states are fetched at each transaction lifecycle state, all data in the DApp are fetched.
175
232
However, some data fields that changes all the time (if considered, the off-chain and on-chain state will always be inconsistent) need to be ignored, e.g., timestamp, to reduce noises in oracle checking.
176
233
In each `<subject>.analyze.ts` script, we provide some rules to ignore those data fields if necessary for each DApp, as well as the folder containing the transactions to analyze.
177
234
178
235
To analyze the transactions of each DApp in `experiment-results` folder, run the follow command:
179
236
```bash
180
-
yarn analysis:<subject>
237
+
yarn analyze:<subject>
181
238
```
182
239
`<subject>` should be substituted with one of the subject names as given above.
183
-
The report (a `.report.json` file) as described as [above](#Experiment Results) shall be generated at the current directory.
240
+
The report (a `.report.json` file) as described as the previous section [Experiment Results](#Experiment Results) shall be generated at the current directory.
241
+
242
+
**Note**: You may need to change the `dataDir` and `roundDirs` variable in the `<subject>.analyze.ts` to analyze the transactions in your reproduced experiments.
0 commit comments