Skip to content

Commit 0ad13e1

Browse files
ezequiel-rodriguezezequiel-rodriguez
authored
Merge pull request #124 from rsksmart/feature/explorer-api-v2
refactor: documentation
2 parents c8109a2 + 6a411a2 commit 0ad13e1

File tree

3 files changed

+122
-220
lines changed

3 files changed

+122
-220
lines changed

‎README.md

+120-218
Original file line numberDiff line numberDiff line change
@@ -1,244 +1,146 @@
1-
# Rsk explorer api
1+
# Rsk Explorer API
2+
3+
# Requisites
4+
- postgres
5+
- node: v16+
6+
- access to JSON/RPC interface of a rskj node >= 2.0.1 with this modules enabled: eth, net, web3, txpool, debug and trace.
7+
8+
# Configuration steps
9+
10+
## Section 1: Environment setup
11+
- create a database 'explorer_db'
12+
- create sql tables using the script prisma/rsk-explorer-database.sql
13+
- Install pm2:
14+
- npm install -g pm2
15+
- Enable pm2 log rotation: pm2 install pm2-logrotate
16+
- Enable log-rotation compression: pm2 set pm2-logrotate:compress true
17+
- For more options:
18+
- see [pm2-logrotate](https://pm2.keymetrics.io/docs/usage/log-management/#pm2-logrotate-module)
19+
- see [pm2-logrotate configuration](https://github.com/keymetrics/pm2-logrotate#configure) to set the rotation options
20+
- Configure database credentials, node urls, etc, in src/lib/defaultConfig.js file:
221

3-
## Description
4-
5-
Rsk blockchain explorer
6-
7-
## Components
8-
9-
### API server
10-
11-
HTTP/WS server, see the documentation [here](doc/api.md).
12-
13-
### Blocks services
14-
15-
Imports blockchain data from rsk node to DB.
16-
17-
- **blocksRouter:** routes messages between services *REQUIRED*
18-
- **blocksListener:** listens to new blocks from node and announces them as service event.
19-
- **blocksRequester:** requests blocks from node.
20-
- **blocksChecker:** checks database for missing blocks and reorgs, emits missing/bad blocks.
21-
- **txPool:** listens to node transactions pool and stores changes in the DB.
22-
- **blocksBalances:** gets historical addresses balances and stores in the DB.
23-
- **blocksStatus:** checks DB and node status and stores changes in the DB.
24-
- **blocksStats:** gets BC stats and stores in the DB.
25-
26-
### API user events service
27-
28-
Allows to update fields on the fly and send async response to clients.
29-
Required by the contract verification module.
30-
31-
## Requisites
32-
33-
- mongodb > 4
34-
- node >= 12.18.2
35-
- access to JSON/RPC interface of a rskj node >= 2.0.1
36-
with this modules enabled: eth, net, web3, txpool, debug and trace.
37-
38-
## Install
39-
40-
- Install dependencies
41-
42-
``` shell
43-
npm install
44-
```
45-
46-
## Create log dir
47-
48-
```shell
49-
sudo mkdir /var/log/rsk-explorer
50-
sudo chown $USER /var/log/rsk-explorer/
51-
chmod 755 /var/log/rsk-explorer
52-
```
53-
54-
Note: You can change the log folder in config.json
55-
56-
## Configuration file
57-
58-
(optional)
59-
60-
``` shell
61-
cp config-example.json config.json
62-
```
63-
64-
see [configuration](#configuration)
65-
66-
## Start
67-
68-
### Services
69-
70-
Services can be started manually one by one, but we recommend to use [PM2](https://github.com/Unitech/pm2).
71-
This repo includes a pm2 ecosystem file that starts all services automatically.
72-
73-
#### Install PM2
74-
75-
``` shell
76-
npm install -g pm2
77-
```
78-
79-
To enable pm2 log rotation
80-
81-
``` shell
82-
pm2 install pm2-logrotate
83-
```
84-
85-
see [pm2-logrotate](https://pm2.keymetrics.io/docs/usage/log-management/#pm2-logrotate-module)
86-
see [pm2-logrotate configuration](https://github.com/keymetrics/pm2-logrotate#configure) to set the rotation options
87-
88-
e.g:
89-
90-
```shell
91-
92-
pm2 set pm2-logrotate:compress true
93-
94-
```
95-
96-
#### Start services
97-
98-
```shell
99-
pm2 start dist/services/blocks.config.js
100-
```
101-
102-
### API
103-
104-
``` shell
105-
pm2 start dist/api/api.config.js
106-
```
107-
108-
## Show PM2 logs in pretty format
109-
110-
All tasks
111-
112-
```shell
113-
114-
:~/rsk-explorer-api$ pm2 log --raw | npx bunyan
115-
116-
```
117-
118-
One task
119-
120-
```shell
121-
122-
:~/rsk-explorer-api$ pm2 log blocksListener --raw | npx bunyan
123-
124-
```
125-
126-
## Commands
127-
128-
Run api in development mode
129-
130-
``` shell
131-
npm run dev
132-
```
133-
134-
Run blocks service in development mode
135-
136-
``` shell
137-
npm run blocks
138-
```
139-
140-
Production build to ./dist folder
141-
142-
``` shell
143-
npm run build
144-
```
145-
146-
## Configuration
147-
148-
**config.json**
149-
See defaults on: **lib/defaultConfig**
150-
*(config.json overrides this values)*
151-
152-
Use:
153-
154-
```shell
155-
node dist/tools/showConfig.js
156-
```
157-
158-
to check current configuration
159-
160-
**Configuration Example:**
161-
162-
``` javascript
163-
"source": {
164-
"node": "localhost",
165-
"port": 4444
22+
```javascript
23+
{
24+
source: {
25+
protocol: 'http',
26+
node: 'localhost',
27+
port: 4444,
28+
url: null
29+
},
30+
sourceRoutes: { // Nod3Router routes, used as default when source is an array of sources
31+
subscribe: 0, // delegates subscriptions to the first node
32+
rsk: 0, // delegates rsk module to the node that handle subscriptions
33+
trace: 1 // delegates trace_ module to the second node
16634
},
167-
"api": {
168-
"address": "localhost",
169-
"port": 3003
35+
db: {
36+
protocol: 'postgres://',
37+
databaseName: 'explorer_db',
38+
host: 'localhost',
39+
port: 5432,
40+
user: 'postgres',
41+
password: 12345678
17042
},
171-
"db": {
172-
"server": "localhost",
173-
"port": 27017,
174-
"database": "blockDB"
175-
}
43+
api: {
44+
address: 'localhost',
45+
port: 3003,
46+
lastBlocks: 30,
47+
MIN_LIMIT: 10,
48+
LIMIT: 50,
49+
MAX_LIMIT: 500,
50+
MAX_PAGES: 10,
51+
allowUserEvents: false,
52+
exposeDoc: false,
53+
// All modules are enabled as default
54+
modules: setAllModules(true),
55+
delayedFields,
56+
allowCountQueries: false
57+
},
58+
blocks: {
59+
blocksQueueSize: 10,
60+
bcTipSize: 120,
61+
batchRequestSize: 20,
62+
debug: false,
63+
ports: [3010], // list of services ports, if the list runs out, the services will try to take the next ports starting from the last
64+
address: '127.0.0.1',
65+
services
66+
},
67+
forceSaveBcStats: true,
68+
enableTxPoolFromApi: true
69+
}
70+
```
17671

177-
```
72+
### api
73+
- **address** [string] api server bind address
74+
- **port** [number] api server port
75+
- **exposeDoc** [boolean]: serve rsk-openapi-ui on /doc to render swagger.json
76+
- **allowUserEvents** [boolean]: allow contractVerifier
77+
78+
### To enable contract verifier module:
17879

17980
The contractVerifier module requires a connection to a [rsk-contract-verifier](https://github.com/rsksmart/rsk-contract-verifier)
18081
instance. The url must be provided on api section:
18182

83+
- set api.allowUserEvents to true
84+
- add the contract verifier url inside api:
18285
```javascript
183-
"api":{
184-
"contractVerifier": {
185-
"url": "ws://localhost:3008"
186-
}
86+
api:{
87+
//... other configs,
88+
contractVerifier: {
89+
url: 'ws://localhost:3008'
90+
},
91+
//... other configs
18792
}
188-
18993
```
19094

191-
### Source
192-
193-
Address of rskj node or array of addresses of rskj nodes
95+
## Section 3: Build process
96+
Run commands:
97+
- npm install
98+
- npm run build
99+
- npx prisma generate
194100

195-
e.g.:
196-
197-
```json
198-
{
199-
"url":"http://localhost:4444"
200-
}
101+
## Start
102+
- Start block service: npm run start-blocks
103+
- Start api: npm run start-api
201104

202-
```
105+
Note: an rsk node must be running in port or url specified in defaultConfig.js. Otherwise, blocks service will crash.
203106

204-
e.g:
107+
## Section 4: Logs
108+
To see block service logs:
109+
- npm run blocks-logs-raw (production)
110+
- npm run blocks-logs-pretty (development)
205111

206-
```json
207-
[
208-
{ "url":"http://localhost:4444" },
209-
{ "url":"http://othernode:4444" }
210-
]
112+
To see api logs:
113+
- npm run api-logs-raw (production)
114+
- npm run api-logs-pretty (development)
211115

212-
```
116+
## Section 5: Tools
213117

214-
### db
118+
Get a block:
119+
- node dist/tools/getBlock.js 4000 (print in console)
120+
- node dist/tools/getBlock.js 4000 --save (print in console and also store in database)
215121

216-
- **server**": "localhost"
217-
- **port**": 27017
218-
- **database**: "explorerDB"
122+
Get missing segments in database:
123+
- node dist/tools/missingSegments.js
219124

220-
**Optionals:**
125+
## Components
221126

222-
- **user**: < user >
223-
- **password**: < password >
127+
The logic in charge of indexing the blockchain in the database and maintaining data integrity consists of 4 services:
128+
- [blocks-checker-service]: ensures integrity of the most recent 120 blocks in database at the moment of running the service
129+
- [savetip-service]: requests blocks inside the tip size threshold (the top 120 newest, starting from latest block)
130+
- [live-syncer-service]: requests new blocks to the RSK node, starting from latest block
131+
- [static-syncer-service]: Stores immutable blocks (starting from latest - 120). To do so, it checks database gaps between already indexed blocks, and requests the missing ones by iterating the gaps
224132

225-
### blocks
226-
227-
- **validateCollections** :[Boolean] Validate collections at blocks service start, default false
228-
- **blocksQueueSize**:[Number] blocksRequester queue size
229-
- **bcTipSize**:[Number] Number of confirmations required to check blocks
230-
- **debug**:[Boolean] Enable logging of nod3 requests, default false
231-
- **services**:[Object] {ServiceName:[Boolean]}. All services are enabled as default, to disable a service use: **serviceName:false**
133+
API server: HTTP/WS server
134+
- API Documentation: [docs](doc/api.md)
135+
- Swagger docs: [open api specification](public/swagger.json)
232136

233-
### api
137+
## Development
234138

235-
- **address** [string] api server bind address
236-
- **port** [number] api server port
139+
Run api in development mode: npm run dev
140+
Run blocks in development mode:
237141

238-
- **allowUserEvents** [boolean]: enable/disable userEventsApi
239-
- **exposeDoc** [boolean]: serve rsk-openapi-ui on /doc to render swagger.json
142+
- npm run build
143+
- npm prisma generate
144+
- npm run blocks-start
240145

241-
## Documentation
242-
243-
- [api documentation](doc/api.md)
244-
- [open api specification](public/swagger.json)
146+
Note Before uploading changes, remember to execute npm run build after upgrading version in package.version, so swagger docs compile the version number too.

‎package.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "rsk-explorer-api",
3-
"version": "2.0.7",
3+
"version": "2.0.8",
44
"description": "",
55
"main": "index.js",
66
"scripts": {

‎public/swagger.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"info": {
33
"title": "rsk-explorer-api",
4-
"version": "2.0.5",
4+
"version": "2.0.8",
55
"description": "explorer API Documentation"
66
},
77
"swagger": "2.0",

0 commit comments

Comments
 (0)