PR to add Reward info to Insight Explorer

[gaeacodes]

This PR just contains the changes needed for the UI.
I have submitted other PR's to @DeckerSU's bitcore-node-komodo(DeckerSU/bitcore-node-komodo#2), insight-api-komodo(DeckerSU/insight-api-komodo#1)

When merged, all of these together will allow reward data to be displayed on the insight explorer.
To test my changes, scripts in the repo https://github.com/gaeacodes/komodo-install-explorer can be used

Make sure komodod is not running and execute the following commands:

git clone https://github.com/gaeacodes/komodo-install-explorer
cd komodo-install-explorer
./setup-explorer-directory.sh
./install-komodo-explorer.sh
./komodo-explorer-start.sh

Then, start komodod with all the indexes like tx,timestamp,spent and address enabled and visit http://localhost:7774 to view the explorer.

Thanks to @gcharang for all the suggestions and help.

The Insight API gets a few addtions to the tx endpoint. To view them, visit http://localhost:7774/insight-api-komodo/tx/<txid>

Here are a few screenshots that show the addtions done to UI.

[ca333]

screenshots look great - will review and test patch tomorrow, thank you @gaeacodes !

[smk762]

I'm reindexing for addresindex etc, should be able to test tonight.

[gaeacodes]

Distinquished all needed changes from 3 repoes (bitcore-node-komodo, insight-api-komodo, insight-ui-komodo) as a patches, made an instruction, all of these already available in commit above. Live demo available here - api.kmd.dev and here - explorer.kmd.sh

From DeckerSU as a comment to the PR for bitcore-node-komodo.

I have also hosted an updated instance of the explorer at http://34.122.103.103:7774

[smk762]

I'm still reindexing - slower than a wet sheep going uphill.
Tried to use a bootstrap but it errored so doing from scratch, about 50% there.

[smk762]

Looking good here, really nice additions!

I'm assuming the KMD Available to Claim is for individual utxos? As for the same address, some older tx are zero etc. most recent utxo having a value if matured.

Is it possible to add sum of pending rewards on address page in this section?

[gaeacodes]

Looking good here, really nice additions!

thanks!

I'm assuming the KMD Available to Claim is for individual utxos?

"KMD Available to Claim" is for individual addresses.All the utxos of a single address are already combined for that element, so the value displayed and reward available to claim are for all the utxos belonging to the single address.

As for the same address, some older tx are zero etc. most recent utxo having a value if matured.

If a utxo is spent we are setting reward to be claimed as 0.
Did you see any older tx that still has to be spent(also have reward to be spend),but the reward is 0?

Is it possible to add sum of pending rewards on address page in this section?

It should be possible.will try!

[smk762]

Did you see any older tx that still has to be spent(also have reward to be spend),but the reward is 0?

I don't believe so, but it was potentially a little confusing seeing same address in some cases having a 0 value, but in others having a non 0 value. Perhaps we should mark the relevant ones as "Rewards: N/A, utxo spent"?

Is there a threshold value for rewards display? I've been comparing rewards value to what is shown in AtomicDEX desktop and noticed a slight difference - have been waiting for some to accrue again to confirm. Example below.

[gaeacodes]

I don't believe so, but it was potentially a little confusing seeing same address in some cases having a 0 value, but in others having a non 0 value. Perhaps we should mark the relevant ones as "Rewards: N/A, utxo spent"?

Will look into it

Is there a threshold value for rewards display? I've been comparing rewards value to what is shown in AtomicDEX desktop and noticed a slight difference - have been waiting for some to accrue again to confirm. Example below.

Will check this too. but, did you refreshed the explorer page just before comparing with atomicdex?
the explorer page doesn't auto update.

[DeckerSU]

My 5 cents: to don't get users confused with KMD Available to Claim: 0 caption, my offer is to change it on Rewards Available and show it only in case if rewards is non-zero, like below.

Before:

<b>KMD Available to Claim:</b> {{vout.totalAvailableRewards}}

After:

<span class="text-muted" data-ng-show="vout.totalAvailableRewards">Rewards Available: {{vout.totalAvailableRewards}} KMD</span>

It's already done on https://explorer.kmd.sh/ instance, so you can see "live-demo".

[smk762]

yep, fresh reload. Noticed a difference in values last night with another address

• Explorer said 1.2530505
• AtomicDEX desktop said 1.26461712

After claiming

• old explorer said FEE: -1.26460712 KMD
• new explorer said FEE: 0.02507101 KMD | REWARD: 1.28967813 KMD (reward - fee = 1.26460712)

Might be due to calculation hours being offset by server/client timezone? FWIW my server is set to UTC time, PC is at GMT+8

[gaeacodes]

yep, fresh reload. Noticed a difference in values last night with another address

Do you remember the address and transaction id by any chance?

[gaeacodes]

@smk762 I wasn't updating the tipTime after the initial start of the explorer.So, most of the calculations were wrong.
Can you please reinstall and test the explorer again?

[gaeacodes]

@smk762 added rewards available to the address summary in address view,limited the reward value to 4 decimals.Please do a fresh install and test when you have time.

[gaeacodes]

http://34.122.103.103:7775/tx/3d5643c80ac4dcc8cb4900508e27abde001affc219cfd12930168a2155a4e2e0
Please take a look at this txn.The user had rewards available but didn't claim them.So, those rewards are being treated as FEE and the rewards are called "Rewards Claimed" in txn summary.I can change the text if needed.

cc: @smk762

[smk762]

Good pickup. Currently it appears to be an exhorbitant fee, perhaps a better display would be to recalculate the fee (as minus the rewards value), and tag the rewards as "Lost rewards", hyperlinked to info on why they didnt get the rewards like https://support.komodoplatform.com/support/solutions/articles/29000024428-komodo-5-active-user-reward-all-you-need-to-know

Rewards value looks ok compared to atomicdex for address RUwodWWAabv3h2jSatVDSMT89cLkTMSutA

I was getting a crash for address RCyANUW2H5985zk8p6NHJfPyNBXnTVzGDh though.

in ~/.komodo/debug.log

2021-02-12 05:49:05 ERROR: GetTransaction: OpenBlockFile failed
2021-02-12 05:49:05 Unable to open file /home/smk762/.komodo/blocks/blk00134.dat
2021-02-12 05:49:05 ERROR: GetTransaction: OpenBlockFile failed
2021-02-12 05:49:05 LevelDB read failure: IO error: /home/smk762/.komodo/blocks/index/457739.ldb: Too many open files
2021-02-12 05:49:05 IO error: /home/smk762/.komodo/blocks/index/457739.ldb: Too many open files
2021-02-12 05:49:05 Unable to open file /home/smk762/.komodo/blocks/blk00134.dat
2021-02-12 05:49:05 ERROR: GetTransaction: OpenBlockFile failed
2021-02-12 05:49:05 Unable to open file /home/smk762/.komodo/blocks/blk00134.dat

In stdout for explorer

[2021-02-12T06:00:49.340Z] info: ::ffff:101.185.133.23 web socket subscribe: sync
[2021-02-12T06:00:49.622Z] info: ::ffff:101.185.133.23 "GET /insight-api-komodo/peer" 304 - 0.649 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" 
[2021-02-12T06:00:49.760Z] info: ::ffff:101.185.133.23 "GET /insight-api-komodo/addr/RLVzC4tr9cNKvuw2z4m8KuMfZURwCehx55/?noTxList=1" 304 - 507.598 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" 
[2021-02-12T06:00:50.178Z] info: ::ffff:101.185.133.23 "GET /insight-api-komodo/txs?address=RLVzC4tr9cNKvuw2z4m8KuMfZURwCehx55&pageNum=0" 200 - 41.257 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" 
[2021-02-12T06:01:07.161Z] info: ::ffff:101.185.133.23 "GET /insight-api-komodo/block/RCyANUW2H5985zk8p6NHJfPyNBXnTVzGDh" 404 9 4.878 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" 
[2021-02-12T06:01:07.521Z] info: ::ffff:101.185.133.23 "GET /insight-api-komodo/tx/RCyANUW2H5985zk8p6NHJfPyNBXnTVzGDh" 400 90 5.141 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" 
[2021-02-12T06:01:08.169Z] info: ::ffff:101.185.133.23 "GET /insight-api-komodo/addr/RCyANUW2H5985zk8p6NHJfPyNBXnTVzGDh/?noTxList=1" 400 53 293.231 "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" 
[2021-02-12T06:01:08.170Z] error: uncaught exception: [Error: Callback was already called.]
[2021-02-12T06:01:08.173Z] error: Error: Callback was already called.
    at /home/smk762/komodo-install-explorer/komodo-explorer/node_modules/bitcore-node-komodo/node_modules/async/lib/async.js:43:36
    at /home/smk762/komodo-install-explorer/komodo-explorer/node_modules/bitcore-node-komodo/node_modules/async/lib/async.js:723:17
    at /home/smk762/komodo-install-explorer/komodo-explorer/node_modules/bitcore-node-komodo/node_modules/async/lib/async.js:167:37
    at /home/smk762/komodo-install-explorer/komodo-explorer/node_modules/bitcore-node-komodo/node_modules/async/lib/async.js:652:25
    at /home/smk762/komodo-install-explorer/komodo-explorer/node_modules/bitcore-node-komodo/lib/services/bitcoind.js:2258:32
    at IncomingMessage.<anonymous> (/home/smk762/komodo-install-explorer/komodo-explorer/node_modules/bitcore-node-komodo/node_modules/bitcoind-rpc/lib/index.js:91:9)
    at emitNone (events.js:72:20)
    at IncomingMessage.emit (events.js:166:7)
    at endReadableNT (_stream_readable.js:923:12)
    at nextTickCallbackWith2Args (node.js:511:9)
[2021-02-12T06:01:08.173Z] info: Beginning shutdown
[2021-02-12T06:01:08.174Z] info: Stopping insight-ui-komodo

Might be a setting or something I need to tweak, strangely not an issue for most other addresses I checked.

[gaeacodes]

tag the rewards as "Lost rewards", hyperlinked to info on why they didnt get the rewards like

will do this

I was getting a crash for address RCyANUW2H5985zk8p6NHJfPyNBXnTVzGDh though.

yes, I was getting similar crashes because of sending too many RPC requests. so, changed async.parallel to async.series. now, such crashes are not happening. but, it still takes too long to display the data. maybe because my server has low thread count/RAM.

here it is: http://34.122.103.103:7775/address/RCyANUW2H5985zk8p6NHJfPyNBXnTVzGDh

I am already returning "not able to estimate rewards" if the address has too many transactions. Example: http://34.122.103.103:7775/address/RD47yEoA4VGriqjeiLdMVKnuE7x7xqFy9r

if you make a fresh installation of the explorer, the crash should go away

[gaeacodes]

@smk762 Displays potential lost rewards when rewards are not claimed and also added the link.

[smk762]

No more crash on RCyANUW2H5985zk8p6NHJfPyNBXnTVzGDh
"Lost reward" message sighted and link operational.

Looking good mate!

I'm about ready to approve this, will give it a day or two for other reviewers to put another set of eyes over it and poke around a bit in case I missed something.

[gaeacodes]

@smk762 Any updates?

[smk762]

Working on my side, and all requested tweaks applied. Thanks @gaeacodes

[ca333]

excellent, reach out on discord ref. bounty - thank you @gaeacodes @smk762 @gcharang

[gaeacodes]

@smk762 @ca333 @DeckerSU @gcharang

Thanks for testing,merging the PR and all the help.
Just a few points of clarification:
1.This PR doesn't deal with difference between smart chains and the KMD chain.If a smart chain explorer is launched using these changes the UI won't be optimal and all the extra computation in the api and bitcore-node is not needed.The best way to use these changes is to create patches like @DeckerSU had done for the intial submitted changes.Here is the link for intial changes: https://github.com/DeckerSU/komodo-explorers-install/tree/e1dc16bf1bc3479085863c55d3f4de0b3a268e0f/rewards
2. Also,this line needs to be changed to Decker's repo after the PR for bitcore node is merged(https://github.com/gaeacodes/bitcore-node-komodo/blob/10b243cc1082c9669d28f62e419d12be12ea5619/lib/scaffold/create.js#L23).I am leaving it like this for now in case someone wants to test these changes from my repos.

[ca333]

@gaeacodes plz reopen PR on https://github.com/KomodoPlatform/insight-ui-komodo/tree/kmd-rewards - yes Decker gave me some good feedback. A few layers should be enhanced - i.e. differentiation between ACs and KMD - spam-protection/prevention ref. rpc interface requests, et cetera.

[ca333]

it will be nice to visualise the reward as a separate input @gaeacodes

example: screenshot from Decker solution proposal

Also - are you able to modify the rpc interface on the komodo side? What we need is adding additional information to the getrawtransaction response - one additional field with the pre-calc of the reward amount. It will prevent 'spamming' rpc interface as it is currently implemented.

special thanks to @DeckerSU for solution-design review

[gaeacodes]

plz reopen PR on https://github.com/KomodoPlatform/insight-ui-komodo/tree/kmd-rewards

will do.

https://user-images.githubusercontent.com/10762374/109789845-08de3280-7c11-11eb-9fce-6cdc89ee141b.png

Will update this view in new PR.

It will prevent 'spamming' rpc interface as it is currently implemented.

With the currently available RPC from komodod, the way I have implemented is the only way to have rewards claim data displayed. But, I agree implementing the reward amount as an output for getrawtransaction RPC will decrease the number of requests and computation being done in the bitcore-node layer.

are you able to modify the rpc interface on the komodo side?

I have some experience with the C programming language and almost none with C++ . But, I will take a look and let you know if I am able to do it.

cc: @ca333