Investigate and fix startup issues

[HenrikJannsen]

Many users experience issues when starting up Bisq.

Here is an overview, analysis and suggestions how to fix some of the issues.

Test setup:

• Using a fresh data directory with release 1.9.15 (released Mar 27, now its May 24 -> about 2 months).
• No local Bitcoin node running
• Fast internet connection

I will post each test run as a new post to not overload the content here.

Actions:

BitcoinJ

• Check timeout, TTL at BitcoinJ
• Check Waiting time at new connection attempt. It seems its increasing with each attempt from 1 sec to 5 sec).
• Check unresolved host errors: Announcing to emzybtc5bnpb2o6gh54oquiox54o4r7yn4a2wiiwzrjonlouaibm2zid.onion/<unresolved>:8333 as: /bitcoinj:0.15.10.bisq.16/Bisq:1.9.15/. /<unresolved> is just a log at InetSocketAddress as addr is null.
• It seems the reconnect attempts are reusing failed connections. Should use other nodes first before retry on failed ones.
• Series of May-24 22:09:53.384 [PeerGroup Thread] INFO org.bitcoinj.core.PeerGroup: Waiting 78610 ms before next connect attempt to [emzybtc454ewbviqnmgtgx3rgublsgkk23r4onbhidcv36wremue4kqd.onion]:8333 logs.
• ConnectTimeout at BlockingClient is 120 sec. Maybe increase.
• Add monitor for BTC nodes.
• Check if some nodes are offline. rfqmn3qe36uaptkxhdvi74p4hyrzhir6vhmzb2hqryxodig4gue2zbyd and azbpsh4arqlm6442wfimy7qr65bmha2zhgjg7wbaji6vvaug53hur2qd seems like having constant issues.
• Consider to remove nodes from inactive contributors.

P2P network

• Increase max Items per type limit from 5000 to 10000 or 20000: DONE with Increase GetDataRequestHandler.MAX_ENTRIES to 20 000. #7113
• Potential bug from warn log : May-24 21:35:06.520 [JavaFX Application Thread] WARN b.n.p2p.peers.getdata.RequestDataManager: We have started already a requestDataHandshake to peer. nodeAddress=sn3emzy56u3mxzsr4geysc52feoq5qt7ja56km6gygwnszkshunn2sid.onion:8000
  We start a cleanup timer if the handler has not closed by itself in between 2 minutes.
• Add duration for request-response pairs in logs

[HenrikJannsen]

Test run 1

Startup sequence

May-24 21:32:29.211: Starting tor

May-24 21:33:01.044: Tor started after 31833 ms

May-24 21:33:01.202: We send a PreliminaryGetDataRequest to peer devinsn3xu...

May-24 21:33:01.349: We send a PreliminaryGetDataRequest to peer sn3emzy56u... (475.525 kB)

May-24 21:33:01.896 Read 411 checkpoints up to time 2024-02-02
May-24 21:33:01.898 chain head is at height 828576 (02-02)
May-24 21:33:01.945 Attempting connection to ... 7 BTC nodes

May-24 21:33:42.305 Tor hidden service published after 41258 ms.

May-24 21:33:51.181 Socket creation to peersNodeAddress sn3emzy56u3mxzsr4geysc52feoq5qt7ja56km6gygwnszkshunn2sid.onion:8000 took 49824 ms

May-24 21:33:52.297
6 BTC node connections failed: Connect timed out, TTL expired, Host unreachable (after 50 sec)
New connection attempts after 1-5 sec delays.

May-24 21:35:03.636: Received a GetDataResponse with 5.019 MB from sn3emzy56u....
Large items:
MailboxStoragePayload: 956 / 3.639 MB
AccountAgeWitness: 4628 / 140.105 kB

May-24 21:35:04.629:We send a GetUpdatedDataRequest to peer sn3emzy56u3mxzsr... (634.982 kB)

Series of Bitcoinj timeouts (7 nodes)
May-24 21:35:06.952 PeerSocketHandler: [emzybtc3ewh7zihpkdvuwlgxrhzcxy2p5fvjggp7ngjbxcytxvt4rjid.onion]:8333: Timed out
Reconnections...
May-24 21:35:06.967 [PeerGroup Thread] INFO org.bitcoinj.core.PeerGroup: Waiting 10000 ms before next connect attempt to ...

May-24 21:35:54.211 Received a GetDataResponse with 2.377 MB from sn3emzy56u3mxzsr4geys...
Large items:
MailboxStoragePayload: 329 / 1.435 MB
AccountAgeWitness: 4632 / 140.227 kB -> hit 5000 limit but not size limit

May-24 21:35:56.500 We send a GetUpdatedDataRequest to peer sn3emzy56u3m... (765.042 kB)
May-24 21:36:18.242 Received a GetDataResponse with 531.652 kB
AccountAgeWitness: 2840 / 85.977 kB
TradeStatistics3: 1837 / 109.862 kB

May-24 21:36:20.354 We send a GetUpdatedDataRequest to peer sn3emzy56u3m.. (875.037 kB)

May-24 21:36:29.214 [JavaFX Application Thread] WARN bisq.core.app.BisqSetup: startupTimeout called -> displays Tor debug window

May-24 21:36:35.609 Received a GetDataResponse with 192.045 kB from sn3emzy56u3mxzsr...

May-24 21:36:35.651 [JavaFX Application Thread] INFO b.n.p2p.peers.getdata.RequestDataHandler:
MailboxStoragePayload: 2 / 6.385 kB
TradeStatistics3: 3000 / 179.761 kB -> was already increased from 3000 to 15000

May-24 21:38:40.520 Received a GetDataResponse with 1.845 kB (all data received) from devinsn3xuzxhj6...

After that logs get filled up with BitcoinJ issues...

BTC sync stuck at 99% at May-24 22:12 -> more then 35 min after start.
DAO sync not started (showing synchronizing DAO)

Full logs:
bisq.log