Cannot push value onto an ended pushable (it-queueless-pushable)

[NiKrause]

libp2p@2.8.2
helia@5.3.0
orbitdb@2.5.0

Severity: Critical - System crash

Description:

I am running an OrbitDB via Helia in a browser which replicates data to a NodeJS Voyager, which crashed during replication. Meaning the browser opens a stream to the peer which then throws an error inside "it-queuless-pushable: Cannot push value onto an ended pushable".

The NodeJS app then crashes.

Replication is difficult, it happens every now and then. It happened now within one hour twice while debugging a replication issue in the peer-to-peer web app. I did a lot of browser reloads, which probably stopped and restarted a lot of streams in a row. Browser and NodeJS peer on localhost.

       file:///Users/xyz/voyager/node_modules/it-queueless-pushable/dist/src/index.js:99
            throw new Error('Cannot push value onto an ended pushable');
                  ^

Error: Cannot push value onto an ended pushable
    at QueuelessPushable._push (file:///Users/xyz/voyager/node_modules/it-queueless-pushable/dist/src/index.js:99:19)
    at QueuelessPushable.push (file:///Users/xyz/voyager/node_modules/it-queueless-pushable/dist/src/index.js:86:20)
    at duplex.sink (file:///Users/xyz/voyager/node_modules/it-byte-stream/dist/src/index.js:54:25)
    at async handleMessages (file://Users/xyz/voyager/src/lib/host.js:26:5)

[haydenyoung]

It can be difficult to address these issues without being able to replicate consistently. We are looking to set up some stress testing on Voyager, so hopefully this will make replicating the problem more consistent.

If you are able to set up any examples, tests or demos that would be a great help.

[NiKrause]

I was now adding 7 different db's in a row to voyager. I added the 8th and received again this error.
It crashed voyager app.

I restarted voyager and did the same procedure again without any issue first but then crashed again. It seems somewhat reproducible now:

Interestingly I got:
voyager:host:error undefined All promises were rejected +0ms <-- 14 times!

I am using on both sides:

libp2p@2.8.2
helia@5.3.0
it-all@3.0.7
it-pipe@3.0.1

I suspect a "crashed" (inaccessible db) which sort of became inaccessible during a broken stream while replicating. (vpn drop, wifi drop or similar).

I recommend to fix logging when trying to replicate the issue: #91.

Another try to replicate the crash with a brand new voyager and the same db's in browser was not possible.

An interesting test on how to replicate the issue would be the following

1. Create a huge database with about 100k oder 1000k records replicate them to a second orbitdb instance.
2. Make a change on one side
3. While replication is going on switch of the network between them