*** Do you want consistent data from binance? ***

[oliver-zehentleitner]

Its important to understand, that the data between a disconnect and a reconnect is LOST!

And reconnects take place! Regarding to https://github.com/binance-exchange/binance-official-api-docs/blob/master/web-socket-streams.md#general-wss-information disconnects have to be expected every 24h.

A single connection to stream.binance.com is only valid for 24 hours; expect to be disconnected at the 24 hour mark

So you have to find a strategy for that!

Ideas to handle this are:

• replace_stream(): start a new stream with the same settings and stop the old one, as soon the new received its first data. This could be scheduled and made automatic, but therefore we have to write a new method, which is not done yet!
• stream the same data through 2 ident websocket connections on one system or better, on different systems
• download missing data via REST after a reconnect.

It would be cool to discuss this as a community, do brainstorming and define a best practise solution. Then we can extend the library with helping features to implement the best practise solution.

I look forward to good suggestions!

Best regards,
Oliver

[DaWe35]

(typo: reconnect ist LOST!˙)
How often does disconnection occur?

[oliver-zehentleitner]

Thanks, fixed typo and added info about disconnect interval to OP.

[daverio]

(just to say, great work!)
I think there is no "best practice" and that both:

1. a way to get back missing data from the rest api. But dont know if this should be part of this library or more at the level of a data management module (which put the data in buffers or a database)
2. a way to insure that there is a little as possible "blind" period (basically your solution 1 or 2)

Both are important. The first one is important to manage a "history", which should have as little as possible missing data. Then the second is important for high frequency, if you get blind, and that your algo is currently executing some trades, that might result on bad behavior (evidently a good algo should have logics to manage this kind of periods), even if the blind period is quite short!

My preference would two ident websockets, I think it is safer as if one of the processes have some issue, the data can always be retrieved from the second. But it does not prevent the case where both are disconnected...

But at the end, I think its pretty impossible to have a system with 0% of missing data!

[oliver-zehentleitner]

thank you for your opinion! maybe you are right and the solution to the problem should not be integrated directly into this module, but the necessity for a good solution is actually there for everyone who uses this lib. we could also solve and share this as an extra module.

@oliver-zehentleitner
Not sure wether or not I should ask my question here
I was thinking of using replace_stream() to have consistent data. I took a look at the source code but I couldn't figure it out completely. My question is that if I start a new stream using this replace_stream() mechanism, will the first data that I receive in the new stream and the old stream be duplicates? I mean, when I pop the data using pop_stream_data_from_stream_buffer(), will their be duplicate data?
thanks

[oliver-zehentleitner]

Yes it will be a duplicate!