proptest_state_machine over PeerMessage

[skaunov]

It's rough yet, and I'd appreciate review of the main approaches in it while I'm dealing with TODO and groom the whole contribution.

I understood @aszepieniec that the interactions are subject to change, so currently it's only outlined how an interaction outcome actually asserted (messages the peer-loop sends to the main, and checking standing). I can only repeat current behavior there, and if there's ideas to tune/amend that I guess it would be right place and time for that.

Another decision pending is weighting PeerMessage variants. I guess it's important to add random "bad" once a while, but uniform distribution washes away valid interactions to the margin. I can only roughly guess the distribution you might see fit, so it's the other open question.

[aszepieniec]

I can only repeat current behavior there, and if there's ideas to tune/amend that I guess it would be right place and time for that.

I wonder if the state machine can be factored out such that:

• the actual peer message handlers are a thin wrapper for dealing with network transmissions, but on the inside invoke the state machine;
• the state machine can be tested in isolation of network effects (no-io).

If that's where this is headed, please point me to the location in the code where I can read it.

Another decision pending is weighting PeerMessage variants. I guess it's important to add random "bad" once a while, but uniform distribution washes away valid interactions to the margin. I can only roughly guess the distribution you might see fit, so it's the other open question.

I don't know off the top of my head what the right weights are. Consider writing a patch for master that will record and report relative frequencies of PeerMessage types. I'll be happy to run that to find out what the frequencies are in practice.

[skaunov]

@aszepieniec ,

I can only repeat current behavior there, and if there's ideas to tune/amend that I guess it would be right place and time for that.

I wonder if the state machine can be factored out such that:

* the actual peer message handlers are a thin wrapper for dealing with network transmissions, but on the inside invoke the state machine;

* the state machine can be tested in isolation of network effects (no-io).

If that's where this is headed, please point me to the location in the code where I can read it.

The state machine you're talking about should then be an implementation under testing, not the reference one. I'd be happy to convert something to this approach, but PeerMessage has quite a variability. Do you have a smaller one object to start with on your mind?

Current implementation indeed heading the way you described. It process the PeerMessage at https://github.com/skaunov/neptune-core/blob/fa768a9db2f891b4531a7973c8982cf100a3171a/src/models/peer/tests/automaton.rs#L166 and then checks the main loop channel and standing. (But is checking for nothing yet: I draft where and how it should be done, and am asking for what to actually check.)

What do you think about not creating proptest-state-machine for PeerMessage at all and instead implement Strategy for [Mock]

neptune-core/src/tests/shared.rs

Line 365 in 73ec193

pub struct Mock<Item> {

and some #[test] inside a proptest!? That feels more elegant since state machine brings benefits when there are actual state changes and we're checking it can't find itself in a wrong one. And as with PeerMessage it's by design looping in the single state that makes inevitable proptest-state-machine managing code just a boilerplate. =(

Another decision pending is weighting PeerMessage variants. I guess it's important to add random "bad" once a while, but uniform distribution washes away valid interactions to the margin. I can only roughly guess the distribution you might see fit, so it's the other open question.

I don't know off the top of my head what the right weights are. Consider writing a patch for master that will record and report relative frequencies of PeerMessage types. I'll be happy to run that to find out what the frequencies are in practice.

I don't see the point repeating the real freqs in Strategy<PeerMessage> since the whole idea is to give attention to those rare cases which aren't happen there. I would propose then just depress the bad messages to happen once in a 100(?) messages on average. And those bad messages are complete gibberish so I guess those are not very useful, but to have few of those is good for testing.