Skip to content

Commit a478ce7

Browse files
authored
Add a (slightly simplified) state graph for pingpong's 1st BFS. (#23)
* Add a (slightly simplified) state graph for PingPong's 1st BFS. The simplifications are: - send 2 commands rather than 10 - omit some details about Node superclass, ClientWorker.
1 parent e4bd6de commit a478ce7

2 files changed

Lines changed: 173 additions & 0 deletions

File tree

labs/lab0-pingpong/README.md

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -466,3 +466,176 @@ sent). Catching this is simpler than it might be because we assume that the
466466
client only sends one ping at a time, and waits for the previous ping to be
467467
acknowledged before sending a different ping. What should be the client's
468468
behavior when it receives a duplicate (late) message?
469+
470+
471+
### More on Search Tests
472+
473+
Now we will explore how the search tests work in more detail. This will help you
474+
to understand errors that you see so that you can modify your implementation to
475+
work best with the model checker. To do this, we'll see how the search test
476+
checks the correctness of our ping-pong implementation. Before continuing, you
477+
should understand the ping-pong protocol. You should also review the "Search
478+
Tests" section of the [top-level README](../../README.md).
479+
480+
At this point, we've told you that search tests check the correctness of the
481+
implementation by exploring the state graph (a directed graph where the vertices
482+
are states of the system and there is an edge from state `u` to `v` if there is
483+
a message or timer which can be delivered in state `u` to reach state `v`). The
484+
state of the system consists of three parts: (1) the states of the nodes in the
485+
system, (2) the queue of timers pending for each node, and (3) the state of the
486+
network. Let's describe each of these in more detail.
487+
488+
First, the state of the nodes is relatively self-explanatory: it consists of all
489+
`Node` objects involved in the system. Next, the timers pending for each node
490+
can be seen as a queue where the queue ordering respects durations (as described
491+
in the top-level README). What's left is the state of the network, and here
492+
things differ from the view presented in the visual debugger. Specifically, in
493+
the visual debugger, when a message is duplicated, a new copy of the message
494+
appears. As messages can be duplicated any number of times, if the network state
495+
included the number of duplicates present in the system, then the state graph
496+
would be infinite. To avoid this, in search tests, the state of the network is
497+
represented by a set of all messages which have been sent. Messages are not
498+
removed from this set when they are delivered. This incorporates duplications,
499+
delays, and drops: once a message is sent, it can be delivered 0 times (drops),
500+
1 time, or many times (duplicates), and it can be delivered after other messages
501+
and timers have been delivered (delays).
502+
503+
Now we can proceed to the example. The search test for lab 0 has a single server
504+
and a single client which sends 10 different `Ping`s. The test first attempts to
505+
find a sequence of events that leads to the client receiving `Pong`s for all of
506+
its `Ping`s. It then explores the entire state graph and checks that the `Pong`s
507+
received by the client match the `Ping`s sent. To simplify the example, we'll
508+
consider a client which only sends two `Ping`s with values "ping-1" and
509+
"ping-2". This is the search that would occur if you replace the first line of
510+
the search test in [the test file](./tst/dslabs/pingpong/PingTest.java) with
511+
512+
```java
513+
initSearchState.addClientWorker(client(1), repeatedPings(2));
514+
```
515+
516+
Let's explore what happens when we run the first BFS which looks for a state
517+
where the client has received results for all of its sent commands. To begin, we
518+
show the states and edges of this state graph, and then we explain the graph in
519+
more detail.
520+
521+
![State Graph](./img/state-graph.png)
522+
523+
The vertex labels in the state graph describe the state of the nodes, timer
524+
queues, and network.
525+
* Recall that the state of a node (server/client) is simply the fields of the
526+
server/client object. The `PingServer` has an `app` field, while the
527+
`PingClient` has a `serverAddress`, `ping`, and `pong` field. The first two
528+
lines describe the state of the two nodes in the system (we omit the fields in
529+
the `Node` superclass as those details are not important to us).
530+
* The next two lines describe the client and server timer queues. These are
531+
given as a list where the list ordering must respect durations.
532+
* The last line(s) store the state of the network, all messages ever sent. These
533+
describe both the message contents and the sender/receiver of the message.
534+
535+
The very first state in the graph is the system's initial state -- that is, the
536+
state after all nodes have been initialized and the client has been told to send
537+
the first command (via `sendCommand(Ping("ping-1"))`). At this point, the client
538+
has sent a `PingRequest` to the server and has set a `PingTimer`, but it has not
539+
yet received any results. After construction and initialization, the server's
540+
state is rather boring: it has a `PingApplication` with no fields, and it
541+
doesn't set any timers in `init()`.
542+
543+
From this starting state there are two possible events: we can fire the timer,
544+
or deliver the message.
545+
* If we fire the `PingTimer`, then the client will see that the timer matches
546+
the last sent `Ping` and there is no `Pong` received for the `Ping`, so it
547+
will resend the `PingRequest` to the server and reset the timer. Thus, the
548+
client's timer queue again has `[PingTimer(Ping("ping-1"))]`. Moreover, the
549+
network set already has this `PingRequest` from client to server, so the
550+
network set doesn't change either. Therefore, we return to the initial state.
551+
* If we deliver the `PingRequest`, the server executes the request and replies
552+
with a `PongReply`. The server's state and timer queue remain the same, but
553+
now the network has the `PongReply` that was sent from server to client. The
554+
original `PingRequest` remains in the network and can be delivered again.
555+
556+
This explains all of the edges from the starting state, and it explains how we
557+
reach the second state in the graph. The client is still waiting for results, so
558+
the BFS continues.
559+
560+
From this state, we can fire the timer or deliver either of the messages.
561+
* **Test your understanding:** you should now be able to explain why firing the
562+
`PingTimer` or delivering the `PingRequest` returns us to the same state.
563+
* When delivering the `PongReply`, many things happen.
564+
* The client determines that the `PongReply` is a reply for the current
565+
command, so it updates its `pong` field to match.
566+
* The framework determines that the client received a result (via
567+
`hasResult()`) and saves the result (via `getResult()`) in the `results`
568+
list.
569+
* Since the client received a result, the framework tells the client to send
570+
the next command (`sendCommand(Ping("ping-2"))`).
571+
* The client updates its `ping` field to this new command, and sets its
572+
`pong` field to null.
573+
* The client sends a `PingRequest` for this command to the server.
574+
* The client sets a `PingTimer` for this command. Since the timer ordering
575+
must respect durations, `PingTimer(Ping("ping-1"))` must fire before
576+
`PingTimer(Ping("ping-2"))`.
577+
578+
So, we've explained the edges leaving the second state, and we've explained why
579+
the third state is different from the second. In the third state, the client is
580+
still waiting for a result, and all received results match the expected results.
581+
Therefore, the BFS continues.
582+
583+
We can fire the first timer in queue or deliver any of the messages.
584+
* If we fire the first `PingTimer` in queue, then the client will recognize that
585+
the timer is out-of-date and therefore drop it. No other actions will be
586+
taken. So, we get a new state which is almost the same as the original, but
587+
the new client queue has only the second timer.
588+
* If we deliver `ping-1`'s `PingRequest` or `PongReply`, then we return to the
589+
same state. Once again, you should be able to explain why this is the case.
590+
* If we deliver the `PingRequest` for `ping-2`, then the server executes the
591+
`Ping` and replies. Thus, node states and timer queues remain the same, but
592+
the network now has a `PongReply` for `ping-2`.
593+
594+
The BFS continues on in this fashion until it either exhausts all edges, finds
595+
an invariant violation (a state where the `Pong` returned doesn't match the
596+
`Ping` sent), or finds a state matching our goal.
597+
598+
In this case, the BFS finishes when it finds the state in the bottom-right,
599+
where the client is done and has received `Pong`s for all `Ping`s in its
600+
workload. All received results match the expected results. So, the BFS finishes
601+
and we have found a state matching the goal. At this point, we know that our
602+
system can reach a result. We also are more confident in the safety of our
603+
system, since in the states we examined, we checked that the received results
604+
matched the expected results.
605+
606+
Here are some questions you can use to test your understanding of search tests:
607+
* Consider the modification outlined earlier in this section: in
608+
`handlePongReply`, the client doesn't check that the pong value matches the
609+
ping value. Carry out the BFS by hand as we did above, and give the state at
610+
which an invariant violation occurs.
611+
* From the correct solution (the initially provided one), comment out the line
612+
`send(new PongReply(p), sender)` in the server's `handlePingRequest`. Now, it
613+
should be impossible for the client to receive results.
614+
* First, the theory: write out the state graph for this modified solution. How
615+
can the search test tell that the client's workload cannot be finished?
616+
* Now, the practice: run the test and see the error that the search test gives
617+
you.
618+
* Optionally, open the visual debugger and try to deliver messages to get a
619+
result to the client. For this simple liveness issue, it may seem
620+
unnecessary to use the visual debugger, but for the more complicated
621+
protocols you will work with, the debugger can help you to see where your
622+
implementation differs from your protocol design (or a situation you
623+
missed when designing the protocol).
624+
* The test in lab 0 has another search:
625+
```java
626+
searchSettings.clearGoals().addPrune(CLIENTS_DONE)
627+
bfs(initSearchState);
628+
assertSpaceExhausted();
629+
```
630+
631+
* This will run the BFS as described earlier, checking that the results match
632+
the expected results, but this test does not have a goal. When the search
633+
reaches a state where the client is done, the test will not end; instead, it
634+
will continue examining any remaining states where the client is not done.
635+
That is to say, the test "prunes" any parts of the search space which follow a
636+
state where the clients are finished. The test asserts that it can exhaust the
637+
search space; in other words, that in the time allotted, it can examine the
638+
entire graph (except the states which were pruned).
639+
640+
Carry out this pruned BFS by hand (still assuming that the client sends only 2
641+
commands rather than 10), and confirm that you can exhaust the search space.
356 KB
Loading

0 commit comments

Comments
 (0)