If server closes a connection just after accepting it, the client is stuck

[starius]

package main

import (
	"log"
	"net"
	"sync"

	"github.com/inconshreveable/muxado"
)

func main() {
	a, b := net.Pipe()
	cli := muxado.Client(a, nil)
	ser := muxado.Server(b, nil)
	var wg sync.WaitGroup
	defer wg.Wait()
	wg.Add(1)
	go func() {
		defer wg.Done()
		x, err := cli.Open()
		if err != nil {
			log.Fatalf("cli.Open: %s.", err)
		}
		buf := make([]byte, 5)
		_, err = x.Read(buf)
		log.Printf("x.Read: %s.", err)
	}()
	wg.Add(1)
	go func() {
		defer wg.Done()
		y, err := ser.Accept()
		if err != nil {
			log.Fatalf("ser.Accept: %s.", err)
		}
		y.Close()
	}()
}

If a client sends something over the connection, it works as expected (client gets io.EOF).
If the connection is closed by the client rather than the server (and the server reads), it also works.

[xenoscopic]

Actually, the real bug here is that connections opened on one side won't be accepted on the other side until data is sent. In your example, the Accept call is actually what's blocking - not the client.

I've seen this as well, and in normal usage it's not a problem because most connections are opened and then used to send data, but it's a bit strange. It's also problematic (and this is where I came across it) if you want to open a fixed number of multiplexing channels up-front rather than opening them dynamically - one side will basically become deadlocked waiting for Accept to unblock. The only workaround is to flush each open request with some data after each open call. It's worth noting that the data doesn't have to be sent on the connection that you've opened and want to have accepted - sending data on any connection seems to push the next Accept call to unblock. But it also doesn't seem to push all pending Accept calls to unblock - it's like the open messages are stuck in the client and only flushed to the server one at a time when a write takes place.

[xenoscopic]

Here's an example program demonstrating the behavior in more detail. If you start uncommenting the Write calls one at a time, you'll see that the server starts managing to Accept one more connection at a time. In clarification to my previous comment, I think what's actually happening is that the server is just accepting the stream that does the write, so if c2 writes before c1, I think c2 will be Accepted first. Perhaps the individual streams don't send their open requests until a write occurs. I guess this could be an optimization for streams that never send data, but it breaks the option to Accept multiple streams up-front.

package main

import (
	"fmt"
	"net"

	"github.com/inconshreveable/muxado"
)

func main() {
	// Create a multiplexing setup over an in-memory connection.
	clientConn, serverConn := net.Pipe()
	clientSession := muxado.Client(clientConn, nil)
	serverSession := muxado.Server(serverConn, nil)

	// Accept connections on a separate Goroutine and track when it completes.
	done := make(chan struct{})
	go func() {
		c1, err := serverSession.Accept()
		fmt.Println("server accept 1", c1 != nil, err)
		c2, err := serverSession.Accept()
		fmt.Println("server accept 2", c2 != nil, err)
		c3, err := serverSession.Accept()
		fmt.Println("server accept 3", c3 != nil, err)
		close(done)
	}()

	// Open connections.
	c1, err := clientSession.Open()
	fmt.Println("client open 1", c1 != nil, err)
	c2, err := clientSession.Open()
	fmt.Println("client open 2", c2 != nil, err)
	c3, err := clientSession.Open()
	fmt.Println("client open 3", c3 != nil, err)

	// (Optionally) Write data to try to flush connection opening.
	// c1.Write([]byte("some data"))
	// c2.Write([]byte("some data"))
	// c3.Write([]byte("some data"))

	// Wait for the acceptor Goroutine to complete.
	<-done
	fmt.Println("All done!")
}

[xenoscopic]

And as one last follow-up, I think that's exactly what's happening, because I think the SYN flag is sent on the first write:

muxado/stream.go

Line 252 in fc182d9

var synFlag bool