Performance benchmarks

[cnudroid]

@steely-glint when I try to send multiple messages with sizes higher than 20-50KB back and forth, I see that the latency is high.
I am wondering if there are some benchmarks done and how are the numbers compared to TCP.

[steely-glint]

I haven't done any throughput benchmarking on the stack.
I know @lgrahl has done some general SCTP benchmarking.
This stack was built to run on small linux machines, so doesn't allocate much buffer. I imagine you are hitting that issue.

If you do a PR to include performance tests I'd be happy to take a look.

[cnudroid]

I created a fork and committed the code to use sctp with bouncy castle dtls transport.

here is the repo:
https://github.com/cnudroid/sctp4j

The code has a MockClient and MockServer.

To start the server, run the DTLSServerTest class which is under test/java/com.env.sctp4j.test.sctp.dtls
To start the client, run the StartDltsClient class which is under the same package as above.

Couple of changes I did in the sctp4j code:

• In BlockingSCTPStream, created a static ExecutorService with a pool of 100 threads.
• The Capacity of DataChunk is increased to 1024

Usage:

To send a message, I am creating a making new SCTPScream, new SCTPMsg, and finally threadAssoc.sendAndBlock()

      SCTPStream result1 = clientAssoc.mkStream(SecureRandom.getInstanceStrong().nextInt());
      SCTPMessage sctpMsg1 = new SCTPMessage(msgBytes, result1);
      clientAssoc.sendAndBlock(sctpMsg1);

Observations:

• when i run StartDltsClient with scheduleMultipleMsgsConcurrently(), the server is not receiving the msg.

[cnudroid]

client.log
server.log

Here are the logs from client and server.

[cnudroid]

@steely-glint also another observation is I am not able to multiple SCTPMessages on a single SCTP Stream. My understanding is we should be able to send multiple messages in a single SCTPStream.

[steely-glint]

I suspect the problem is that we assume that you'll be using WebRTC's DCEP to create the channels.
DCEP has a side effect of setting the message handling behaviour.
Without that you'll get the default - which is clearly not working for you.
I'll take a look at your fork.

[cnudroid]

Thanks.

…

On Thu, Nov 5, 2020 at 12:00 PM Tim Panton ***@***.***> wrote: I suspect the problem is that we assume that you'll be using WebRTC's DCEP to create the channels. DCEP has a side effect of setting the message handling behaviour. Without that you'll get the default - which is clearly not working for you. I'll take a look at your fork. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARD3R7MUVMOGDAW3ZKHKW73SOL75JANCNFSM4TILH3WA> .

[steely-glint]

I changed your client code, to use stream.send() This ensures that the association actually makes the message. If you don't do this then the sequence numbers on the SCTP messages aren't set, so the stream can't be ordered. With no DCEP (see above) the default is 'ordered+reliable' messages.
So messages on a stream are delivered in sequence and whole !

Here's a diff
- SCTPMessage sctpMsg1 = new SCTPMessage(msgBytes, result1);

- clientAssoc.sendAndBlock(sctpMsg1);

+ result1.send(msgBytes);

[cnudroid]

thanks. so far looks good. I will run some more tests and let you know.

[cnudroid]

update from latest runs :

• when I schedule to send 1MB of data at a fixed rate both from client and server, I see that the data is being stuck after a certain time.
  Here is the log :

"DEBUG: 1604537832345 pool-3-thread-2->MaySend false rwnd = 0 cwnd = -740 sz = 1024"

To overcome this, I tried to set rwnd value to transportMTU when it's 0 in sackDeal(SackChunk sack) of ThreadAssociation.

_rwnd = sack.getArWin() - totalDataInFlight; if(_rwnd == 0){ _rwnd = _transpMTU; }

After this, I could see the messages delivered back and forth.
We don't think this is the right fix.

• I increased the size of _freeBlocks (ArrayBlockingQueue) to 1000.

Post these changes, I did a comparison between TCP transport( Websocket over HTTP/TCP) and this SCTP over UDP and observed that TCP seems to be faster.
I expected this protocol with a single stream to perform better than TCP.
What are your thoughts?

[steely-glint]

The stack shouldn't stall (unless the consume side stalls of course) If you can produce a test case that replicates this I'd be happy to take a look.

I'm mostly using this in the context of webRTC, where the browsers tend to throttle throughput anyhow to stay within their bandwidth limits, so I have not done this sort of performance test. It is great that you are doing so!

The default SCTP congestion control is essentially the same as TCP - so I'd expect them to be comparable.
SCTP does allow you to replace the congestion control and get different characteristics. I think @lgrahl has done some experiments in that area on a different SCTP implementation.

[cnudroid]

@steely-glint, I pushed my changes into the forked repo https://github.com/cnudroid/sctp4j
To reproduce the issue, we need to run the main method of DTLSServerTest and StartDltsClient classes.
I used Java 11 as a runtime environment.

The communication stops after the client receives the first data.

To have continued communication, I have to uncomment the following code in ThreadAssocation around line 512.
// if(_rwnd == 0){
// _rwnd = _transpMTU;
// }

Also, is it recommended to new stream for every independent SCTPMessage to avoid congestion?

[cnudroid]

@steely-glint, Did you get a chance to look at this?
Let me know if you have any issues in reproducing.

[steely-glint]

@cnudroid - sorry, I've been working to a deadline on a paid project, so not had time to look at this yet.