Interpretting your server benchmarks

[iselind]

I'm having a hard time interpretting your server benchmarks. As i interpret the numbers, aioftp is a lot slower than pyftpdlib it's being compared to. pyftpdlib as i gather doesn't work in an asynchronous way.

Am i interpretting the numbers correctly? If so, why is aioftp so much worse?

[pohmelie]

@iselind
First of all, pyftpdlib is asynchronous, but it can't be used with asyncio right now (but they have uncompleted PR). pyftpdlib also use many things to speed up work. aioftp was started as "missing asyncio ftp client" library and more later server side was implemented. It use many high-level asyncio abstractions, so bottle neck is somewhere there.
I will try to explain benchmark results.
STOR/RETR: aioftp bottleneck here is disk io. Definitely, pyftpdlib use memory cache and/or some optimizations, since you can't write to disk faster, than it limit. So, there is some "cheating", pyftpdlib said, that everything is done, your file is on disk, but actually they are storing foreground.
LOGIN/QUIT: only high-level streams overhead here. Timings are pretty similar.
Here is benchmark result for in-memory storage:

STOR (client -> server)                              433.46 MB/sec
RETR (server -> client)                              493.69 MB/sec
200 concurrent clients (connect, login)                0.17 secs
STOR (1 file with 200 idle clients)                  430.04 MB/sec
RETR (1 file with 200 idle clients)                  517.03 MB/sec
200 concurrent clients (RETR 10.0M file)              10.74 secs
200 concurrent clients (STOR 10.0M file)              13.60 secs
200 concurrent clients (QUIT)                          0.04 secs

This numbers shows aioftp python bottleneck, price for using high-level asyncio api and many abstraction layers.

[iselind]

Thanks a lot for your quick reply.

Because pyftpdlib cheat by using a memory cache, then the data isn't secure on the drive when the call completes. If the computer suffer a power outage here i'll lose data, sounds like a bad implementation. aioftp doesn't have this flaw i guess?

I'm looking for a ftp client, preferably usable in a asyncio context. I guess pyftpdlib isn't such a great choice as it seems to be a server only. Are there any other FTP clients than aioftp that you know of? Google only show aioftp when i search 'python asyncio ftp client'.

[pohmelie]

aioftp doesn't have this flaw i guess?

Yes.

pyftpdlib cheat

This is "cheat" for benchmark. For users caching is very good thing, since you can get your data faster (from cache).

Are there any other FTP clients than aioftp that you know of?

Unfortunately, I don't know any other asyncio ftp client. That was the main reason to make aioftp library.

[pohmelie]

Some new test for versoin 0.8.1 with in-memory storage:
CPython 3.6

STOR (client -> server)                              366.06 MB/sec
RETR (server -> client)                              516.45 MB/sec
200 concurrent clients (connect, login)                0.12 secs
STOR (1 file with 200 idle clients)                  406.52 MB/sec
RETR (1 file with 200 idle clients)                  518.47 MB/sec
200 concurrent clients (RETR 10.0M file)              10.41 secs
200 concurrent clients (STOR 10.0M file)              13.45 secs
200 concurrent clients (QUIT)                          0.03 secs

CPython 3.6 + uvloop

STOR (client -> server)                              418.34 MB/sec
RETR (server -> client)                              626.34 MB/sec
200 concurrent clients (connect, login)                0.09 secs
STOR (1 file with 200 idle clients)                  418.53 MB/sec
RETR (1 file with 200 idle clients)                  627.51 MB/sec
200 concurrent clients (RETR 10.0M file)               2.08 secs
200 concurrent clients (STOR 10.0M file)               5.41 secs
200 concurrent clients (QUIT)                          0.03 secs

PyPy 3.5-5.9.0

STOR (client -> server)                              262.79 MB/sec
RETR (server -> client)                              863.06 MB/sec
200 concurrent clients (connect, login)                0.14 secs
STOR (1 file with 200 idle clients)                  268.81 MB/sec
RETR (1 file with 200 idle clients)                  890.49 MB/sec
200 concurrent clients (RETR 10.0M file)               1.56 secs
200 concurrent clients (STOR 10.0M file)               5.93 secs
200 concurrent clients (QUIT)                          0.04 secs