Cannot run agoo benchmark

[ioquatix]

koyoko% ruby ./agoo_bench.rb
Traceback (most recent call last):
./agoo_bench.rb:6:in `<main>': undefined method `new' for Agoo::Server:Module (NoMethodError)

[ioquatix]

Maybe it's related to ohler55/agoo#20

[ohler55]

Agoo changes as of version 2.0.0 so that the server is the Agoo::Server module. I have some examples on how to start I with and without rackup. For basic Rack with no static asset serving it is just

#config.ru
class FlyHandler
  def call(req)
    [ 200, { }, [ "flying fish" ] ]
  end
end
Rack::Handler::Agoo.run(FlyHandler.new)

Start with

ruby -r agoo config.ru

Of course there are additional options to run but that is the basic.

[ioquatix]

• So it's not possible to have multiple servers?

• Can you make something similar to falcon serve which loads the config.ru file? The config.ru you give above won't work with other servers.

[ioquatix]

Oh, I guess I can use rackup --server agoo?

[ohler55]

yes, you can. I'll post a revised benchmark file here in a moment.

[ioquatix]

I thought I'd give you some additional feedback.

Running the benchmark middleware (hello.rb)

• Agoo with something similar to agoo_bench.rb had amazing performance with something like 130,000 requests/s. It's not the best I've seen, considering native implementations of web servers, but it's certainly very good.

• Falcon had about 60,000 requests/second running the config.ru file. Using htop shows it using all 8 cores. Falcon is pure ruby by the way, so this is no small achievement.

• Agoo (using rackup) only had 9000 requests/second using the config.ru file. Using htop makes it look like only using one core. This is a similar limitation to falcon when using rackup so I'm sure we can improve this.

[ioquatix]

I'd suggest that we try to test all servers on a level playing field, using rackup. We should be able to provide tuning options through rackup. It should have good defaults.

[ohler55]

Agoo does not yet have clustering. That is in the works but not there yet. I know that is needed to scale the Ruby calls. Agoo has raw speed a single core down fairly well but it lacks the clustering option of Falcon. You must have a nice machine to have 8 cores 😄

I am fine using rackup but I'd also like to test the static file options for those servers that support it. Agoo now has an option to specify assets directories so even with rack it can serve static assets. For benchmarking it would be nice to show.

• straight rackup with clustering
• rackup with out clustering
• static file serving if the server supports that

I think arguments could be made for each of those modes. What do you think?

[ioquatix]

Only you can make a decision what works best in terms of options.

The defacto standard for static assets is to use public/. It's the default for passenger, puma, nginx+passenger, etc.

There is very little reason to change it IMHO, but having it as an option wouldn't hurt.

rackup is kind of shitty.

It works, and you should support it, but I don't know if you can expect great performance from it.

rackup inserts the lint middleware which makes it slow by default.

I don't think it's stupid that agoo executed by itself defaults to serving the application in config.ru in the current directory. I should probably make that the default in falcon too.

[ohler55]

I'm a bit surprised you are only getting 9K req/sec though. Are you ubuntu or OS X? OS X is pretty poor for IO. I'll have to check and see what is going on. I was getting 100K with Ubuntu earlier.

[ioquatix]

I agree that macOS has a shitty IO layer.

I was running on the same linux box with (4 core / 8 hyper threads). The upper limit with a simple TCP epoll server is around 400,000 connections per second. epoll doesn't scale linearly unfortunately. Single core performance can be more like 150,000 connections per second.

[ioquatix]

I made a config.ru file similar to the puma config, and then used rackup to run it with agoo, because I think this better reflects real world baseline performance. Obviously in this case there are some performance issues since involving agoo directly is much better performance.

[ohler55]

Is the default /public or ./public? I could easily have Agoo default to that for rack. Rails uses something different and Agoo detects and provides those.

Maybe it is rackup that is slowing things down. I'd prefer not to have that be the bottleneck when running benchmarks. How about we keep the benchmarks focussed on the servers and leave rackup out of the equation.

[ohler55]

Maybe some explanation or comparison between rackup and without?

[ioquatix]

The default is File.expand_path('public', __dir__).

I'm pretty sure that is also the default for rails - if you make your own rails website it will make a public/ directory.