Autocannon benchmark

This repo contains benchmarking / load testing scripts for Solid servers. The tests are created using autocannon. The code uses the DPoP flow for authorization and tokens.

The benchmark has been tested against the following server implementations:

• PHP Solid Server
• Solid Nextcloud
• Node solid server

Installation

npm install

Setup the environment

cp env.example .env

Edit the file and add the appropriate values for your setup

Running a test

node --env-file=.env ./login/run.js

Available tests:

• fetching static html
• fetching .well-known
• client registration
• login
• consent
• authorize
• token exchange
• solid profile card
• fetch things from storage

Quick overview

node --env-file=.env ./homepage/run.js
node --env-file=.env ./well-known/run.js
node --env-file=.env ./register/run.js
node --env-file=.env ./login/run.js
node --env-file=.env ./consent/run.js
node --env-file=.env ./authorize/run.js
node --env-file=.env ./token/run.js
node --env-file=.env ./profile/run.js
node --env-file=.env ./storage/run.js

Authentication for environments with cookies

Some servers (such as Solid-Nextcloud) are a bit hard to convince to do automated logins. For this scenario, it is possible to directly supply a login cookie to the benchmark tools.

If you want to run a benchmark on an environment that uses a cookie-based login, you can run the benchmark with a logged in session by setting

SOLID_LOGIN_COOKIE

in the env file. The contents should be whatever cookie the logged in session is sending to the server. You can find this in the developer tools of your browser, in the Set-Cookie header of an authenticated request.

Notes

Do not run without consent

Running these benchmarks is not a friendly thing to do on servers outside of your own control. Depending on the configuration, autocannon can open a lot of connections which is needed to benchmark the performance of the server.

Benchmark = hardware performance

Running a benchmark on a Solid installation will give some insight on how many requests per second a given setup can handle. It is aimed to compare and improve the performance of a given deployment.

It can give some insights on how fast the software itself is, but keep in mind that the comparisons only make sense if they are running on similar hardware in similar circumstances.

Funding

This project was funded through the NGI0 Core Fund, established by NLnet with financial support from the European Commiss ion's Next Generation Internet programme. Learn more at the NLnet project page