It would help newbies if some explanation is added to benchmarks results like lower is better or higher is better.
I think this explanation should be printed by rewrk by defualt.

I have a problem, Windows 11 Pro 22621

thread 'main' panicked at 'failed to bind listener: Os { code: 10048, kind: AddrInUse, message: "Solo se permite un uso de cada dirección de socket (protocolo/dirección de red/puerto)" }', C:\Users\afcar\.cargo\registry\src\github.com-1ecc6299db9ec823\astra-0.1.3\src\server.rs:108:58

In the part of bench-bot, seems that kill is not fast enough
Line 185
let _ = server.kill(); //This only works with a delay
I just added delay before killing and now it works
thread::sleep(Duration::from_secs(5));
This should be a PR but I'm not sure, anyways if someone has this problem it's a simple fix

ntex is here.

Even though Hyper and Axum have very close request/sec their MB/sec is very different. I believe this is due to the default headers returned by different frameworks. If all the frameworks respond with the same amount of data per request, that could change the results, but I am not sure by how much.

Background

It would be nice if there is a table showing the result of the benchmark between each web framework.

Expectation

It is something like that:

Anything else

I am willing to add this feature.

In rewrk output, actix web double scores Rocket in requests but in the table rocket sends more. How can it be?

• xitca-web
• may-minihttp

osinfo:

ProductName: macOS
ProductVersion: 11.5.2
BuildVersion: 20G95

kern.maxfilesperproc: 49152
kern.maxfiles: 98304

result:

➜ echo "Results for hyper:" && rewrk -t 12 -c 300 -d 10s -h http://localhost:3000/

Results for hyper:
Beginning round 1...
Benchmarking 300 connections @ http://localhost:3000/ for 10 second(s)
failed to run benchmark round due to error: error combining results: connection closed

use wrk it's ok:

➜ wrk -t12 -c400 -d30s --latency http://localhost:3000/
wrk http://localhost:3000/
Running 30s test @ http://localhost:3000/
12 threads and 400 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 808.44us 247.22us 3.95ms 72.97%
Req/Sec 23.51k 9.67k 141.20k 79.11%
Latency Distribution
50% 819.00us
75% 0.96ms
90% 1.10ms
99% 1.40ms
7020424 requests in 30.10s, 589.18MB read
Socket errors: connect 157, read 89, write 0, timeout 0
Requests/sec: 233235.83
Transfer/sec: 19.57MB