omerisimo / em_underground Goto Github PK

Not a real issue, but an interesting point to make.

A benchmark for EM vs NodeJs, in a 'hello world' sample is not that interesting for any app that makes sense. However, as you clearly stated, the results you mention (and others that are published across the web) are interesting, simply because they make little sense considering the hype :)

I went a head and tried to validate these results.

Testing with ab, I got similar results:

$> ab -c 100 -n 10000 http://0.0.0.0:8080/

#NodeJs:
Requests per second:    2957.08 [#/sec] (mean)
#EM:
Requests per second:    5354.17 [#/sec] (mean)

However, when testing with alternative tools, it looks different:

With wrk (https://github.com/wg/wrk):

$> wrk -t20 -c400 -d10s http://0.0.0.0:8080/

NodeJs:
Requests/sec:   4416.09

EM:
Requests/sec:   1675.22

Alternative settings of wrk (just to make sure I don't accidentally hit a nodejs sweet spot):

$> wrk -t40 -c1000 -d10s http://0.0.0.0:8080/
NodeJs
Requests/sec:   3737.44
EM:
Requests/sec:   1608.93

$> wrk -t100 -c1000 -d10s http://0.0.0.0:8080/

Requests/sec:   1645.77
Requests/sec:   3604.65


With siege (https://github.com/tail/siege):
````sh
$> siege -c1000 -d1 -t10S -q http://0.0.0.0:8080/
NodeJs
Transaction rate:       996.11 trans/sec
EM:
Transaction rate:       768.96 trans/sec

I've also tested with weighttp (https://github.com/lighttpd/weighttp) , to receive results similar to ``ab`.

Bottom line for me, and I might be misinterpreting these results, but I suspect in such a simple test as 'hello world', the overhead by tool being used and the specific design of it actually renders the results invalid.

The fact that two tools (wrk vs ab) shows such contradicting results is too off to make sense.