Giter VIP home page Giter VIP logo

binakot / abstract-rest-service-benchmark Goto Github PK

View Code? Open in Web Editor NEW
16.0 2.0 14.0 337 KB

A set of template services in different languages and frameworks. An abstract performance comparison based on the endpoint throughput. There are no objective benefits, just an elementary comparison of primitive RESTful API services. :shipit:

License: MIT License

C# 33.86% Java 21.75% Go 2.68% JavaScript 8.22% Rust 4.69% Python 9.29% Haskell 6.73% F# 10.14% Dockerfile 2.63%
benchmark template service rest-api test performance throughput hacktoberfest hacktoberfest-accepted

abstract-rest-service-benchmark's Introduction

Abstract Rest Service Benchmark

Requirements

  • PORT: 8080

  • API request: /api/test

  • API response: Hello, World! (text/plain, UTF-8, 13 bytes)

Benchmark

Command for WRK:

$ wrk -t8 -c512 -d2m --timeout 10s --latency http://localhost:8080/api/test

Results

# Service Language Framework RPS
Java Light 4J Java oracle jdk 9.0.4 + ligth4j 401511.94
.Net Core C# dotnet 2.1.4 249758.46
Haskell Warp Haskell ghc 7.10.3 + warp 191020.62
Rust Iron Rust rust 1.24.1 + iron 186726.17
Go Gorilla/Mux Golang go sdk 1.9.4 + gorilla/mux 131499.79
.Net Core ASP.NET MVC C# dotnet 2.1.4 99091.85
NodeJS JavaScript nodejs 8.9.4 94132.92
Python Sanic Python python 3.5.2 + sanic 74854.41
NodeJS Express JavaScript nodejs 8.9.4 + express 58769.29
Java Spring Boot Java oracle jdk 9.0.4 + spring boot 56253.12
.Net Core Freya F# dotnet 2.1.4 + freya-hopac 49736.08
Python Aiohttp Python python 3.5.2 + aiohttp 4603.01
Python Flask Python python 3.5.2 + flask 200.44
.Net Core Websocket C# dotnet 2.1.4 + websocket TBD
.Net Core Giraffe F# dotnet 2.1.4 + giraffe TBD
Python API Star Python python 3.5.2 + apistar TBD
Rust Rocket Rust rust 1.24.1 + rocket TBD

Services

dotnet-core-logo

.Net Core by @vanbukin

Language: C#

Framework: .Net Core 2

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./dotNetCoreRestService/bin/Release/netcoreapp2.1/publish
$ dotnet dotNetCoreRestService.dll

Result:

TBD

.Net Core ASP.NET MVC by @binakot

Language: C#

Framework: .Net Core 2, ASP.NET MVC

Main tutorial: https://docs.microsoft.com/en-us/aspnet/core/tutorials/first-web-api

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./dotNetCoreRestService/bin/Release/netcoreapp2.0/publish
$ dotnet dotNetCoreRestService.dll

Result:

TBD

.Net Core Websocket by @petriashev

Language: C#

Framework: .Net Core 2, Websocket

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./bin/Release/netcoreapp2.0/publish
$ dotnet dot-net-core-websocket.dll

Result:

TBD

.Net Core Freya by @Szer

Language: F#

Framework: .Net Core 2, Freya

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./bin/Release/netcoreapp2.0/publish
$ dotnet dot-net-core-hopac-freya.dll

Result:

TBD

.Net Core Giraffe by @Szer

Language: F#

Framework: .Net Core 2, Giraffe

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./bin/Release/netcoreapp2.0/publish
$ dotnet dot-net-core-giraffe.dll

Result:

TBD

golang-logo

Go Gorilla/Mux by @binakot

Language: Go

Framework: Go SDK, Gorilla/Mux

Main tutorial: https://www.codementor.io/codehakase/building-a-restful-api-with-golang-a6yivzqdo

Run:

$ go build
$ ./go

Result:

TBD

haskell-logo

Haskell Warp by @kchugalinskiy

Language: Haskell

Framework: GHC, Warp

Main tutorial: http://taylor.fausak.me/2014/10/21/building-a-json-rest-api-in-haskell

Requirements:

Run:

$ stack setup
$ stack build
$ stack exec .stack-work/dist/**/build/test-exe/test-exe

Result:

TBD

java-logo

Java

Language: Java

Framework: Oracle JDK 8 or above

Run:

$ gradle clean run

Result:

TBD

Java Light 4J by @0ffer

Language: Java

Framework: Oracle JDK 9, Light4J

Main tutorial: https://github.com/networknt/light-example-4j/tree/master/demo

Run:

$ mvn clean install
$ java -jar target/service-example-0.1.0.jar

Result:

TBD

Java Spring Boot by @binakot

Language: Java

Framework: Oracle JDK 9, Spring Boot

Main tutorial: http://spring.io/guides/gs/rest-service

Run:

$ gradle clean build
$ java -jar build/libs/java-spring-boot-rest-service-1.0-SNAPSHOT.jar

Result:

TBD

nodejs-logo

NodeJS by @WilixLead

Language: JavaScript

Framework: Node.js

Main tutorial: https://nodejs.org/api/http.html#http_class_http_server

Run:

$ node ./index.js

Result:

TBD

NodeJS Express by @WilixLead

Language: JavaScript

Framework: Node.js, Express

Main tutorial: http://expressjs.com/en/starter/hello-world.html

Run:

$ npm i
$ node ./index.js

Result:

TBD

python-logo

Python Aiohttp by @TyVik

Language: Python

Framework: Python 3, Aiohttp

Main tutorial: https://aiohttp.readthedocs.io/en/stable/

Run:

$ pip install -r requirements.txt
$ python app.py

Result:

TBD

Python API Star by @sergeibershadsky

Language: Python

Framework: Python 3, API Star

Main tutorial: https://github.com/encode/apistar

Run:

pip install -r requirements.txt
uvicorn app:app --workers=8 --bind=0.0.0.0:8080 --pid=pid

Results:

TBD

Python Flask by @TyVik

Language: Python

Framework: Python 3, Flask

Main tutorial: http://flask.pocoo.org/

Run:

$ pip install -r requirements.txt
$ python app.py

Result:

TBD

Python Sanic by @TyVik

Language: Python

Framework: Python 3, Sanic

Main tutorial: http://sanic.readthedocs.io/en/latest/

Run:

$ pip install -r requirements.txt
$ python app.py

Results:

TBD

rust-logo

Rust Iron by @kchugalinskiy

Language: Rust

Framework: Rust SDK, Iron

Main tutorial: https://github.com/iron/router/blob/master/examples/simple.rs

Run:

$ cargo run --release

Result:

TBD

Rust Rocket by @qezz

Language: Rust

Framework: Rust SDK, Rocket

Main tutorial: https://rocket.rs/guide/getting-started/#hello-world

Run:

$ ROCKET_ENV=prod cargo run --release

Result:

TBD

Hardware

  • Server (rest-service) host:
TBD
  • Client (wrk) host:
TBD

abstract-rest-service-benchmark's People

Contributors

binakot avatar eitoball avatar ju-bear avatar kchugalinskiy avatar petriashev avatar qezz avatar tyvik avatar worldbeater avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

abstract-rest-service-benchmark's Issues

Move wrk to the separate instance

WRK load ~120% CPU during benchmark. This is can significantly affect test results, because ~12% CPU spent on switching context. Please, run wrk on another instance in same local network.

1 CPU vs Multiple-CPU test environment?

I have no idea which is better for rest service benchmarking. In production we are using multiple CPU instance. If we test our projects on 1 CPU, how it will affect on our understaning of every frameworks and its comparison.

What do u think? With 1 CPU we can use something free of cloud provider? (#23)

Remove python-aiohttp

Please, remove python-aiohttp because it does not support multiprocessing. gunicorn is prefered in production mode as process supervisor.

Provide repeatable environment for tests

Please provide public environment for tests: droplet on DigitalOcean, VPS on Linode, EC2 instance or etc. I'm pauper and I have not 8x Intel Xeon CPU E5-2660 v3 @ 2.6GHz with 16 Gb RAM :(

Add service authors.

Add author to every service with link to github profile.

P.S. Thanks everyone for PR with services (and hacks ๐Ÿ˜„ ).

Add Rust Rocket

I'm going to add hello-world implemented in rocket-rs, so benchmark results should be added to readme

The only drawback is that it requires rust nightly

UPD: see #29

Add table with results

Something like this to compare services

| Service             | Language   | Framework   | RPS  |
| ------------------- | ---------- | ----------- | ---- |
| [Service 1](/path/) | Python 3.6 | Django 1.11 | 1000 |

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.