Giter VIP home page Giter VIP logo

dyu / ffi-overhead Goto Github PK

View Code? Open in Web Editor NEW
641.0 27.0 42.0 91 KB

comparing the c ffi (foreign function interface) overhead on various programming languages

License: Apache License 2.0

Shell 4.44% C 52.56% Go 1.72% Lua 1.66% Nim 1.43% Rust 1.94% Java 2.76% C++ 8.93% Dart 2.40% D 1.42% JavaScript 2.32% Haskell 1.58% Julia 1.19% C# 1.66% Makefile 0.96% OCaml 6.87% Elixir 1.41% Zig 3.18% V 1.07% Wren 0.52%
ffi c benchmark

ffi-overhead's Introduction

ffi-overhead

comparing the c ffi overhead on various programming languages

Requirements:

  • gcc
  • tup
  • zig
  • nim
  • v
  • java7
  • java8
  • go
  • rust
  • d (dmd and ldc2)
  • haskell (ghc)
  • ocaml
  • csharp (mono)
  • luajit
  • julia
  • node
  • dart
  • wren
  • elixir

My environment:

- Intel i7-3630QM laptop (4cores, HT) with 16g ram
- Ubuntu 14.04 x64
- gcc/g++ 5.4.1
- tup 0.7.4
- zig 0.2.0
- nim 0.14.3
- v 0.2.2 aebb551
- java 1.7.0_72 and 1.8.0_91
- go 1.8.0
- rust 1.17.0-nightly (c0b7112ba 2017-03-02)
- dmd 2.0.71.1
- ldc2 1.9.0
- ghc 7.10.3 (at /opt/ghc)
- ocaml 4.06.1
- mono 5.12.0.226
# dynamic languages 
- luajit 2.0.4
- julia 0.6.3
- node 6.9.0 (at /opt/node)
- dart 1.22.0 (at /usr/lib/dart)
- wren 0.1.0
- elixir 1.6.5 (Erlang/OTP 20)

Initialize

tup init

Compile

./compile-all.sh

Compile opts:

  • -O2 (gcc - applies to c/jni/nim)
  • -C opt-level=2 (rust)

Run

./run-all.sh 1000000

Measurement:

  • call the c function "plusone" x number of times and print out the elapsed time in millis.
int x = 0;
while (x < count) x = plusone(x);
  • 2 samples/runs

Results (500M calls)

./run-all.sh 500000000
The results are elapsed time in milliseconds
============================================

luajit:
891
905

julia:
894
889

c:
1182
1182

cpp:
1182
1183

zig:
1191
1190

nim:
1330
1330

v:
1779
1781

rust:
1193
1196

d:
1330
1330

d ldc2:
1191
1189

haskell:
1197
1198

ocamlopt:
1634
1634

ocamlc:
4299
4302

csharp mono:
2697
2690

java7:
4469
4472

java8:
4505
4472

node:
9163
9194

node scoped:
15425
15409

go:
37975
37879

dart:
31265
31282

dart scoped:
61906
69043

wren:
14519
14514

elixir:
23852
23752

ffi-overhead's People

Contributors

dextorious avatar dyu avatar jarred-sumner avatar ken0x0a avatar let-def avatar minond avatar nattynarwhal avatar phmarek avatar puffnfresh avatar slofurno avatar timgates42 avatar yallop avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ffi-overhead's Issues

Add ldc (D) compiler

While dmd does generate reasonable code, dmd's backend isn't exactly this century (even though it does very well).

For a better comparison against C++ with gcc, it would be best to use ldc (maybe even gdc?) as well.

Is the Go result correct?

The results from your test showed go was really slow. For simplicity I combined everything into it's own file. I'm getting a result of 110 ms (vs 37879). I may be misunderstanding something.

package main

/*
#include <stdio.h>
#include <sys/time.h>

int plus(int x, int y)
{
    return x + y;
}

int plusone(int x)
{
    return x + 1;
}

long long current_timestamp()
{
    struct timeval te; 
    gettimeofday(&te, NULL); // get current time
    long long milliseconds = te.tv_sec*1000LL + te.tv_usec/1000; // caculate milliseconds
    
    return milliseconds;
}
*/
import "C"
import "fmt"
import "os"
import "strconv"

func run(count C.int) {
    start := C.current_timestamp()
    
    var x C.int = 0
    for x < count {
        x = C.plusone(x)
    }
    
    fmt.Println(C.current_timestamp() - start)
}

func main() {
    if (len(os.Args) == 1) {
        fmt.Println("First arg (0 - 2000000000) is required.")
        return
    }
    
    count, err := strconv.Atoi(os.Args[1])
    if err != nil || count <= 0 || count > 2000000000 {
        fmt.Println("Must be a positive number not exceeding 2 billion.");
        return
    }
    
    // load
    C.plusone(C.int(C.current_timestamp()))
    
    // start
    run(C.int(count))
}

go version go1.9.2 darwin/amd64

Core i5 Mac

Unroll the loops?

Since you benchmark the entire program elapsed time, the loop itself โ€” evaluating the exit condition, looking up the variable, and jumping back or out โ€” adds to the execution time. The body of the loop is tiny, just a single FFI call, so the loop overhead is very likely significant, especially in the interpreted languages.

A simple fix is to unroll the loop several times. Maybe something like (here's the Wren one):

class FFI {
    foreign static plusone(x)
    foreign static count()
    foreign static start()
    foreign static stop()
}
var x = 0
var count = FFI.count()
// try call
FFI.plusone(x)

FFI.start()
while (x < count) {
    x = FFI.plusone(FFI.plusone(FFI.plusone(FFI.plusone(FFI.plusone(x)))))
}
FFI.stop()

My hunch is that you'll see quite different relative numbers if you do this for all the languages. Right now, your benchmarks probably show the relative performance of assigning variables and looping more than they do FFI.

.NET Core 2.1 test

.NET Core 2.1 runs on linux and installs-compiles from command line. I hope to see C# or F# test. Just plug here issues so that will return and do the stuff when trying dotnet toolchain on linux.

Rerun for more current performance readings

Go and Dart have had optimization passes on their FFI implementations. Rust has also gotten some compiler improvements that may impact this. A fresh update of the benchmark is due.

Java version 25%

Hey I was looking more into the Java test and I found your comment of the added 25%. You should remove that and just do a + b inline because you are already paying the price of jumping from Java to native land so it makes no sense to add yet another indirection that is not needed. I would like to see what the best possible is.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.