i10416 / bench Goto Github PK

slinc vs jni benchmark

Environment

• Scala 3.2.2
• JVM: OpenJDK Runtime Environment Zulu19.30+11-CA (build 19.0.1+10)
• slinc: 0.1.1-110-7863cb
• Apple clang version 13.1.6 (clang-1316.0.21.2.5)
• slinc 0.1.1-110-7863cb

In general, it is clear that upcall JVM method from native is quite slow. The following change makes a program extremely slower(check SimpleNativeCallBenchmarks. jniNativeQSort and SimpleNativeCallBenchmarks.jniQSort).

(JNIEnv *jenv, jobject jobj, jintArray jarr){
  jint *arr = (*jenv)->GetIntArrayElements(jenv,jarr, 0);
  jsize len = (*jenv)->GetArrayLength(jenv,jarr);
-  qsort(arr,len,sizeof(int),compare_int);
+  qsort(arr,len,sizeof(int),upcall_compare_int);
  (*jenv)->ReleaseIntArrayElements(jenv,jarr,arr, 0);
  return;
};

There's only a small difference in performance between array copy back and forth and array copy without copy back. See SimpleNativeCallBenchmarks.slincQSortWithCopyBack and SimpleNativeCallBenchmarks.slincQSortWithoutCopyBack.¹

As is mentioned in the comment, I confirmed that SimpleNativeCallBenchmarks.slincQsortAllocCallbackForEachIteration is much slower than slincQSortWithCopyBack and slincQSortWithoutCopyBack. Allocating upcall seems costly operation.

Having cloned your bench and having the callback allocated once (rather than per benchmark iteration), I see a improvement in performance of Slinc's upcall code to just 2x slower than JNI, rather than 5x slower

scala-interop/slinc#81 (comment)

SlinC one is around 2 times slower than JNI one when allocating callback in advance and 5x~ times slower when allocating callback for each iteration.

SlinC one is around 5 times slower than JNI one probably because SlincQsort transfers array between JVM and native, but I suspect there are other reasons why SlinC ones are slow because there is not large difference between slincQSortWithCopyBack and slincQSortWithoutCopyBack, which implies data transfer is not the bottleneck

Feedback from SlinC author(@markehammons)

Having cloned your bench and having the callback allocated once (rather than per benchmark iteration), I see a improvement in performance of Slinc's upcall code to just 2x slower than JNI, rather than 5x slower. I think there may be more performance improvements to be found, but first I should make us able to generate an upcall from a method rather than a lambda and see what the performance from that looks like.

For the following routine,

1. copy string (jvm to native)
2. invoke native call (jvm to native)
3. dereferencing pointer (native to jvm)
4. copy object (jvm to native)
5. copy object (native to jvm)

with the following environment,

• Scala 3.2.2
• JVM: JDK 17.0.3, OpenJDK 64-Bit Server VM, 17.0.3+7-LTS
• slinc: 0.1.1-110-7863cb
• Apple clang version 13.1.6 (clang-1316.0.21.2.5)
• slinc 0.10-110-7863cb

slinc one takes around 3x~ longer than jni.

Result

However, just updating JDK to Zulu19.30+11-CA, slinc one gets nearly as fast as jni one.

JVM: OpenJDK Runtime Environment Zulu19.30+11-CA (build 19.0.1+10)

JNI or FFI is not always the fastest solution as modern JVM is quote performant and communication between native and jvm is not free.

For example, see benchmark for quicksort and you can find SimpleNativeCallBenchmarks.slincQSortJVM is the fastest.

You should take overhead into consideration. Find a significant bottleneck and carefully measure the performance benefits before resorting to FFI.