Request to explore JavaCPP as an extension to model Python FFI. about jyni HOT 4 OPEN

Hello @abhi18av,
note that JyNI is not yet another Java FFI, but actually emulates the existing CPython C-extension API.
This API is very involved and needs rather special treatment in terms of dynamic loading, memory management and various further aspects.

• it must bridge reference counting and mark-and-sweep gc
• it is especially challenging to get finalizers and weak references right in this context
• it must resolve GIL handling
• it needs to control RTLD flag, which isn't trivial via JNI

For details have a look at this paper.
E.g. with JavaCPP it is only possible to call into native code and to pass callbacks, while CPython extension API requires to export symbols (which is not supported by JavaCPP AFAICT).
Even if it would be feasible with JavaCPP, I would not see the benefit, because the JNI glue code is really the trivial part of JyNI and is already done.
Then, JavaCPP requires C/C++ compilation for each extension specifically, which is a major drawback compared to e.g. JNR ("interestingly" they ignore that aspect in their readme entirely and somehow describe JavaCPP as superior to e.g. JNR).
On the other hand, JavaCPP's main advantage - tight C++ integration - would not pay off for JyNI at all, because CPython is written in plain C.
Also, I doubt there is potential for performance gain, because JavaCPP boils down to JNI as well. In contrast to that, JNR is said to be slightly faster than plain JNI due to some smart use of assembler. Anyway, I would not explore any beyond-JNI optimization before project panama turns out, as that is by far the most promising advancement in terms of Java FFI performance (presumingly, JyNI 3 will make use of that one day).

from jyni.

After your wonderful explanation, I do have a better understanding of the scope of the project @Stewori . Thanks for taking out the time to be so thorough 👍

from jyni.

I do wish that Python3 transition would pick up speed in the Jython world :)

from jyni.

Hello, author of JavaCPP here. I agree, CPython doesn't need C++. :)
But I do hope that this is all going to get integrated one day...

from jyni.