stewori / jyni Goto Github PK

PyByteArray would be the next step towards PyByteArray, BufferProtocol, SciPy and Pandas support. It should be implemented in context of overall support for the BufferProtocol

To import nltk I need to import numpy first, otherwise I get:

>>> import nltk
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/nltk/__init__.py", line 89, in <module>
    from nltk.internals import config_java
  File "/usr/lib/python2.7/dist-packages/nltk/internals.py", line 31, in <module>
    from nltk import compat
  File "/usr/lib/python2.7/dist-packages/nltk/compat.py", line 51, in <module>
    class UTC(tzinfo):
TypeError: Error when calling the metaclass bases
    'getset_descriptor' object is not callable

The tk download dialog works fine, but it crashes the JVM when I select something and hit "download":

>>> import numpy
>>> import nltk
>>> nltk.download()
showing info https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml
*** Error in `java': free(): invalid pointer: 0x00007fe2adc24cb8 ***
fish: “java -cp jython.jar:build/JyNI.…” terminated by signal SIGABRT (Abort)

This issue is sourced out from #32, see #32 (comment) and #32 (comment).
Reproducible by this code:

import sys

sys.path.insert(0, '/data/workspace/linux/numpy/1.13.3')

import numpy as np
from java.lang import System
import time

for i in range(2000):
	print i
	np.array(range(7))

System.gc()
time.sleep(2)

It may or may not be possible to reproduce this somehow without NumPy or even without JyNI. If the issue triggers it results in the following log (excerpt), pointing to JyAttribute.delAttr:

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
J 2143 C1 org.python.core.JyAttribute.delAttr(Lorg/python/core/PyObject;B)V (169 bytes) @ 0x00007f19297499bb [0x00007f19297496c0+0x2fb]
J 2148 C1 JyNI.JyNI.clearNativeHandle(Lorg/python/core/PyObject;)V (37 bytes) @ 0x00007f192974d02c [0x00007f192974cd60+0x2cc]
v  ~StubRoutines::call_stub
V  [libjvm.so+0x666a0b]
V  [libjvm.so+0x6886fe]
V  [libjvm.so+0x68b02f]
C  [libJyNI.so+0x46f7e]  JyNI_CleanUp_JyObject+0xfc
C  [libJyNI.so+0xf42fe]  meth_dealloc+0x240
C  [libJyNI.so+0xfc87b]  JyGC_clearNativeReferences+0xefa
C  [libJyNI-Loader.so+0x561c]  Java_JyNI_JyNI_JyGC_1clearNativeReferences+0x34
j  JyNI.JyNI.JyGC_clearNativeReferences([JJ)Z+0
j  JyNI.gc.JyWeakReferenceGC$GCReaperThread.run()V+251
v  ~StubRoutines::call_stub
V  [libjvm.so+0x666a0b]
V  [libjvm.so+0x663fe4]
V  [libjvm.so+0x6645c7]
V  [libjvm.so+0x6a78a4]
V  [libjvm.so+0xa15547]
V  [libjvm.so+0xa15a0c]
V  [libjvm.so+0x8bae12]
C  [libpthread.so.0+0x76ba]  start_thread+0xca

This is caused by some native jobject that should be representing a org.python.core.PyObject actually does not. Still, GetObjectRefType from JNI API reports the object as a valid reference of type JNIWeakGlobalRef. The object is not NULL and not null. Calling getClass on the object on Java side results in java.lang.object. Apparently, no Java-side exception is pending. If there was one, it got cleared.

The issue can be fixed in symptomatic sense by changing JyNI.clearNativeHandle to the following:

clearNativeHandle(PyObject object) {
		if (object == null) {
			System.err.println("JyNI-Warning: clearNativeHandle called with null!");
			return;
		}
		if (object instanceof PyCPeer)
			((PyCPeer) object).objectHandle = 0;
		else {
			if (!(object instanceof PyObject))
				System.err.println("JyNI-Warning: clearNativeHandle received Non-PyObject!");
			else
				JyAttribute.delAttr(object, JyAttribute.JYNI_HANDLE_ATTR);

However we should investigate under which circumstances the invalid jobject is created. I am somewhat confident that native JyNI code never creates a plain java.lang.object. Is it possible that the JVM creates faulty native jobjects under heavy memory stress or if it runs out of native references? (c.f. EnsureLocalCapacity in JNI API) This suspect is supported by the observation that the issue can be avoided by calling System.gc right after import numpy:

import numpy as np
System.gc()
time.sleep(2)

The JVM does not take natively used memory into account to decide when it's time to run GC. Maybe we should trigger GC explicitly in JyNI from time to time. E.g. after importing a native module or if the number of JyGCHeads exceeds some limit. A proper strategy needs to be figured out.

Nested imports cause deadlocks. The deadlock is caused by native libraries which are loading (indirectly) other native libraries before calling "LEAVE_JyNI". Then the next JyNI_loadModule (JyNI.c) call will stop at ENTER_JyNI.

One possible solution might be to add LEAVE_JyNI and ENTER_JyNI around the call of the dynamic load function (return value of _PyImport_GetDynLoadFunc) in _PyImport_LoadDynamicModuleJy (importdl.c) i.e. around line (*p)();

Another solution might be to check in loadModule in JyNI.java, if one was called from the native library and then calls LEAVE_JyNI and at the end of the function restores the state by calling ENTER_JyNI.

Stefan, I would be happy to hear your opinion on that, since I do not know what requirements on this locking mechanism exist.

Hi,

I am trying to import multiarray as a next step towards numpy support. As test I use the following snippet:

import sys

sys.path.append('/usr/lib64/python2.7/lib-dynload')
sys.path.append('/usr/lib64/python2.7/site-packages/numpy')
sys.path.append('/usr/lib64/python2.7/site-packages/numpy/core')

print 'before multiarray import'

import datetime

import multiarray

print 'after multiarray import'

a = multiarray.array([1, 2, 3])

print a

Running this script usually fails with something like the following besides some rare cases where it actually works: (I can also provide the hs_err files if needed.)

> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00007f4998fe073c, pid=11479, tid=139954600363776
> #
> # JRE version: OpenJDK Runtime Environment (8.0_72-b15) (build 1.8.0_72-b15)
> # Java VM: OpenJDK 64-Bit Server VM (25.72-b15 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # C  [libJyNI.so+0xe073c]  pushExStack+0xa9
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /home/<user>/programming/jyni/JyNI/hs_err_pid11479.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> 

This looks like some side effect due to some memory, which is overwritten. It looks like this is caused in the _PyImport_LoadDynamicModuleJy, when the "dyn load func" is called. I would appreciate any hints about this before I dig deeper into this.

Thanks,

Lothar

Hi @Stewori

This is not an issue so please do feel free to close it down. This is more like a query/suggestion regarding the project. The basic question is -

Why is a seperate JyNI project needed to accomplish the linkage to python native extensions?

I'd like to suggest that perhaps, https://github.com/bytedeco/javacpp might be a worthy option to explore in this space as well. It's being used by ND4J and DeepLearning4J projects plus there are a number of bindings already generated located here https://github.com/bytedeco/javacpp-presets .

I hope that JavaCpp might take us closer to a proper package compatibility. I'm really looking forward to using proper python packages with Jython , Thanks again :)

However when I replaced np.hsplit(x,2) with np.hsplit(x,[2]), it can work . So I wonder how can I use np.hsplit(x,2)

Subpackagenames are not resolved by find_module in JyNIImporter. The reason is that find_module does not descent into the directories.

This is also relevant for local import paths (from . import multiarray) since they are changed to global paths (numpy.core.multiarray) and handed over to find_module as name.

I implemented a solution for this and will upload it after further testing.

umath fails to load with the message: AttributeError: _ARRAY_API not found

Cause for this is a bug in JyNI-C/src/Python/import.c in the function PyObject * PyImport_ImportModule(const char *name). The current implementation calls directly the import method from Jythons' builtin class. This leads to the fact, that function calls like
PyImport_ImportModule("A.B.C")
will return the top module A instead of A.B.C as it should be.

In umath this is location in the numpy/core/src/umath/umathmodule.c file within the numpy package
. In the function initumath in the line

PyObject *numpy = PyImport_ImportModule("numpy.core.multiarray");

the variable numpy will ironically be the numpy package and not the multiarray subpackage. Which leads to the observed error, if later the attribute "_ARRAY_API" is checked with this line:

c_api2 = PyObject_GetAttrString(numpy, "_ARRAY_API");
The solution to this bug is to change the function PyImport_ImportModule(const char name) to call PyImport_Import(PyObject module_name) as in the cpython implementation and uncomment the original PyImport_Import(PyObject* module_name) in JyNI-C/src/Python/import.c and let it call the Jython methods after the silly_list has been built up.

In order to do this a comment how these lines

r = PyObject_CallFunction(import, "OOOOi", module_name, globals,
                              globals, silly_list, 0, NULL);

should be called in Jython would be very helpful.

... is currently broken:

Traceback (most recent call last):
  File "/data/workspace/linux/JyNI/JyNI-Demo/src/JyNINumPyTest.py", line 68, in <module>
    import numpy as np
  File "/data/workspace/linux/numpy/1.14.0/numpy/__init__.py", line 168, in <module>
    from . import ma
  File "/data/workspace/linux/numpy/1.14.0/numpy/ma/__init__.py", line 44, in <module>
    from . import core
  File "/data/workspace/linux/numpy/1.14.0/numpy/ma/core.py", line 6333, in <module>
    masked = masked_singleton = MaskedConstant()
  File "/data/workspace/linux/numpy/1.14.0/numpy/ma/core.py", line 6270, in __new__
    cls.__singleton = MaskedArray(data, mask=mask).view(cls)
  File "/data/workspace/linux/numpy/1.14.0/numpy/ma/core.py", line 2790, in __new__
    _data = ndarray.view(_data, cls)
AttributeError: 'type$1' object has no attribute 'update'

Stick to NumPy 13.3 until this is solved...

I have found out when I was trying to debug numpy that building all stuff JyNI related can be hairy. Could you provide a guide (and maybe a short screencast)?

Hi, is JyNI OK to use with Jython 2.0-final? The final release came out in May, but JyNI says it needs Jython 2.0-b4.

In #10 it was explained how to build JyNI in debug mode. That requires editing the makefile so we should better have a build target for debug. I created this issue to keep track of that idea.

This is an issue with mtrand.c, which is a Cython-generated file. In the mtrand folder, NumPy bundles a script called generate_mtrand_c.py. This is used to generate mtrand.c, in NumPy 13.2 prebuilt with Cython 0.27. If you build it using Cython 0.25 everything works fine (0.26 not tested).
For convenience I prepared a Cython-0.25-built version of mtrand.c.

You can either replace mtrand.c if you build NumPy by yourself or use the binary mtrand.so (Linux 64 only as of this writing; I will add more binaries soon). You can put the binary on sys.path such that it preceeds NumPy. Then it should override the original mtrand.so.

Sure, this needs to be fixed in JyNI, but I cannot tell when I'll find time. So, until then use this workaround for NumPy 13.2.

In the first pic, i used mmap to read file with python. the second is jython and JyNI. the string is truncated when i used JyNI.

Hi,

I wrote my custom module in C for Python and I can imported like this:
java -cp jython-standalone-2.7.2.jar:Lib/JyNI.jar org.python.util.jython test.py
The problem is that my functions are not exported and I can't call them allthough the module is successfully imported.

test.py contains the following:

import mtrand print(dir(mtrand)) print(mtrand.__file__)

and the output is
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 'mtrand']
If I run the same thing under python2 I can see all the exported funcitons:

`python
Python 2.7.18 (default, Mar 8 2021, 13:02:45)
[GCC 9.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import mtrand
dir(mtrand)
['author', 'builtins', 'doc', 'file', 'name', 'package', 'test', 'mt_rand', 'mt_rand_ab', 'mt_rand_orig', 'mt_srand']

`

So, why are my functions not exprted with JyNI?

Thank you!

Dear @Stewori ,

Thank you for the clarifications provided on the Processing.py git thread.

I (and some other folks from the Processing community) would love to see JyNI working in Processing Python mode but are completely overwhelmed by the installation instructions. Would you mind give us a hand with this ?

Here's what I could do so far:

• looked for the jython.jar file on my computer. It lies in a Processing sub-folder called 'mode' with 2 other jar files, PythonMode.jar and guava-17.0.jar

• set a JRE_HOME variable pointing to the JRE installation path C:\Program Files (x86)\Java\jre1.8.0_211

This leaves me with 2 questions:

• Do I need to put JyNI.jar and JyNI.dll in this 'mode' sub-folder (with the other jar files) or somewhere inside the JRE installation, like in C:\Program Files (x86)\Java\jre1.8.0_211\lib for example ?
• How do I make these 2 files available to the Java classpath ? (That's the million dollar question for us)

Your help, even guidelines or a simple hint, would be greatly appreciated.

Sincerely,

solub

The error I'm getting when running:

import numpy as np

is:

TypeError: Error when calling the metaclass bases
    'getset_descriptor' object is not callable at 1668:0 in defchararray.py

This occurs with the following setup:

• JyNI 2.7-alpha5
• Jython 2.7.1
• numpy 1.13.1
• macOS 10.12
• JDK 8u144

What can I do to help troubleshoot further? Am I missing a dependency?

This is exciting and looks very promising! I was able to download the binaries and run them on my machine (Arch Linux). I was able to import numpy and tested basic functionality, e.g. this works:

import numpy
img = numpy.zeros( (3, 4) ) + 3
random = np.random.rand()

I was not able to generate a random int, though:

>>> np.random.randint(3)
Traceback (most recent call last):
  File "mtrand.pyx", line 970, in mtrand.RandomState.randint (numpy/random/mtrand/mtrand.c:15903)
TypeError: argument of type 'dict' is not iterable

This is a migration to a separate issue from #2 . Following up on @Stewori's comment I looked for PyDict_Iter but my search was not successful. My best guess would be to get the iterator from the list returned by PyDict_Items when requested. Maybe that happens already and it will be resolved automatically once ListIter is supported.

I am not very familiar with the Python C API but I would definitely have a look into it. If you could point me to where the iterator methods for the python objects are defined, that would make it a lot easier for me!

You can probably add Arch Linux to the systems JyNI has been tested on. I did not build it myself, though.

If you are interested in why we would like to use JyNI: I created a compatibility layer between numpy and imglib2 based on pyjnius: imglib2-imglyb. This library is the core for modern ImageJ and with that compatibility layer we try to get the best of both worlds. The python code of imglyb needs only slight modificatoins to work with JyNI, and shipping JyNI with Fiji would probably be much easier than shipping a whole CPython distribution, so we are very interested in what happens with JyNI.

JyNI-C/src/Python/pythonrun.c:1956:19: error: storage size of ‘context’ isn’t known
struct sigaction context, ocontext;
^
JyNI-C/src/Python/pythonrun.c:1956:28: error: storage size of ‘ocontext’ isn’t known
struct sigaction context, ocontext;
^
JyNI-C/src/Python/pythonrun.c:1961:10: error: ‘SIG_ERR’ undeclared (first use in this function)
return SIG_ERR;

The recent release of Jython 2.7.2b2 breaks JyNI support.
Luckily, I was able to fix this in Jython, so there is no new JyNI release necessary for this issue. However, the fix is not included in Jython 2.7.2b2 but will be in the final 2.7.2 release (and in further betas if there should be any).

Even with this fix in place, I noticed that NumPy 1.13.3 is broken. However, NumPy 1.13.1 seems to work like usual. (1.13.2 and 1.x.y with x > 13 are broken anyway.)

So, to keep using NumPy with JyNI and Jython 2.7.2, stick to Jython master, 2.7.2b3+ or final release and role back from NumPy 1.13.3 to 1.13.1.

import pyspark
File "/usr/lib/python2.7/site-packages/pyspark/init.py", line 51, in
from pyspark.context import SparkContext
File "/usr/lib/python2.7/site-packages/pyspark/context.py", line 31, in
from pyspark import accumulators
File "/usr/lib/python2.7/site-packages/pyspark/accumulators.py", line 97, in
from pyspark.serializers import read_int, PickleSerializer
File "/usr/lib/python2.7/site-packages/pyspark/serializers.py", line 72, in
from pyspark import cloudpickle
File "/usr/lib/python2.7/site-packages/pyspark/cloudpickle.py", line 145, in
_cell_set_template_code = _make_cell_set_template_code()
File "/usr/lib/python2.7/site-packages/pyspark/cloudpickle.py", line 109, in _make_cell_set_template_code
return types.CodeType(
AttributeError: 'tablecode' object has no attribute 'co_stacksize'

Hi,

I have problems in the testing. When I run the datatime test, I got an error relative to the function leaveRecursiveCall. This is the output:

java -cp ../jython-standalone-2.7-b3.jar:build/JyNI.jar org.python.util.jython JyNI-Demo/src/JyNIDatetimeTest.py
[sudo] password for felix:
Traceback (most recent call last):
File "JyNI-Demo/src/JyNIDatetimeTest.py", line 15, in
import datetime
at JyNI.JyNI.initJyNI(Native Method)
at JyNI.JyNI.(JyNI.java:138)
at JyNI.JyNIImporter.load_module(JyNIImporter.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
java.lang.NoSuchMethodError: java.lang.NoSuchMethodError: leaveRecursiveCall

What am I missing?
Thanks,
Félix

Hey,

can you lay out some steps to get my feet wet in direction of Numpy Support? Maybe some small "junior" jobs would help in general, since this both involves Python-C-binding and Jython/JVM-C-binding experience.

Cheers,
Christian

Hello,

I don't understand well the sequence of steps to start usage of JyNI with numpy (Windows platform). I already created an issue on Jython site:
https://bugs.jython.org/issue2930
But I'm not sure that it is a correct place, maybe it is question to you.

I read your paper https://arxiv.org/pdf/1607.00825v1.pdf and tried to reproduce the simplest example:

import sys
sys.path.append(’path_to_numpy1.12_pre-release’)
import numpy as np
a = np.array([2, 5, 7])
...

But what is here the 'path_to_numpy1.12'? Where should I place the content of your archive JyNI-2.7-alpha.5-bin-win-amd64.zip ? Now I just unpacked it and placed both files into C:\JyNI , then added this directory to sys.path.append. But it seems that it is not a right way. Results are the following:

Traceback (most recent call last):
File "C:/siams/computer-vision/stare-python/jython-experiments/net/algart/jython/tests/SimpleJythonNumpyTest.py", line 3, in
import numpy as np
File "C:\jython2.7.2\Lib\site-packages\numpy_init_.py", line 142, in
from . import core
File "C:\jython2.7.2\Lib\site-packages\numpy\core_init_.py", line 71, in
raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the multiarray numpy extension module failed. Most
likely you are trying to import a failed build of numpy.
Here is how to proceed:

• If you're working with a numpy git repository, try git clean -xdf
  (removes all files not under version control) and rebuild numpy.
• If you are simply trying to use the numpy version that you have installed:
  your installation is broken - please reinstall numpy.
• If you have already reinstalled and that did not fix the problem, then:

  1. Check that you are using the Python you expect (you're using C:\jython2.7.2\bin\jython.exe),
     and that you have no directories in your PATH or PYTHONPATH that can
     interfere with the Python and numpy versions you're trying to use.

  2. If (1) looks fine, you can open a new issue at
     https://github.com/numpy/numpy/issues. Please include details on:

     • how you installed Python
     • how you installed numpy
     • your operating system
     • whether or not you have multiple versions of Python installed
     • if you built from source, your compiler versions and ideally a build log

     Note: this error has many possible causes, so please don't comment on
     an existing issue about this - open a new one instead.

Original error was: No module named _multiarray_umath

Hi all,

I've successfully compiled JyNI on OSX and none of the unit tests is failing.
But there's trouble with the Tkinter demo: java starts an osx application (with the Tk icon), that shows no window and is not responding at all.

Has anyone experienced the same issue?
I'll look into the problem and try to collect all information about it in this thread.

Best,
Malte

As you said, it's better to create a new issue,

When u speak about Nexus u probably refer to the phone, but me i refer to Maven and his nexus
(this : http://www.objis.com/formation-java/tutoriel-maven-deploiement-archiva.html)

I can confirm this to you, when u rename the JyNI.jar like Jyni.jar or JyNITEST.jar it doesn't work
I tried many times before contacting you

JyNI need to be called JyNI.jar but i don't know why

I've a Netbeans project which can confirm this to you if u want

I have been investigating #13, and I found that numpy.random.randint() segfaults occasionally. I'm not sure why it occurs so rarely or what the root cause is, but (after finding myself reading assembly) the actual seg fault is connected to PyObject_Call being passed PyExc_TypeError as the function to be called. This then causes a segfault as soon as a property of the ob_type is accessed (in this case: ob_type->tp_flags). I have written a simple C function that reliably causes this bug:

static PyObject* test_TypeError(PyObject *self, PyObject *args){
	PyObject* tup = PyTuple_Pack(1, Py_BuildValue("s","Hello"));
	PyObject_Call(PyExc_TypeError, tup, NULL);
	return tup;
}

CPython happily runs this (when it is part of a C extension), while JyNI segfaults. In JyNI, for PyExc_TypeError->ob_type==NULL, I think this is the cause of the segfault. I haven't been able to work out if PyExc_TypeError in CPython has ob_type==NULL. However, I can't find any checks for ob_type==NULL throughout the python codebase, which suggests the property should never be NULL. I did find a check in JyNI for this, and it has this comment just before it:

//JyNI note: if ob_type is NULL, we assume that base is a Type object that was
//not initialized because of our use-on-demand philosophy regarding PyType_Ready.

I don't quite understand what this means but it seems related, could you (@Stewori) possibly explain a bit about what is happening/what this means?

This could be solved in this case by working around it in the PyObject_Call function, it's possible to work out you are dealing with an error/exception and figure out what type without accessing any properties of ob_type. But if ob_type should never be NULL, perhaps it would be better to solve this more generally instead of trying to work around it.

With regards to numpy.random.randint(), it is possible that this error is always caught internally since it doesn't seem to set an error message or call the type error properly. Maybe in JyNI it just segfaults before it can be caught. But I'm not entirely sure, I haven't been able to track the particular problem down as it seems to be affected by timing (so adding print statements or running in debug mode make it less likely to occur), and it is random when it occurs(so I need to run numpy.random.randint() millions of times to get the error to appear).

Hello @Stewori

I'm currently using JyNI to import Numpy in a Java APP. but now i need to import SciPy or Scikit-Learn

I would like to know if there is a way to import them, or if JyNI need an update to import them

if it need an update could u explain me how to do it? or are u planning to do it?

I have explored supporting matplotlib font loading, which require PyFile support.

I have commented in

FILE *
PyFile_AsFile(PyObject *f)
{
    printf("Accessing file.\n");
    if (f == NULL || !PyFile_Check(f))
        return NULL;
    else
        return ((PyFileObject *)f)->f_fp;
}

Which is called and immediately SIG_ABORTS. I get a dump file from the JVM:

Register to memory mapping:

RAX=0x00007fb7cbdd8280: PyFile_Type+0 in /home/christian/Development/JyNI/build/libJyNI.so at 0x00007fb7cbaa9000
RBX=0x00007fb7f8070958 is an unknown value
RCX=0x00007fb7c055d390 is an unknown value
RDX=0x0000000000000001 is an unknown value
RSP=0x00007fb8278190e8 is pointing into the stack for thread: 0x00007fb82000a000
RBP=0x00007fb7f8032e30 is an unknown value
RSI=0x00007fb7c055d390 is an unknown value
RDI=0x00007fb7f8032e30 is an unknown value
R8 =0x0000000000000000 is an unknown value
R9 =0x0000000000000004 is an unknown value
R10=0x0000000000000319 is an unknown value
R11=0x00007fb7cbb08b65: PyErr_SetString+0 in /home/christian/Development/JyNI/build/libJyNI.so at 0x00007fb7cbaa9000
R12=0x0000000000000000 is an unknown value
R13=0x0000000000000000 is an unknown value
R14=0x00007fb827819130 is pointing into the stack for thread: 0x00007fb82000a000
R15=0x0000000000000001 is an unknown value


Stack: [0x00007fb82771f000,0x00007fb827820000],  sp=0x00007fb8278190e8,  free space=1000k
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 3935  JyNI.JyNI.callPyCPeer(JLorg/python/core/PyObject;Lorg/python/core/PyObject;J)Lorg/python/core/PyObject; (0 bytes) @ 0x00007fb81183612c [0x00007fb8118360c0+0x6c]
j  JyNI.PyCPeerType.__call__([Lorg/python/core/PyObject;[Ljava/lang/String;)Lorg/python/core/PyObject;+36
J 2911 C1 org.python.core.PyObject.__call__(Lorg/python/core/PyObject;)Lorg/python/core/PyObject; (16 bytes) @ 0x00007fb811ae0fec [0x00007fb811ae0e80+0x16c]
J 5454 C2 org.python.core.PyObject.__call__(Lorg/python/core/ThreadState;Lorg/python/core/PyObject;)Lorg/python/core/PyObject; (6 bytes) @ 0x00007fb8125b22e0 [0x00007fb8125b22a0+0x40]
j  matplotlib.font_manager$py.createFontList$16(Lorg/python/core/PyFrame;Lorg/python/core/ThreadState;)Lorg/python/core/PyObject;+673
j  matplotlib.font_manager$py.call_function(ILorg/python/core/PyFrame;Lorg/python/core/ThreadState;)Lorg/python/core/PyObject;+368

Just returning NULL is not sufficient as a quick hack and I guess that I need to wrap a Jython class for PyFile instead of accessing the pointer directly. You said it would be easy to fix, but in the comment you say:

//Providing the following method via JyNI would be very hard or impossible.
//However, we could create a FILE-Pointer to the same file and also adjust
//io mode and seek-position to the values of the given PyFile.
//But this may lead to conflicts in systems not supporting multiple file
//access (i.e. windows) and would not stand equality-checks via pointer-wise "==".

I had some frustrations with filesystem APIs lately for my kv-store implementation, so I would also be careful with double access. What are your suggestions to proceed? Also do you have some chat channel? This might be easier to get started.

Factoring out from #11

Yes, it is reproducible. I have just tried to find the part in matplotlib where it is called to shrink the case, but it is a bit messy with all the backend monkey patching. I am not sure where show() calls into multiarray (and why), I can instrument with print debugging, but this is a bit inefficient. I tried to use the Emacs Python debugger with CPython, but I have had no success yet. Any suggestions? How do you pin things like that down?

Since the crash happens in multiarray, and matplotlib uses numpy, I think it is a numpy issue.