Comments (21)
After adding .../numpy/core folder to sys.path I can do
import umath
without failure. Like with issues #7 and #8, can you provide a minimal code sample that triggers this issue for you?
from jyni.
Same behavior with self-built current NumPy.
Also: I scanned through umathmodule.c and it appears there is no line
PyObject *numpy = PyImport_ImportModule("numpy.core.multiarray");
and actually no call PyImport_ImportModule
in the entire file.
Is there a typo in the issue description?
from jyni.
With the latest commit that enables raising of import-exceptions in JyNI imports the issue is now also visible on my system. Here it shows up as
ImportError: numpy.core.multiarray failed to import
I'll now try to follow your hints for the cause...
from jyni.
You did quite a good job in identifying the source of this issue. I think I fixed loading of dotted module names in 66a74e0.
This actually solves the _ARRAY_API
problem.
However umath still doesn't load; now it segfaults at a later point. I'll continue on this front tomorrow or so.
from jyni.
Great. Just another step forward. I will try your fix at the weekend and report the behaviour with my setup. Same with the other issue.
I would then continue to look at the next bug on the way to load umath. I will keep you up to date to avoid duplicate work.
from jyni.
Sounds good. I already started investigation a bit, but it appears to be a difficult bug. It segfaults at about four different possible locations and has also a small chance of passing through, like your reported about multiarray.
Only thing I found out so far is that the crash always relates to the type
numpy.ufunc
Sometimes during creation, sometimes during cleanup. My suggestion would be to reduce the function initumath
step by step from the end until a working partial implementation was discovered. Then we can expand from that point and fix problems one by one. Hopefully this then sorts the possible points of failure a bit.
from jyni.
Also note that since import umath
triggers an import numpy.core.multiarray
, Jython will also attempt to import numpy
and numpy.core
, i.e. it will process numpy/__init__.py
and numpy/core/__init__.py
, which is both currently not feasible. To focus on umath in this thread I mostly out-commented these init-procedures and you should do so as well. Of course they need to be made workable too, but that should be investigated in distinct issue threads.
from jyni.
Okay, already fixed it as of 238ba94 I suppose. Took some time, but I traced it around until I concluded it's because ufunc
uses PyMem_Malloc (indirectly via PyArray_Malloc) for object allocation. My original assumption was that only PyObject_Malloc would be used for PyObject allcoation while PyMem_Malloc was only intended for low-level purpose memory. However once recognized it could be fixed by modifying PyMem_Malloc and friends accordingly.
Now umath
module loads without error on my system, given that __init__.py
module initializers of numpy and numpy.core are stripped down to a workable portion for now.
(If you can confirm this you can close this issue then...)
from jyni.
In my case the program stops, when trying to load _ctypes. But JyNIctypesDemo.sh is also not running. It looks right now, that my setup with the ctypes library is not working. I have to look into that before I a can give an answer to the status of this issue.
from jyni.
In case it's not just because of an invalid setup, please file a ctypes-scoped issue here. Getting ctypes support stable has even higher priority than NumPy.
But JyNIctypesDemo.sh is also not running.
What about JyNI-Demo/src/test_JyNI_ctypes.py
? (as called by JyNI_unittest.sh)
from jyni.
Was just a setup error which I had fixed for the last fork as well, but forgot to mention. On Suse Linux the dyn-load path is different.
This line
lib_dynload = '/usr/lib64/python2.7/lib-dynload'
in JyNI-Lib/ctypes/init.py fixes it. Just add "64".
The error message was: ImportError: No module named _ctypes
from jyni.
Yes, managing Python-path/Jython-path/Java-classpath whatever properly for JyNI is a topic for beta phase I suppose. In general this topic is more complex than just adding the right path, because we will also have to resolve conflicts between native modules and Jython-supplied versions (e.g. datetime and also ctypes so far). Actually this is something we should maybe also discuss on Jython-dev at its time.
I suppose your message implies, umath now loads on your system...?
from jyni.
An import of umath is not succesfull. The program just stops at one point. Reminds me of the deadlock I described in another issue, but I have seen you added the fix. :) Have to look into the details to give a meaningful answer.
BTW: import numpy results in this error: NameError: name 'NUMPY_SETUP' is not defined
from jyni.
Okay. Looking forward to hear about the location it hangs on! Try to add debug output to ENTER_JYNI
and LEAVE_JYNI
in JyNI.h
if not yet done.
Regarding NUMPY_SETUP'
, let's first identify the script location that triggers this (in a distinct issue would be ideal).
from jyni.
Still looking into this. Debug output is influencing the behavior of the program control. Therefore limiting down the location is possible, but does not lead to the cause of the error.
from jyni.
Did you apply jstack to identify the deadlock when it is hanging? (Please consider to post the plain jstack-output here too, so I can also have a look.)
Sensitivity to log-output points to a parallelity-related issue (usually this is main thread vs something gc-related). In such a case I usually out-comment code step-wise to identify the latest stable point. Also, feel free to post to what you limited down the location.
I am currently working to fix another deadlock bug (gc again) that popped up in a JyOpenGL test. Probably it's better if you wait for this commit, just in case it's the same bug. Hope to get it done until tomorrow evening.
from jyni.
(fixed the other (maybe unrelated) deadlock issue as of the last two commits)
from jyni.
BTW: import numpy results in this error: NameError: name 'NUMPY_SETUP' is not defined
This side-issue is fixed as of ed06aa7.
Now import numpy
fails at a later point with this symbol-error:
symbol lookup error: /data/workspace/linux/numpy/numpy/core/umath.so: undefined symbol: PyThreadState_GetDict
from jyni.
Fixed the symbol error
symbol lookup error: /data/workspace/linux/numpy/numpy/core/umath.so: undefined symbol: PyThreadState_GetDict
as of 9fa540f.
Now there are issues regarding static vs. dynamic memory detection. scalartypes.c
declares boolean-singletons:
NPY_NO_EXPORT PyBoolScalarObject _PyArrayScalar_BoolValues[] = {
{PyObject_HEAD_INIT(&PyBoolArrType_Type) 0},
{PyObject_HEAD_INIT(&PyBoolArrType_Type) 1},
};
JyNI segfaults on these, because its current detection approach misconcepts them as dynamically allocated PyObjects. It appears that numpy always calls PyObject_Init
on dynamically created objects. I will try to improve the detection mechanism based on this fact.
from jyni.
With 54c8e16 import numpy
passes properly on my system. It would be good if you could check this issue after a fresh pull. Also see #2 (comment)
(Only works with latest numpy 1.12 dev-version from repository)
from jyni.
Given that JyNI significantly changed since the last activity here and I cannot reproduce this issue any more for two months or so now, I will close it.
If you encounter problems again, feel free to re-open.
from jyni.
Related Issues (20)
- Iterator support (TypeError: argument of type 'dict' is not iterable) HOT 10
- Request to explore JavaCPP as an extension to model Python FFI. HOT 4
- JyNI-C/src/Python/pythonrun.c:1956 isnt' know struct sigaction context ocontext HOT 5
- 'getset_descriptor' object is not callable at 1668:0 in defchararray.py HOT 3
- Patch: NumPy 13.2 broken for JyNI 2.7-alpha.5 HOT 1
- nltk download crashes HOT 15
- NumPy 1.14+ compatibility HOT 8
- Name JyNI.jar Mandatory HOT 15
- support for the BufferProtocol HOT 8
- SciPy HOT 1
- mmap: String access is truncated or causes NullPointerException HOT 11
- Add debug target to makefile. HOT 2
- PyExc_TypeError has ob_type == NULL HOT 25
- Under memory stress JVM builds an invalid native jobject, potentially causing segfault during GC
- When using numpy, we call np.split(x,2),np.hsplit(x,2),np.vsplit(x,2), x is a numpy array. But the command didn't output anything or report any mistakes HOT 1
- Setting JyNI to work with Processing HOT 5
- Jython 2.7.2, NumPy advice HOT 1
- Can't seem to load the co_stacksize extension
- How to correcty install and use JyNI with numpy support on Jython? HOT 2
- custom module imported but functions not exported HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jyni.