Comments (5)
Every thing is fine when go Calc("aaa.csv")
called the first time. But the fatal error emits the second time when go Calc("aaa.csv")
is called.
from go-python3.
I haven't looked at your code in detail, but one thing I've noticed is the absence of any <PyObject>.DecRef()
calls. Python objects referenced in Go should get decref'ed when they're no longer needed (i.e. bArgs.DecRef()
).
from go-python3.
It seems we could only PyImport_ImportModule
some module for only one time. And even after that, calling functions in multiple goroutines still hit similar problems.
from go-python3.
Describe what happened:
Environments:
On MacOS (Catalina Version 10.15.4)
Python3.7.6
Go1.13.8
I want to use go-python3 to invoke an algorithm written in Python3, but as described, an fatal error will generated when the second time I invoke this algorithm. From the output message, it seems thatPyImport_ImportModule
causes this error.fatal error: unexpected signal during runtime execution [signal SIGSEGV: segmentation violation code=0x1 addr=0xa pc=0x91256a3] runtime stack: runtime.throw(0x4967a75, 0x2a) /usr/local/go/src/runtime/panic.go:774 +0x72 runtime.sigpanic() /usr/local/go/src/runtime/signal_unix.go:378 +0x47c goroutine 41 [syscall]: runtime.cgocall(0x4637740, 0xc000063c18, 0x48a4660) /usr/local/go/src/runtime/cgocall.go:128 +0x5b fp=0xc000063be8 sp=0xc000063bb0 pc=0x4004d0b github.com/DataDog/go-python3._Cfunc_PyImport_ImportModule(0x8061d90, 0x0) _cgo_gotypes.go:3780 +0x4a fp=0xc000063c18 sp=0xc000063be8 pc=0x462c2fa github.com/DataDog/go-python3.PyImport_ImportModule(0x49501f5, 0x8, 0x0) /Users/zhao/go/pkg/mod/github.com/!data!dog/[email protected]/import.go:24 +0x87 fp=0xc000063c80 sp=0xc000063c18 pc=0x462e267 PPGServer/pkg/algo.ImportModule(0x4964926, 0x26, 0x49501f5, 0x8, 0x1) /Users/zhao/go/src/PPGServer/pkg/algo/ppg.go:42 +0x4cb fp=0xc000063d98 sp=0xc000063c80 pc=0x46332db PPGServer/pkg/algo.CalcPre(0xc0003560c0, 0xd, 0x0, 0x0) ....
Here is the sample code.
A wrapper of PyImport_ImportModule:// ImportModule will import python module from given directory func ImportModule(dir, name string) *python3.PyObject { fmt.Println("python3.PyImport_ImportModule before") sysModule := python3.PyImport_ImportModule("sys") // import sys fmt.Println("python3.PyImport_ImportModule success") path := sysModule.GetAttrString("path") // path = sys.path ob := python3.PyList_GetItem(path, 1) fmt.Println("check:", python3.PyUnicode_Check(ob)) fmt.Println("path:", python3.PyUnicode_AsUTF8(ob)) fmt.Println("sysModule.GetAttrString success") python3.PyList_Insert(path, 0, python3.PyUnicode_FromString("/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages")) python3.PyList_Insert(path, 0, python3.PyUnicode_FromString(dir)) fmt.Println("After module insert:", python3.PyUnicode_AsUTF8(python3.PyList_GetItem(path, 0))) fmt.Println("module name:", name) return python3.PyImport_ImportModule(name) }
Each time the algorithm is called in a goroutine.
func CalcPre(dataFilePath string) (sbpI int, dbpI int) { python3.Py_Initialize() if !python3.Py_IsInitialized() { fmt.Println("Error initializing the python interpreter") os.Exit(1) } gstate = python3.PyGILState_Ensure() fmt.Println("Py_Initialize success") vbp := ImportModule("/Users/zhao/Desktop/lab/ppython", "value_bp") fmt.Println("ImportModule success") b := vbp.GetAttrString("estimate") fmt.Printf("[FUNC] b = %#v\n", b) bArgs := python3.PyTuple_New(1) python3.PyTuple_SetItem(bArgs, 0, python3.PyUnicode_FromString(dataFilePath)) re := b.Call(bArgs, python3.Py_None) sbp := python3.PyTuple_GetItem(re, 0) dbp := python3.PyTuple_GetItem(re, 1) defer func() { python3.Py_Finalize() fmt.Println("python3.Py_Finalize()") }() sbpI = python3.PyLong_AsLong(sbp) dbpI = python3.PyLong_AsLong(dbp) python3.PyGILState_Release(gstate) return } func Calc(dataFilePath string) { CalcPre(dataFilePath) }
Sample caller like this:
go Calc("aaa.csv")
.Describe what you expected:
We can call go Calc() many times without a fatal error.Steps to reproduce the issue:
Use the code above and environments above, put these code into a goroutine, likego Calc("aaa.csv")
.For simplicity, you may just remove the algorithm part and just have the skeleton remained.
Hello, do you have a solution to this problem?
from go-python3.
maybe as a starting point:
- here's an example for creating a Go function around a Python function that you can loop over (= it works a second time): code / explanation
- if you add locking and releasing the GIL and locking the thread to it you should be able to use it in a go routine. Here's a example (it uses the sbinet/go-python instead of datadog/go-python3 but you get the idea). code / explanation
from go-python3.
Related Issues (20)
- How do I create an Object HOT 2
- fatal error: Python.h: No such file or directory #include "Python.h" HOT 4
- get PyObject's refcount
- Error: C.PyEval_ReInitThreads
- could not determine kind of name for C.PyImport_GetModule HOT 2
- How to stop or terminate python execution? HOT 1
- Hi, How to capture function return when use python3.PyRun_AnyFile HOT 2
- could not determine kind of name for C.PyDict_ClearFreeList HOT 16
- PyEval_ReInitThreads removed since python 3.8 HOT 2
- build error on debian HOT 6
- Undefined reference to 'Py_True' HOT 4
- No module named xxx problem ,when the xxx is a so file .. HOT 2
- How could I Pass parameters from the Go side to the python file? HOT 2
- run on windows HOT 3
- SyntaxError: invalid syntax (uuid.py, line 138) HOT 1
- PyList_Insert and PyList_Append functions has an extra b HOT 2
- Goroutine calls Python but deadlocks HOT 2
- how can i use tensorflow with library HOT 2
- How do I create a Python bytes object HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from go-python3.