gjcarneiro / pybindgen Goto Github PK

The docs state that pygccxml doesn't work with python3 which isn't the case (at least now!)

however current version of pybindgen doesn't seem to work with it.

this is what happens when I run the last code fragment in the tutorial

./gen-make-binding.py dunno
Traceback (most recent call last):
  File "./gen-make-binding.py", line 14, in <module>
    my_module_gen()
  File "./gen-make-binding.py", line 11, in my_module_gen
    module_parser.parse([sys.argv[1]], includes=['"../raylib/src/raylib.h"'], pygen_sink=FileCodeSink(sys.stdout))
  File "/home/chris/.local/lib/python3.8/site-packages/pybindgen/gccxmlparser.py", line 598, in parse
    self.parse_init(header_files, include_paths, whitelist_paths, includes, pygen_sink,
  File "/home/chris/.local/lib/python3.8/site-packages/pybindgen/gccxmlparser.py", line 694, in parse_init
    self.gccxml_config = parser.gccxml_configuration_t(**gccxml_options)
AttributeError: module 'pygccxml.parser' has no attribute 'gccxml_configuration_t'

To get my app to work on Windows, I had to fix the generated .c file to add the usual annoying __declspec(dllexport) thing for the PyInit function. I'm surprised this hasn't come up before! I gather there is a _wrap_ macro that should be used for this.

in module.py:841:

    #define MOD_INIT(name) PyObject* PyInit_##name(void)

needs to be wrapped or conditionally have the __declspec(dllexport) added for windows:

    #define MOD_INIT(name) PyObject* __declspec(dllexport) PyInit_##name(void)

I'm not using version 2 but probably that has the same issue?

This is for https://github.com/go-python/gopy -- using pybindgen to access Go from python. Thanks!

on Xcode 7 or clang-3.6, ns-3 bindings generated by pybindgen raise many warnings such as below for potentially evaluated expression in the argument:

src/topology-read/bindings/ns3module.cc:1482:16: error: expression with side effects will be evaluated despite being used as an operand to 'typeid' [-Werror,-Wpotentially-evaluated-expression]
if (typeid((*const_cast<ns3::Node *> (ns3::PeekPointer (retval)))).name() == typeid(PyNs3Node__PythonHelper).name())

To reproduce, check out ns-3.24 release (not ns-3.24.1) and compile with python bindings enabled on either clang-3.6 or Xcode 7.0 machine (may have to disable Werror).

We are suppressing this for now in the wscript, but I wonder if an upstream fix is possible to move the possibly side-effect-inducing expression out of the argument.

Is this project dead?, it is already 9 months since it has no update. Also in the documentation says that there is official support only until Python 3.4

In relation with #5 , I tried to bind a container of non-copyable data types (They inherit from boost::noncopyable) but pybindgen explictly tries to copy the inner data types. Given how the the code is generated right now, I don't think that there is an easy way to solve this within Pybindgen (I ended up creating a free function that would return a copyable version of the non-copyable objects) but it would be nice to maybe specify it in the documentation ?

Cheers,
Hervé

I have a C library containing several pointer-to-pointers that are tripping up the typehandler.

star-star.tar.gz contains a reproducer that has the following files:

my-module.h

void Print (int **value);

my-module.c

#include <stdio.h>
void Print (int **value)
{
  printf ("%d\n", **value);
}

mymodulegen.py

from pybindgen import *

def generate(file_):
    mod = Module('mymodule')
    mod.add_include('"my-module.h"')
    mod.add_function('Print', None, [param('int **', 'value')])
    mod.generate(file_)

setup.py
See attached.

When I try to run by typing "python setup.py build" I get the following error
pybindgen.typehandlers.base.TypeLookupError: ['int **']

I looked in pybindgen/examples and pybindgen/tests and could not find any pointer-to-pointers. I also looked in pybindgen/pybindgen/typehandlers/*.py and noticed that pointer types are supported, but not any pointer-to-pointer types.

Please recommend where I can go from here. Thank you!

this is the output of ./waf coomand
Waf: Entering directory /home/lucky/ns-allinone-3.29/pybindgen/build' Waf: Leaving directory /home/lucky/ns-allinone-3.29/pybindgen/build'
source not found: 'version.py' in bld(source=[/home/lucky/ns-allinone-3.29/pybindgen/pybindgen/init.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/castxmlparser.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/container.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/converter_functions.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/cppattribute.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/cppclass.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/cppclass_container.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/cppcustomattribute.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/cppexception.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/cppmethod.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/enum.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/function.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/gccxmlparser.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/module.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/overloading.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/pytypeobject.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/settings.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/utils.py, /home/lucky/ns-allinone-3.29/pybindgen/pybindgen/wrapper_registry.py, 'version.py'], target='', meths=['process_rule', 'process_source'], features=[], path=/home/lucky/ns-allinone-3.29/pybindgen/pybindgen, idx=1, tg_idx_count=1, install_path='${PYTHONDIR}/pybindgen', prio=50, posted=True) in /home/lucky/ns-allinone-3.29/pybindgen/pybindgen

I am trying to update/fix several things in the nix ns3 and DCE packages so I wanted to fetch pybindgen from github (with the latest castxml patch) instead of pypi but the scm version number seems to break ns3 identification:

Checking for python module 'pybindgen'                             : 0.0.post1+gef30ba2.d20180527 
Checking for pybindgen version                                     : 0.0.post1+gef30ba2.d20180527 
Traceback (most recent call last):
  File "/home/teto/scratch/source/.waf-1.8.19-b1fc8f7baef51bd2db4c2971909a568d/waflib/Scripting.py", line 110, in waf_entry_point
    run_commands()
  File "/home/teto/scratch/source/.waf-1.8.19-b1fc8f7baef51bd2db4c2971909a568d/waflib/Scripting.py", line 171, in run_commands
    ctx=run_command(cmd_name)
  File "/home/teto/scratch/source/.waf-1.8.19-b1fc8f7baef51bd2db4c2971909a568d/waflib/Scripting.py", line 162, in run_command
    ctx.execute()
  File "/home/teto/scratch/source/.waf-1.8.19-b1fc8f7baef51bd2db4c2971909a568d/waflib/Configure.py", line 87, in execute
    super(ConfigurationContext,self).execute()
  File "/home/teto/scratch/source/.waf-1.8.19-b1fc8f7baef51bd2db4c2971909a568d/waflib/Context.py", line 93, in execute
    self.recurse([os.path.dirname(g_module.root_path)])
  File "/home/teto/scratch/source/.waf-1.8.19-b1fc8f7baef51bd2db4c2971909a568d/waflib/Context.py", line 134, in recurse
    user_function(self)
  File "/home/teto/scratch/source/wscript", line 464, in configure
    conf.recurse('bindings/python')
  File "/home/teto/scratch/source/.waf-1.8.19-b1fc8f7baef51bd2db4c2971909a568d/waflib/Context.py", line 134, in recurse
    user_function(self)
  File "/home/teto/scratch/source/bindings/python/wscript", line 198, in configure
    if not (split_version(pybindgen_version) >= split_version(REQUIRED_PYBINDGEN_VERSION)):
  File "/home/teto/scratch/source/bindings/python/wscript", line 75, in split_version
    return (int(ver[0]), int(ver[1]), int(ver[2]), int(ver[3].split('post')[1]))
ValueError: invalid literal for int() with base 10: 'post1'

I don't know this setting but I wonder if it could be configured in a way that better matches current version numbers

Also it makes fetching the package from github harder since one can't use the github archive anymore.
NixOS/nixpkgs#41136 (comment) / https://github.com/kennethreitz/setup.py/issues/21

Hi!
Thank you for your work!
In the process of using it, I did not understand how to make multiple returns of values from a function. PyPI says "Supports multiple return values (for example, due to output/input parameters);" but I have not found a single test or example of how to implement this. I really hope for your help.
Thank you in advance!

One https://pypi.org/project/PyBindGen/ the home page is still https://launchpad.net/pybindgen which then lead to a homepage of https://code.google.com/archive/p/pybindgen/ ... ;)
I think both are outdated?

The recent set of pull requests from Cosmos Nicolaou has caused a regression in the compilation of ns-3 bindings.

This can be reproduced by using the latest pybindgen with the latest ns-3-dev, using the bake tool, such as follows:

$ git clone https://gitlab.com/nsnam/bake.git
$ cd bake
$ python3 bake.py configure -e ns-3-allinone
$ python3 bake.py download
$ python3 bake.py build -vvv

The compilation error is:

src/core/bindings/ns3module.cc:50653:18: error: ‘None’ was not declared in this scope
     (ternaryfunc)None,          /* tp_call */
                  ^~~~
src/core/bindings/ns3module.cc:50653:18: note: suggested alternative: ‘clone’
     (ternaryfunc)None,          /* tp_call */
                  ^~~~
                  clone

Waf: Leaving directory `/home/buildslave/git/bake/source/ns-3-dev/build'

The relevant part of ns3module.cc is here:

PyTypeObject PyNs3CallbackImpl__Ns3ObjectBase___star___Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Type = {
    PyVarObject_HEAD_INIT(NULL, 0)
    (char *) "_core.CallbackImpl__Ns3ObjectBase___star___Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty",            /* tp_name */
    sizeof(PyNs3CallbackImpl__Ns3ObjectBase___star___Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty),                  /* tp_basicsize */
    0,                                 /* tp_itemsize */
    /* methods */
    (destructor)_wrap_PyNs3CallbackImpl__Ns3ObjectBase___star___Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty_Ns3Empty__tp_dealloc,        /* tp_dealloc */
    (printfunc)0,                      /* tp_print */
    (getattrfunc)NULL,       /* tp_getattr */
    (setattrfunc)NULL,       /* tp_setattr */
#if PY_MAJOR_VERSION >= 3
    NULL,
#else
    (cmpfunc)NULL,           /* tp_compare */
#endif
    (reprfunc)NULL,             /* tp_repr */
    (PyNumberMethods*)NULL,     /* tp_as_number */
    (PySequenceMethods*)NULL, /* tp_as_sequence */
    (PyMappingMethods*)NULL,   /* tp_as_mapping */
    (hashfunc)NULL,             /* tp_hash */
    (ternaryfunc)None,          /* tp_call */
    (reprfunc)NULL,              /* tp_str */
    (getattrofunc)NULL,     /* tp_getattro */
    (setattrofunc)NULL,     /* tp_setattro */
    (PyBufferProcs*)NULL,  /* tp_as_buffer */
    Py_TPFLAGS_DEFAULT,                      /* tp_flags */

The (ternaryfunc) is not NULL but None, causing the error.

I rolled back pybindgen to commit adeb7ce (25 Dec 2019) and the issue disappears, but I didn't try to bisect further through the merged pull requests.

Hi,
With the latest PR it might be worth adding to setup.py a mention to cxxfilt as a mandatory or optional dependency (I am not sure which it is, it is certainly necessary for ns3 bindings)

Recently the master branch of pygccxml was updated and ported to python3 so it might be helpful if the development version is at least updated according to it.

Let's say that my C++ code has

namespace {
   const std::string foo = "Hello";
}

What is the right way for me to expose the foo constant via PyBindGen?

Option to generate bindings against stable ABI for binary compatibility between different minor versions of py3 would be a good thing.

Here's some related discussion about the subject on SWIG project: swig/swig#1009

Hi there, great project.

Would it be possible to add a type handler for long and long * data types?

At present they produce this error UserWarning: exception TypeLookupError(['long int *']) , after wrapping original header functions that have long *.

If this project is still being actively developed this could help support the common data type. Otherwise, at least in this instance for me, a project does not appear able to be wrapped automatically using PyBindGen.

Thanks!!

I have a longstanding question about building pybindgen. The README says that Waf can be used instead of setuptools; however, if one does not use setuptools first, there will be no version.py file generated and the Waf build will fail.

Should Waf be extended to create version.py upon configuration?

Some new proposed code for ns-3 makes use of an abstract base class interface, in which one of the methods returns a const reference to a struct. A minimal version of this is shown here:

struct AStruct
{
  int a;
};
class ReturnConstRef : public Object
{
public:
  static TypeId GetTypeId ();
  virtual const AStruct & ReturnMyAStruct (void) = 0;
};

This API can be successfully scanned, but compilation later fails. The binding generated for this is:

+    ## return-const-ref.h (module 'core'): ns3::AStruct const & ns3::ReturnConstRef::ReturnMyAStruct() [member function]
+    cls.add_method('ReturnMyAStruct', 
+                   'ns3::AStruct const &', 
+                   [], 
+                   is_pure_virtual=True, is_virtual=True)
+    cls.set_cannot_be_constructed("pure virtual method %r not wrapped" % 'ReturnMyAStruct')

The specific error is:

DEBUG:pybindgen.typehandlers:try to lookup type handler for 'ns3::AStruct const &' => success ((<class 'pybindgen.cppclass.CppClass.__init__.<locals>.ThisClassRefReturn'>, None, <pybindgen.typehandlers.ctypeparser.TypeTraits object at 0x7f064bfcfeb8>))
...
  File "/path/to/bindings/build/lib/PyBindGen-0.20.0.post0+g82c5fd2.d20190829-py3.5.egg/pybindgen/cppclass.py", line 3307, in get_c_error_return
pybindgen.typehandlers.base.NotSupportedError

In castxmlparser.py, it is failing at this point:

                try:
                    method_wrapper = class_wrapper.add_method(member.name, return_type, arguments, **kwargs)
                    method_wrapper.castxml_definition = member
                except NotSupportedError as ex:
                    if pure_virtual:
                        class_wrapper.set_cannot_be_constructed("pure virtual method %r not wrapped" % member.name)
                        class_wrapper.set_helper_class_disabled(True)
                        pygen_sink.writeln('cls.set_cannot_be_constructed("pure virtual method %%r not wrapped" %% %r)'
                                           % member.name)
                        pygen_sink.writeln('cls.set_helper_class_disabled(True)

Is this a limitation in pybindgen that needs to be fixed, or is there an underlying reason that this construct can't be supported? Note that if the method returns a POD type, it compiles, or if the method is not pure virtual, it also compiles; it is the combination of returning a const reference to a non-POD type from a pure virtual method that is failing.

I can rewrite the ns-3 API to work around this but wonder if the longer term solution should be in pybindgen.

When trying to scan the bindings for a module (ns-3), I get an error with the tokens because it is None. Tracking down the error, I see the cast parser uses the partial declaration to normalize a name [normalize_name(alias.decl_type.partial_decl_string)]. However, the problem comes because the 'decl_string' and 'partial_decl_string' are different:

namespace ns3 {

struct GabA
{
  uint32_t m_varU {0};
  double m_varD {0.0};
};

struct GabD
{
  uint32_t m_varU {0};
  double m_varD {0.0};
};

typedef std::function<GabD & (const std::shared_ptr<GabA> &ga)> GabSharedFn;
typedef std::function<GabD & (const std::unique_ptr<GabA> &ga)> GabUniqueFn;
}

When the code is parsed, these are the decl_string and partial_decl_string for the two typedefs:

1. GabSharedFn

partial_decl_string:'::std::function< ns3::GabD &(const std::shared_ptr< ns3::GabA > >'
decl_string:'::std::function<ns3::GabD &(const std::shared_ptr<ns3::GabA> &)>'

2. GabUniqueFn

partial_decl_string:'::std::function< ns3::GabD &(const std::unique_ptr< ns3::GabA, std::default_delete< ns3::GabA > > >'
decl_string:'::std::function<ns3::GabD &(const std::unique_ptr<ns3::GabA, std::default_delete<ns3::GabA> > &)>'

As you can see, both partial declaration strings are missing &) and that is why I get the error. I had to hard-code this inside the typehandlers directory because I could not figure out where the classes are parsed (i.e., where partial_decl_string and decl_string are created). It seems to me that pygccxml takes care of that but I am not sure. Do you know where the partial_decl_string and decl_string are created?

I would appreciate your feedback.

Thanks!

Hi ,

This is a courtesy to let you know that commit 71852b1 with the gccxml replacement appears to have broken the PyBindGen tutorial listed at https://pybindgen.readthedocs.io/en/latest/tutorial/#header-file-scanning

with its accompanying code below:

#! /usr/bin/env python

import sys

import pybindgen
from pybindgen import FileCodeSink
from pybindgen.gccxmlparser import ModuleParser

def my_module_gen():
module_parser = ModuleParser('a1', '::')
module = module_parser.parse([sys.argv[1]])
module.add_include('"a.h"')

pybindgen.write_preamble(FileCodeSink(sys.stdout))
module.generate(FileCodeSink(sys.stdout))

if name == 'main':
my_module_gen()

I have a C++ library with one function that takes a function pointer (let's call it register(my_callback_t cb)) as a callback. I want to wrap this with PyBindGen so that I can call the register function from Python, passing in a Python function object as the function to call in the callback.

Based on examples, I created the ForwardWrapperBase derived class for the my_callback_t function type, where I map the Python caller's PyObject * for the callback to the internal C++ callback. Then, when the internal C++ callback is called by my library, I manually call the Python object again, using the following pattern:

PyGILState_STATE gstate;
gstate = (PyEval_ThreadsInitialized() ? PyGILState_Ensure() : (PyGILState_STATE) 0);

// Convert C++ arguments to Python objects
PyObject *arglist = Py_BuildValue("(sssO)", id.c_str(), name.c_str(), value.c_str(), valid ? Py_True: Py_False);

// Using the ID, find the PyObject * which corresponds to the registered callback
PyObject *result = PyObject_CallObject(cb_map.at(id), arglist);
Py_DECREF(arglist);

if (NULL == result)
{
    if (NULL == PyErr_Occurred())
    {
        printf("@@@ PythonBinging: Unknown error occurred in subscription callback\n");
    }
    else
    {
        PyErr_Print();
    }
}
Py_XDECREF(result);

if (PyEval_ThreadsInitialized())
{
  PyGILState_Release(gstate);
}

The above works well for the most part, but occasionally causes SEGFLT in the main python thread after the entirety of both the Python and C++ callbacks complete.

The two questions I have are:

1. Do you see something wrong with the above?
2. I noticed that there is a thing called ReverseWrapperBase in PyBindGen, which seems to be related to C code calling back into Python. However, I cannot figure out how to use it and there do not appear to be any examples using it. Can you please provide an example?

module.add_container() fails if the type you are trying to wrap is a container of a container, for example std::vector<std::vector<uint8_t>>.

After some debugging I found that the root cause is that the tokenizer is treating the ending >> as a single token, instead of two separate closing brackets.

A workaround is to insert a space between the brackets when registering the container, for example: mod.add_container('std::vector<std::vector<uint8_t> >', 'std::vector<uint8_t>', 'vector'), this is tokenized properly.

I ran into issues trying to use C++11 scoped enums with pybindgen, but found a relatively simple workaround that required one modification of the source in enum.py. Basically the line creating "PyModule_AddIntConstant" needs to detect if a C++ compiler is being used, and if so use "static_cast" on the enum value. It is also necessary to add the enum values in the pybindgen input code (i.e., "add_enum") using the tuple form (in order to provide a valid python name as well as the fully scoped C++ value), but that is already possible in the current interface.

I'm attaching the modified source for "enum.py" as well as the diff file to this issue. I'm welcome to better suggestions as to how this issue should be handled too! Clearly it would be nicer to properly support scoped enums as more than just integers from python, since this is the whole point of using scoped enums in C++. However, this workaround is a minimal change that at least allows them to be bound and used via the current enum support.
enum_source.zip

https://pypi.org/project/PyBindGen/ has 0.20.0 now, which does not support python 3 and is not supported by the latest ns-3-dev.

I need a new version to make https://gitlab.com/nsnam/ns-3-dev/merge_requests/88 work without building pybindgen from git.

Hello,

I recently needed to wrap triplets. Unfortunately, it looks like Pybindgen does not support either of:

• std::pair
• std::tuple
• std::array
  (I ended up using an std::vector)

I think that std::pair/std::tuple could be easily mapped to python tuples, and array to a python list. What are your thoughts on this ?

With my version of g++ (9.2.1) I'm getting the following warning a lot (about 200 times for my bindings):

src/plugins/python/pythonbind.cpp:7211: warning: cast between incompatible function types from ‘PyObject* (*)(PyTiledMapObject*)’ {aka ‘_object* (*)(PyTiledMapObject*)’} to ‘PyCFunction’ {aka ‘_object* (*)(_object*, _object*)’} [-Wcast-function-type]
 7211 |     {(char *) "type", (PyCFunction) _wrap_PyTiledMapObject_type, METH_NOARGS, "type()\n\n" },
      |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~

I'm compiling with -Wall -Wextra, but not sure what's necessary to trigger this warning.

Of course I don't know if there's potential problems here, but it would be nice to not need to suppress this warning.

I have a question regarding the failure of bindings compilation for some proposed changes to ns-3 NetDevice.

To reproduce:
git clone https://github.com/stavallo/ns-3-dev-git
cd ns-3-dev-git
git checkout -b queue-disc-exp remotes/origin/queue-disc-exp
./waf configure --enable-modules=network --with-pybindgen=/path/to/pybindgen
./waf build
./waf --apiscan=network
./waf build

fails with:

In file included from src/network/bindings/ns3module.cc:1:0:
src/network/bindings/ns3module.h: In member function ‘virtual bool PythonCallbackImpl1::operator()(ns3::Ptrns3::NetDevice, ns3::Ptr, short unsigned int, const ns3::Address&, const ns3::Address&, ns3::NetDevice::PacketType)’:
src/network/bindings/ns3module.h:4944:96: error: ‘PyNs3NetDevice__PythonHelper’ was not declared in this scope

Here is some context on this change. Until now, NetDevice was pure abstract, but the changes proposed in this class add some non-virtual methods and private data.

If I modify NetDevice with some trivial non-virtual methods and private data, pybindgen still works, so it has to do with something about the specific nature of the proposed changes and not generically about moving from pure abstract to abstract.

The compilation is failing in the definition of one of the callbacks:

bool
operator()(ns3::Ptr< ns3::NetDevice > arg1, ns3::Ptr< ns3::Packet const > arg2, unsigned short arg3, ns3::Address const & arg4, ns3::Address const & arg5, ns3::NetDevice::PacketType arg6)
{
    if (typeid(*(const_cast<ns3::NetDevice *> (ns3::PeekPointer (arg1)))).name() == typeid(PyNs3NetDevice__PythonHelper).name())
    {
        py_NetDevice = (PyNs3NetDevice*) (((PyNs3NetDevice__PythonHelper*) const_cast<ns3::NetDevice *> (ns3::PeekPointer (arg1)))->m_pyself);
        py_NetDevice->obj = const_cast<ns3::NetDevice *> (ns3::PeekPointer (arg1));
        Py_INCREF(py_NetDevice);
    } else {
        wrapper_lookup_iter = PyNs3ObjectBase_wrapper_registry.find((void *) const_cast<ns3::NetDevice *> (ns3::PeekPointer (arg1)));

where the PyNs3NetDevice__PythonHelper does not exist because this is an abstract class still.

In the previous version of the bindings, the if () branch does not exist, and the function moves directly to
wrapper_lookup_iter = PyNs3ObjectBase_wrapper_registry.find((void *) const_cast<ns3::NetDevice *> (ns3::PeekPointer (arg1)));

so what appears to be the problem is that something is triggering pybindgen to include this TypeId check, which is failing due to lack of the PyNs3NetDevice__PythonHelper class.

At this point I'm not clear whether it is a pybindgen issue or not.

Hello there I am trying to use this project to generate bindings to libsm64 the generate script I am using is https://pastebin.com/WtWWLZs5 and the setup.py script runs the my_module_gen function
when running I get a load of errors (here is the log file)
log.txt
mostly telling that delete is a type name and to prefix some new structs with struct and mising ;
any help would be apprecieated

pygccxml.parser.gccxml_configuration_t is used in pybindgen.gccxmlparser, but this is deprecated as of pygccxml-1.7, and it will be removed as of pygccxml-1.9. It should be replaced with pygccxml.parser.xml_generator_configuration_t (apparently renamed to reflect their now preferring CastXML). However, a more subtle arrangement may be necessary if pybindgen is to maintain compatibility with pygccxml-1.6 and earlier.

I encountered a problem upon scanning this newly added ns-3 method (position-allocator.h:101)

  void Add (const std::string filePath,
            double defaultZ = 0,
            char delimiter = ',');

pybindgen is complaining as follows:

==> ./mobility/bindings/ns3modulegen.log <==
ValueError: Error: optional parameter followed by a non-optional one ('delimiter
') (debug: self._parse_tuple_parameters=[('s#', ['&filePath', '&filePath_len'],
'filePath', False), ('d', ['&defaultZ'], 'defaultZ', True), ('c', ['&delimiter']
, 'delimiter', False)])

Is this a current pybindgen limitation that we should work around?

Full error log is below:

Traceback (most recent call last):
  File "bindings/python/ns3modulegen-modular.py", line 131, in <module>
    main(sys.argv)
  File "bindings/python/ns3modulegen-modular.py", line 127, in main
    root_module.generate(out)
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/module.py", line 928, in generate
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/module.py", line 767, in do_generate
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/cppclass.py", line 2009, in generate
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/cppclass.py", line 2401, in _generate_methods
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/utils.py", line 187, in call_with_error_handling
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/overloading.py", line 143, in generate
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/overloading.py", line 106, in _normalize_py_method_flags
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/utils.py", line 187, in call_with_error_handling
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/cppmethod.py", line 410, in get_py_method_def_flags
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/typehandlers/base.py", line 982, in get_py_method_def_flags
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/typehandlers/base.py", line 897, in generate_body 
  File "/path/to/build/lib/PyBindGen-0.21.0.post13+gac96636-py3.5.egg/pybindgen/typehandlers/base.py", line 318, in get_parameters
ValueError: Error: optional parameter followed by a non-optional one ('delimiter') (debug: self._parse_tuple_parameters=[('s#', ['&filePath', '&filePath_len'], 'filePath', False), ('d', ['&defaultZ'], 'defaultZ', True), ('c', ['&delimiter'], 'delimiter', False)])

On ns-3 (downstream) we are seeing build failures on Python 3.9 due to these deprecation warnings. I read through issue #37 suggesting needing to call PyEval_InitThreads only for Python <3.7, and perhaps similarly, these PyEval_ThreadsInitialized() methods should be enabled only for the older Python versions.

Just faced exactly the same issue and got a solution with the listed patch:
https://www.gitmemory.com/issue/gjcarneiro/pybindgen/19/510713636

Just want to report here, in case it was not by the original author.

In scanning ns-3 APIs with Python 3, everything nearly works but for a failure on one method in castxmlparser.py.

If I make this change to the file:

--- a/pybindgen/castxmlparser.py
+++ b/pybindgen/castxmlparser.py
@@ -1754,8 +1754,12 @@ pybindgen.settings.error_handler = ErrorHandler()
                 #print >> sys.stderr, "(lookup %r: %r)" % (name, class_wrapper)
                 return class_wrapper
 
-            if not type_traits_classes.is_convertible(cls, argument_types[0]):
-                return
+            try:
+                if not type_traits_classes.is_convertible(cls, argument_types[0]):
+                    return
+            except TypeError as e:
+                 print("Ignoring exception: %s %s %s" % (e, cls, argument_types[0]))
+                 return

and scan ns-3, I will obtain debugging prints on the following operators and classes:

Ignoring exception: '<' not supported between instances of 'NoneType' and 'NoneType' ns3::DoubleValue [class] ns3::DoubleValue [class]
Ignoring exception: '<' not supported between instances of 'NoneType' and 'NoneType' ns3::IntegerValue [class] ns3::IntegerValue [class]
...

i.e. all of the ns-3 'Value' classes throw an exception on this statement in Python 3, while they do not in Python 2.

We could work around by ignoring the exception such as above but am wondering about a more direct solution either in pybindgen/castxmlparser.py or in ns-3 custom bindings code. We are moving to only support Python 3 in ns-3 going forward so a backward-compatible solution isn't needed.

Steps to reproduce the failure:

1. create and enter a Python3 venv
2. clone bake and configure ns-3-allinone target with some substitute repositories and ensure that scanning is enabled
3. try to rescan

export PATH=$PATH:`pwd`/build/bin
export PYTHONPATH=$PYTHONPATH:`pwd`/build/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:`pwd`/build/lib
./bake.py configure -e ns-3-allinone
./bake.py download
# replace pybindgen directory with latest pybindgen git tip on master branch
# replace ns-3-dev with the 'python3' branch of https://gitlab.com/tomhenderson/ns-3-dev.git 
./bake.py build
cd source/ns-3-dev
./waf configure
./waf --apiscan=core