error in example about binder HOT 18 CLOSED

please try to follow this instead:
https://github.com/RosettaCommons/binder/blob/master/examples/example_struct/make_bindings_via_bash.sh

maybe the documentation alone is not clear enough? but I think that file + documentation should help you figure out what is happening

from binder.

Hi @danpf

I have taken a second look at the test_struct example and made it working as it intended. The example demonstrated a simple usage of one class under the same directory, but in real world scenario, most c++ project are organise into nested subfolders, and various CMakeList.txt. It would be helpful to understand what is require to make that work.

It would be very helpful to see an example with few nested folders and header files in a separate directory.

For example, if I want to generate all the bindings under this folder https://github.com/supreme-core/rsocket-cpp-pybind/tree/master/rsocket and my binder binary was located at /opt/Binder/binder/build/llvm-4.0.0/build_4.0.0.linux.lap-computer.release/bin/binder

I would run something like:

$ mkdir rsocket-cpp-binder && cd rsocket-cpp-binder
$ git clone https://github.com/supreme-core/rsocket-cpp-pybind
$ cd rsocket-cpp-pybind/rsocket
$ grep -rh "#include" * | sort -u > all_header_includes.hpp
$ cd ..
$ mkdir gen_bindings
$ /opt/Binder/binder/build/llvm-4.0.0/build_4.0.0.linux.lap-computer.release/bin/binder
--root-module pyrsocket --prefix $PWD/gen_bindings/
--bind rsocket $PWD/rsocket/all_header_includes.hpp
-- -std=c++11 -DNDEBUG

But, it doesn't work. What is missing here.

from binder.

Hi @supreme-core, apologies for the delay.

one of the limitations of binder is that: in order to build bindings, you need to convert all of your # includes to angle bracket includes. This forces you to not only convert to using full paths like:

#include <transports/RSocketTransport.h>

I'm betting that's your problem, but without more logs that's the best explanation I can give you.

It would be very helpful to see an example with few nested folders and header files in a separate directory.

I realize this and I am working on one, however I don't have time currently to finish that side-project :(

from binder.

Hi,

First of all, thank you all for the effort and time on this beautiful project. I don't know if I'm in the good place for my question but by reading this thread I thought it would be okay... my apologies if not:

Context:

I'm presently working (trying) on binding a lot of cpp libraries (consisting of .cpp files and .h files). Since a little while, I've successfully compiled and use binder with your ''test_struct'' example.

The thing is : test_struct example consist of only a .hpp file. I didn't see an example with a .cpp and a .h file.

I try to bind with my .cpp files contained in a ''source folder'' and my .h files contained in a ''include folder'' . Since then, I did not succeed to indicate to binder how to do it.

Question :

Do I have to put all the content (implementation) of the .cpp file in a .h file to succeed? If not, what am I missing?

Say I have this header file :

#pragma once

#include <cstddef>
#include <vector>

class Wave
{
	public:
		Wave();
		Wave(float amplitude, float phase);
		Wave(float amplitude, float phase, int nbSample);
		Wave(std::vector<float>& samples);
		~Wave();

		float m_amplitude = 0;
		float m_phase = 0;
		int m_nbSamples = 0;

		std::vector<float> m_samples;

		bool createSinWave();
		bool createSinWave(float amplitude, float phase, int nbSample);
		bool createCosWave();
		bool createCosWave(float amplitude, float phase, int nbSample);
};

and its corresponding source file :

#include "Wave.h"

Wave::Wave() {
	m_amplitude = 0;
	m_phase = 0;
	m_nbSamples = 0;
	m_samples.clear();
};

Wave::Wave(float amplitude, float phase) {
	m_amplitude = amplitude;
	m_phase = phase;
};

Wave::Wave(float amplitude, float phase, int nbSample) {
	m_amplitude = amplitude;
	m_phase = phase;
	m_nbSamples = nbSample;
};

Wave::Wave(std::vector<float>& samples) {
	m_samples = samples;
	m_nbSamples = samples.size();
};

Wave::~Wave() {
};

bool Wave::createSinWave() {
	if (m_nbSamples > 0 && m_phase >= 0 && m_amplitude >= 0) {
		for (int nthSample = 0; nthSample < m_nbSamples; nthSample++)
			this->m_samples.push_back( m_amplitude * cos((double)nthSample) + m_phase);
		return true;
	}
	return false;
};

bool Wave::createSinWave(float amplitude, float phase, int nbSample) {
	m_amplitude = amplitude;
	m_phase = phase;
	m_nbSamples = nbSample;

	if (m_nbSamples > 0 && m_phase >= 0 && m_amplitude >= 0) {
		for (int nthSample = 0; nthSample < m_nbSamples; nthSample++)
			this->m_samples.push_back(m_amplitude * sin((double)nthSample) + m_phase);
		return true;
	}
	return false;
};

bool Wave::createCosWave() {
	if (m_nbSamples > 0 && m_phase >= 0 && m_amplitude >= 0) {
		for (int nthSample = 0; nthSample < m_nbSamples; nthSample++)
			this->m_samples.push_back(m_amplitude * cos((double)nthSample) + m_phase);
		return true;
	}
	return false;
};

bool Wave::createCosWave(float amplitude, float phase, int nbSample) {
	m_amplitude = amplitude;
	m_phase = phase;
	m_nbSamples = nbSample;

	if (m_nbSamples > 0 && m_phase >= 0 && m_amplitude >= 0) {
		for (int nthSample = 0; nthSample < m_nbSamples; nthSample++)
			this->m_samples.push_back(m_amplitude * cos((double)nthSample) + m_phase);
		return true;
	}
	return false;
};

I know the title of this thread is "error in example" ... I don't have problem with the example, but my question (which I would prefer to send an message/email to you guys instead of spamming this thread) is about an example where we would have a this wave.cpp and wave.h bind to Python with binder and so on for other .cpp and .h files.

Thank you for your time

from binder.

@MrParris to create Python bindings for your C++ code two steps is needed:

1. Use Binder to generate bindings source code.
2. Compile generated code to produce shared library that could be imported in Python.

To do [1] Binder will need list of all header files from your C++ project. During this phase Binder will parse header files and collect information about your classes, functions etc and use this info to generate bindings source code. If your project have typical structure when each class/enum/function declared in .h file and defined in .cpp file then for phase 1 only header files would be needed.

During phase [2] you will need to build compile and link generated code and compile and link your library code as well. This is where you will need to use .cpp files from your original project.

Hope this helps,

from binder.

Thanks a lot for your quick response!
I think I see clearer now...

Sorry, I didn't give enough information about my situation:

You see, my goal is to create many .pyd files (windows python .dll) for many libraries I need to use in Python with Pybind11. I was hoping to generate binded .cpp files only (doing only the phase [1] you mentioned) and skipping the part where we create .so files (equivalent to .pyd files but for Linux). Even though it would compile the first time in Visual Studio 2019 (with proper configurations to create .pyd files), I realized the cpp file generated in phase [1] would lack a lot of information on how to expose the behavior of the declared components/functions in the header file.

Now if I understand well, only in phase [2] by using g++, source files from my original project will be used? I didn't understand that because in your example your not using any .cpp files other than binder's cpp file (EDIT : which is weird, why using binder .cpp files [?] when the compiled binder executable has already done its binding job. Precisely, how come binder source files are used when their only use is for the compilation of the binder executable itself and "test_struct" is an another implementation not linked to binder...?).
i.e. from your documentation :

pybase=`which python3`
g++ \
  -O3 \
  -I${my_python_include_directory} -I${pybind11_include_directory} -I${my_project_directory} \
  -I${binder_source_directory} -shared  \
  -std=c++11  -c ${bindings_code_to_build_object_file_from}  \
  -o ${output_object_file_name} -fPIC

I'm a beginner with g++ and gcc but according to what you're saying, I would need to also add an argument pointing a source folder or source files so according to the ''skeleton'' call of g++ just above you'd propose in your documentation : my g++ call, if it's possible, would be something like

pybase=`which python3`
g++ \
  -O3 \
  -I${my_python_include_directory} -I${pybind11_include_directory} -I${my_project_directory} \
  -I${binder_source_directory} -shared  \
  -std=c++11  -c ${bindings_code_to_build_object_file_from}  \
   -[PROPER OPTION ARGUMENT] ${MY_CODE_SOURCE_PATH}  \
  -o ${output_object_file_name} -fPIC

?

I'll look more into the g++ doc

from binder.

Let me try to clarify: Say we want to produce Python bindings for MyLibAAA C++ library so we can use it in Python. in phase [2] we build generated bindings to produce shared library (.so/.pyd file) that could be imported from Python. To do this we need to: (a) compile generated code from phase [1] and (b) link this code to your original C++ library. Linking could be done as either directly compiling code from MyLibAAA .cpp files and used resulted .o object files to produce final .so/.pyd file or you just compile .cpp files produced by Binder in step [1] and link them against shared library of MyLibAAA lib. I recommend first way since it allow you to distribute results as single file.

from binder.

One way that might help you learn is to actually use cmake first and then use that to fill in the blanks.

in the build directory ninja -v will actually spit out the g++ commands that are used.

say you have a new file called
fresh.hpp

#ifndef ffresh_hpp
#define ffresh_hpp

#include <string>

namespace testers {

struct ffedd {
	int an_int;
	std::string a_string;

	ffedd();
};

}

#endif

fresh.cpp

#include <test_struct/ffresh.hpp>

namespace testers {
ffedd::ffedd() {
	an_int = 355;
	a_string = "ffedd";
}
}

notice the lines that start with +

project(test_struct)
include_directories(/home/danpf/git/binder/examples/example_struct/../../source)
include_directories(/home/danpf/git/binder/examples/example_struct/include)
include_directories(/home/danpf/git/binder/examples/example_struct/include)
include_directories(/home/danpf/git/binder/examples/example_struct/../../build/pybind11/include)
include_directories(/home/danpf/.local/share/pyenv/versions/3.7.0/include/python3.7m)
set_property(GLOBAL PROPERTY POSITION_INDEPENDENT_CODE ON)
add_definitions(-DNDEBUG)
+ add_library(fedd SHARED ../include/test_struct/ffresh.cpp) # muse use path relative to build dir (can use STATIC or SHARED)
add_library(test_struct SHARED
	test_struct.cpp
	test_struct/test_struct.cpp
)
+ target_link_libraries(test_struct fedd)  # Link your new library to your final library.
set_target_properties(test_struct PROPERTIES PREFIX "")
set_target_properties(test_struct PROPERTIES SUFFIX ".so")

then you can build with

rm -rf CMakeCache.txt test_struct.so CMakeFiles && cmake -G Ninja && ninja -v && python -c "import test_struct"

ninja -v can/will help you learn what g++ is doing under the hood.

from binder.

I'm also having this issue. If I try the example_struct example with the header file split into a header/implementation pair then it doesn't work.
test_struct.hpp

#ifndef TEST_STRUCT_H
#define TEST_STRUCT_H

#include <string>
#include <vector>

namespace testers {
struct test_my_struct {
	int an_int;
	std::string a_string;
	std::vector<int> a_vector;
	float a_float;

    void increment_int();
    void add_float();
    void append_vec();
};
}

#endif

test_struct.cpp

#include <test_struct/test_struct.hpp>

namespace testers {
testers::test_my_struct::test_my_struct() 
{
    an_int = 27;
    a_string = "TEST_STRING";
    a_vector = std::vector<int>{1,2,3,4,5};
    a_float = 88.88;
}

void testers::test_my_struct::increment_int()
{
    ++an_int;
}

void testers::test_my_struct::add_float()
{
    a_float += 22.22;
}

void testers::test_my_struct::append_vec()
{
    a_vector.push_back(a_vector.back()+1);
}
}

These are the generated binding files:

test_struct.cpp

#include <map>
#include <memory>
#include <stdexcept>
#include <functional>
#include <string>

#include <pybind11/pybind11.h>

typedef std::function< pybind11::module & (std::string const &) > ModuleGetter;

void bind_test_struct_test_struct(std::function< pybind11::module &(std::string const &namespace_) > &M);


PYBIND11_MODULE(test_struct, root_module) {
	root_module.doc() = "test_struct module";

	std::map <std::string, pybind11::module> modules;
	ModuleGetter M = [&](std::string const &namespace_) -> pybind11::module & {
		auto it = modules.find(namespace_);
		if( it == modules.end() ) throw std::runtime_error("Attempt to access pybind11::module for namespace " + namespace_ + " before it was created!!!");
		return it->second;
	};

	modules[""] = root_module;

	std::vector< std::pair<std::string, std::string> > sub_modules {
		{"", "testers"},
	};
	for(auto &p : sub_modules ) modules[p.first.size() ? p.first+"::"+p.second : p.second] = modules[p.first].def_submodule(p.second.c_str(), ("Bindings for " + p.first + "::" + p.second + " namespace").c_str() );

	//pybind11::class_<std::shared_ptr<void>>(M(""), "_encapsulated_data_");

	bind_test_struct_test_struct(M);

}

test_struct/test_struct.cpp

#include <sstream> // __str__
#include <test_struct/test_struct.hpp>

#include <pybind11/pybind11.h>
#include <functional>
#include <string>

#ifndef BINDER_PYBIND11_TYPE_CASTER
	#define BINDER_PYBIND11_TYPE_CASTER
	PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>)
	PYBIND11_DECLARE_HOLDER_TYPE(T, T*)
	PYBIND11_MAKE_OPAQUE(std::shared_ptr<void>)
#endif

void bind_test_struct_test_struct(std::function< pybind11::module &(std::string const &namespace_) > &M)
{
	{ // testers::test_my_struct file:test_struct/test_struct.hpp line:10
		pybind11::class_<testers::test_my_struct, std::shared_ptr<testers::test_my_struct>> cl(M("testers"), "test_my_struct", "");
		cl.def( pybind11::init( [](){ return new testers::test_my_struct(); } ) );
		cl.def( pybind11::init( [](testers::test_my_struct const &o){ return new testers::test_my_struct(o); } ) );
		cl.def_readwrite("an_int", &testers::test_my_struct::an_int);
		cl.def_readwrite("a_string", &testers::test_my_struct::a_string);
		cl.def_readwrite("a_vector", &testers::test_my_struct::a_vector);
		cl.def_readwrite("a_float", &testers::test_my_struct::a_float);
		cl.def("increment_int", (void (testers::test_my_struct::*)()) &testers::test_my_struct::increment_int, "C++: testers::test_my_struct::increment_int() --> void");
		cl.def("add_float", (void (testers::test_my_struct::*)()) &testers::test_my_struct::add_float, "C++: testers::test_my_struct::add_float() --> void");
		cl.def("append_vec", (void (testers::test_my_struct::*)()) &testers::test_my_struct::append_vec, "C++: testers::test_my_struct::append_vec() --> void");
	}
}

from binder.

@cain986 could you please elaborate of what exactly have not worked? It looks like bindings was generated etc.

from binder.

When I build it, I get the following error (it's like the make_bindings_via_bash.sh example):

$ g++ -O3 -I$(pwd)/../include -I/usr/include/python3.6m $(python3 -m pybind11 --includes) -shared -std=c++11 -c test_struct.cpp -o test_struct.o -fPIC
$ g++ -O3 -I$(pwd)/../include -I/usr/include/python3.6m $(python3 -m pybind11 --includes) -shared -std=c++11 -c test_struct/test_struct.cpp -o test_struct/test_struct.o -fPIC
$ lstest_struct  test_struct.cpp  test_struct.modules  test_struct.o  test_struct.sources
$ file test_struct
test_struct: directory
$ file test_struct.o 
test_struct.o: ELF 64-bit LSB relocatable, x86-64, version 1 (GNU/Linux), not stripped
$ g++ -shared test_struct.o test_struct/test_struct.o -o test_struct.so
$ ls
test_struct  test_struct.cpp  test_struct.modules  test_struct.o  test_struct.so  test_struct.sources
$ file test_struct.so
test_struct.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=d07e457aea9960f1cd2022e1cd5e6db437f44ed1, not stripped
$ python3 -c "import test_struct"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: /data/projects/binder/examples/example_struct/bash_bindings/test_struct.so: undefined symbol: _ZN7testers14test_my_struct13increment_intEv

from binder.

looks like something with the linking, maybe try to change order to g++ -shared test_struct/test_struct.o test_struct.o -o test_struct.so. And i would also recommend to change names (so you do not have multiple test_struct.o which might get confusing)

from binder.

Same error. I can also do this:

$ nm test_struct.o | grep _ZN7testers14test_my_struct13increment_intEv
$ nm test_struct/test_struct.o | grep _ZN7testers14test_my_struct13increment_intEv
                 U _ZN7testers14test_my_struct13increment_intEv
$ nm test_struct.so | grep _ZN7testers14test_my_struct13increment_intEv
                 U _ZN7testers14test_my_struct13increment_intEv

This is why I was wondering about the bindings, because the symbols don't seem to be getting defined during compilation

from binder.

hm... maybe check that test_struct.o actually does contain this function? Like create a minimal main() that allocate test_my_struct and call increment_int? - it should not be really any magic here...

from binder.

main.cpp

#include <test_struct/test_struct.hpp>
#include <iostream>

using namespace std;

int main(int argc, char* argv[])
{
    testers::test_my_struct test;
    cout << "Calling increment_int" << endl;
    test.increment_int();
    cout << "test.an_int: " << test.an_int << endl;
    return 0;
}

$ g++ -I$(pwd)/../include main.cpp -o main -L$(pwd) -ltest_struct
/tmp/user/1000/ccGsNXbj.o: In function `main':
main.cpp:(.text+0x5b): undefined reference to `testers::test_my_struct::increment_int()'
collect2: error: ld returned 1 exit status

from binder.

i meant different: g++ -I$(pwd)/../include main.cpp -o main -L$(pwd) test_struct.o

from binder.

Lots of linker problems (file attached)
binder-example-linker-error-output.txt

from binder.

well, looks like wrong file is linked. I would recommend to rename files so it is clear which one generate with binder and which one was compiled from source and double check all build/links command line.

Closing this for now since error is not related to Binder.

from binder.