Build failed on ubuntu 15.10 about jocl HOT 29 CLOSED

CMake tends to have 15 ways to do something and I'm never sure which one is "correct" :)

I agree, this is a hassle. However, it's hard to tell who is to blame here. The goals of CMake are ambituous, and I'm not aware of a solution that comes closer to these goals. Nevertheless, googling and copy+pasting snippets from stackoverflow like "On Voodoo Linux 3.2.1.3-b2010 you have to enable the CMAKE_ADD_SOME_CHICKEN_BLOOD flag" always feels like blind guesswork.

However, I wonder why it did not set the fPIC flag in your case. The set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fPIC -ldl") approach is (as far as I understand) not a very elegant way of doing it, but it should be rather "brutal", in the sense that it should always set the flag, regardless of any CMake- or GCC version.

@oddeirik , and maybe also @blueberry (where it seemed to behave differently for some unknown reason) : Can you confirm that changing the part

elseif(CMAKE_HOST_UNIX)
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fPIC -ldl")
endif()

to

elseif(CMAKE_HOST_UNIX)
    set(CMAKE_POSITION_INDEPENDENT_CODE TRUE)
endif()

in https://github.com/gpu/JOCLCommon/blob/master/CMake_c_flag_overrides.cmake and in https://github.com/gpu/JOCLCommon/blob/master/CMake_cxx_flag_overrides.cmake works in both environments?

(I'd really like to have a solution that causes as few headaches for contributors as possible. Still, CMake is a beast, and it's hard to get the details right. I won't even dare to ask why this flag does not seem to be necessary in JCuda at all...)

from jocl.

When I del the CL_EXT_PREFIX__VERSION_1_2_DEPRECATED
then I got another error message:
[ 8%] Building CXX object JOCLCommon/CMakeFiles/JOCLCommon.dir/src/main/native/Logger.cpp.o
[ 16%] Building CXX object JOCLCommon/CMakeFiles/JOCLCommon.dir/src/main/native/CLJNIUtils.cpp.o
[ 25%] Building CXX object JOCLCommon/CMakeFiles/JOCLCommon.dir/src/main/native/JNIUtils.cpp.o
[ 33%] Building CXX object JOCLCommon/CMakeFiles/JOCLCommon.dir/src/main/native/PointerUtils.cpp.o
[ 41%] Building CXX object JOCLCommon/CMakeFiles/JOCLCommon.dir/src/main/native/ConversionsCL.cpp.o
[ 50%] Linking CXX static library libJOCLCommon.a
[ 50%] Built target JOCLCommon
[ 58%] Building CXX object CMakeFiles/JOCL_0_2_0-linux-x86_64.dir/src/main/native/JOCL.cpp.o
[ 66%] Building CXX object CMakeFiles/JOCL_0_2_0-linux-x86_64.dir/src/main/native/CLFunctions.cpp.o
[ 75%] Building CXX object CMakeFiles/JOCL_0_2_0-linux-x86_64.dir/src/main/native/FunctionPointerUtils.cpp.o
[ 83%] Building CXX object CMakeFiles/JOCL_0_2_0-linux-x86_64.dir/src/main/native/FunctionPointerUtils_Linux.cpp.o
[ 91%] Building CXX object CMakeFiles/JOCL_0_2_0-linux-x86_64.dir/src/main/native/FunctionPointerUtils_Win.cpp.o
[100%] Linking CXX shared library nativeLibraries/libJOCL_0_2_0-linux-x86_64.so
/usr/bin/ld: JOCLCommon/libJOCLCommon.a(CLJNIUtils.cpp.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
JOCLCommon/libJOCLCommon.a: error adding symbols: 错误的值
collect2: error: ld returned 1 exit status
CMakeFiles/JOCL_0_2_0-linux-x86_64.dir/build.make:199: recipe for target 'nativeLibraries/libJOCL_0_2_0-linux-x86_64.so' failed
make[2]: *** [nativeLibraries/libJOCL_0_2_0-linux-x86_64.so] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/JOCL_0_2_0-linux-x86_64.dir/all' failed
make[1]: *** [CMakeFiles/JOCL_0_2_0-linux-x86_64.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

from jocl.

Sorry, the update for the cl_platform.h was missing in the previous commit. This should be done now, in commit gpu/JOCLCommon@b8228d0 , including an update of the function pointer typedefs in 862f2ab

from jocl.

Regarding the second issue: Can you confirm that adding

set_target_properties(${PROJECT_NAME} PROPERTIES COMPILE_FLAGS -fPIC)

at the bottom of https://github.com/gpu/JOCLCommon/blob/master/CMakeLists.txt solves this issue? (I think it will have to be deactivated for windows, but would like to know whether it works on Linux with this flag)

from jocl.

linker@linker-pc:/gits/JOCL$ make
[ 50%] Built target JOCLCommon
[ 58%] Linking CXX shared library nativeLibraries/libJOCL_0_2_0-linux-x86_64.so
/usr/bin/ld: JOCLCommon/libJOCLCommon.a(CLJNIUtils.cpp.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
JOCLCommon/libJOCLCommon.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
make[2]: *** [nativeLibraries/libJOCL_0_2_0-linux-x86_64.so] Error 1
make[1]: *** [CMakeFiles/JOCL_0_2_0-linux-x86_64.dir/all] Error 2
make: *** [all] Error 2
linker@linker-pc:/gits/JOCL$

from jocl.

Added -fPIC to link flags, then it failed again.

from jocl.

JOCLCommon

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 489fc8b..a77c17e 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -27,7 +27,7 @@ include_directories(
${JNI_INCLUDE_DIRS}
)

-add_library(JOCLCommon
+add_library(JOCLCommon SHARED

from jocl.

gpu/JOCLCommon#2

from jocl.

Thanks, that actually looks far more resonable than manually fiddling with compiler flags! I'll try it on windows today (but am pretty sure that it will not cause any problems) and then pull it accordingly.

from jocl.

Sorry, this does not seem right. The JOCLCommon library is intended to be linked statically. But adding SHARED causes it to create a .DLL (or an .SO on Linux, respectively). From what I have head so far, adding fPIC should work (and it does seem to work in a similar setup).

So when you add

# Enable position independent code for 64bit Linux platforms
set_target_properties(JOCLCommon PROPERTIES COMPILE_FLAGS -fPIC)

as the last line of the JOCLCommon CMakeLists.txt, what exactly is the error message that the linker shows? Does it still ask for the fPIC flag then?!

from jocl.

Alternatively, something like

if (UNIX AND NOT WIN32)
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fPIC")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC")
endif(UNIX AND NOT WIN32)

should have the same (?) effect - not sure whether this one works on Linux, but worth a try.

from jocl.

According to #5 (comment) , and because there have been no further comments, I assume that this build problem was solved for @linkerlin by adding SHARED somewhere, but is no longer a general problem with the Linux build. If this is wrong, the issue may be reopened.

from jocl.

As if troubling you with JCuda wasn't enough, I'll bug you here as well! :)

I discovered this same issue when trying to build on Ubuntu 14.04. For some reason the -fPIC flag was not added properly when building JOCLCommon as a static library. I first worked around it by building JOCLCommon as a shared library, but I figured I'd try to solve it. Here's the change I did:

diff --git a/CMake_cxx_flag_overrides.cmake b/CMake_cxx_flag_overrides.cmake
index ac507b1..0e8cd59 100644
--- a/CMake_cxx_flag_overrides.cmake
+++ b/CMake_cxx_flag_overrides.cmake
@@ -9,5 +9,7 @@ if(MSVC)
 elseif(ANDROID)
     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fpermissive")
 elseif(CMAKE_HOST_UNIX)
-    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -ldl")
+    if(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64")
+        add_definitions(-fPIC)
+    endif()
 endif()

This adds the -fPIC flag when building on Linux 64-bit and the final library links just fine. I'm not sure whether this flag is also required for 32-bit hosts (as I don't have any available 32-bit Linux VMs at the moment) but in that case it's just a matter of removing the if block, of course.

Edit: Come to think of it, perhaps add_compile_options() is more correct for this flag, rather than add_definitions. CMake tends to have 15 ways to do something and I'm never sure which one is "correct" :)

Edit2: A less GCC-centric approach might be setting the CMAKE_POSITION_INDEPENDENT_CODE property to TRUE instead (see https://cmake.org/cmake/help/latest/variable/CMAKE_POSITION_INDEPENDENT_CODE.html).

from jocl.

@oddeirik Just for the reference, everything builds just fine as it is on 64bit arch linux.

from jocl.

Yeah, maybe it's down to CMake version or GCC? I'm on Ubuntu 14.04 with GCC 4.8 and CMake 3.2.2. I'm guessing Arch is slightly ahead in that department :)

We'll probably move to Ubuntu 16.04 soon-ish, but we're stuck on 14.04 for a few more months at least.

from jocl.

Not blaming CMake at all, it's probably the most sane C/C++ build system out there. It just feels overwhelming at times :)

I can confirm that the fix of using "set(CMAKE_POSITION_INDEPENDENT_CODE TRUE)" works on Ubuntu 14.04 x64, the final library compiles and links just fine.

As to why the compiler flags weren't set properly in the first place, I have no idea. Could be a bug in CMake version <=3.2.2 (the version I'm currently using), or it could be something completely different.

from jocl.

Sure, I didn't mean to blame it either. Let's instead blame the guy who once thought: "Hey, I'll add some strange dependency of the code to the memory location of where the code resides, which should not be relevant at all, but will drive people crazy with strange linker errors". (There's a reason of why I'm creating all these Java <3 bindings ;-))

However, it's hard to imagine that this is a regression in CMake, because the CMAKE_C_FLAGS are likely one of the few things that have been there since CMake 1.0.0. But I'll update the makefile accordingly (probably Monday), preferably if blueberry confirms that it works on a different system as well. At least, the CMAKE_POSITION_INDEPENDENT_CODE is likely the cleaner solution (if it works) than manually fiddling in a specific flag.

(BTW: The ldl wasn't really necessary either...?)

from jocl.

@gpu sorry, I won't be able to try it before wednesday

from jocl.

No problem, it's only a minor issue and not urgent - people seem to manage to compile it despite having to add a certain flag. It should be resolved, of course, but whether today or next week will not matter much.

from jocl.

Just a quick bump. Fix works here Xenial x86_64 👍

from jocl.

Thanks for this info @Lewiscowles1986 . Since there has been no further feedback, I'll consider this to be resolved. (It may be reopened if there's still something wrong).

from jocl.

Sorry, you've misunderstood me #3 (comment) is the fix I had to patch master branch with. I'm happy to raise a PR if you're busy or are open to receiving, but it is your contribution.

I thought I'd give an update on the open issue so that anyone downloading could use. Closing the issue makes that harder.

from jocl.

No, sorry, that was my fault - this issue had been open for so long, I should have re-read it more carefully. I'll integrate this fix ASAP then.

from jocl.

So hopefully this is fixed with the latest commit. (I still have to set up a Linux VM to better test the Linux builds locally...)

from jocl.

Want me to whip up and configure a vagrant for you?

from jocl.

@Lewiscowles1986 Uh - I've never worked with vagrant. Only heard a bit about it (and not only positive aspects - it sounded like it was something where it may take some time to become acquainted with...). My naive approach would have been to just set up a VirtualBox (and I think that even if there is a more sophisticated alternative, this may still be helpful for local debugging and tests).

So this may be a dumb question, but what would be the benefit of Vagrant here, considering that this is really about a single VM that is supposed to run locally on a single machine?

from jocl.

It's a scripted Virtualbox you (or anyone visiting the repo) can spin up. I use them to setup a set of dependencies, downloads the code and run it for me.

You can also delete a bit like a container, or cloud VPS; but it's slower because the content is bundled inside, it fetches dependencies and it's emulating a computer and OS rather than like a container just running apps isolated from the system.

Right now I'm having issues with JDK8 downloading. It's installing but 403'ing the JDK aspect (thanks Oracle). This is making cmake behave like a c#%*.

Which JDK are you working with?

from jocl.

I'm currently using the Oracle JDK, jdk1.8.0_131

The possible advantages of a VM are clear, and particularly its "scripted construction" (which I think is what vagrant mainly does). However, repeatedly setting up a whole VM just to trigger a build process has some issues, too...

(Maybe I'm just overly skeptical: In my day job we're using docker, and I find it somewhat ridiculous to see it pulling several GB, over the net, repeatedly, during each build. A whole VM certainly may involve even more traffic. Well, I guess I have to upgrade my ~1MBit connection soon anyhow...)

from jocl.

You've got it in one, vagrant is a scripted VM tool.

Also vagrant and docker should only pull several GB on rebuild. After that in vagrant a box can be exported, docker always needs an image to run, but you can also push those to dockerhub or a local registry.

You can also use plugins to vagrant like cachier to make sure that you are responsibly using rpm or deb repo's. Docker generally caches build steps so that it will only get new packages if they change or a prior step changes (or you tell it to build without cache).

The core thing I was aiming for is to make the build repeatable and it serves as not-bad documentation for users on how to setup dependencies rather than having in a README or combing issues. Even cooler, people can embed multiple scripts or one giant script to "provision" the machine.

This is basically also something that you personally may not run a lot 89 commits since 2010. So it can serve as a running log of what happened when, where as far as system-level deps go (a bit like how maven handles project deps)

from jocl.