Error with OpenMP about qiskit-aer HOT 14 CLOSED

Seems like an issue with how the package is built. I think the MacOS packages must be rebuilt and redistributed with the flag
-DSTATIC_LINKING=False

Doing this locally fixed the issue for me. I am opening a PR to make the installation instructions more clear for other people who want to build locally on Mac.

from qiskit-aer.

Not really, this happens because we link with libomp, but Numpy which is our only dependency is usually linked against libiomp (Intel implementation). Both implementations are incompatible, so when the runtime linker load both there could be conflicts. Paul told me that this is something Conda distribution has already fixed in the latest version, so doing a: conda update will fix this issue.

from qiskit-aer.

I will try and confirm either way tonight

from qiskit-aer.

I had this problem and two fixes I found were to either installing numpy/scipy with pip instead of conda, or if you use conda to install the nomkl package (conda install nomkl).

from qiskit-aer.

conda update and conda install nomkl did not work.

from qiskit-aer.

I still can’t get this working with pip. I have tried going back to numbly 1.14 as well. I was able to build it following the source instructions and not see this crash.

from qiskit-aer.

I notice that qasm_simulator and unitary_simulator are OK; but, statevector_simulator occurs the OMP errror. I checked it with the following code.

from qiskit import execute, Aer, QuantumCircuit, QuantumRegister

q = QuantumRegister(1)
qc = QuantumCircuit(q)
qc.h(q[0])
#backend = Aer.get_backend('qasm_simulator')  # OK
#backend = Aer.get_backend('unitary_simulator')  # OK
backend = Aer.get_backend('statevector_simulator')  # NG
job_sim = execute(qc, backend)

from qiskit-aer.

Ok, so the proper way to fix this is by redistributing Aer Terra-Addon dynamically linked with libomp.dylib, and redistribute libomp.dylib along with the package (the license allow us to do so), so the user doesn't need to manually install it.
Setting up this in the build system will take me some time, so in the meantime, if someone is facing this very same error, rebuilding Aer locally will fix the issue (as the default build uses dynamic linking, and installs libomp via homebrew).

from qiskit-aer.

I do not understand why we cant just build using clang (llvm from homebrew) that includes the iomp5 lib? gcc is not a standard build process on osx. This would fix the mkl issue, and cause no problems otherwise because osx has no openmp, and openblas is not built with it by default. Is there something that I am missing?

from qiskit-aer.

Not sure how far this will go, but the workaround suggested in the OMP warning message may work if you don't want to do your own build and simply want to run from pip install: set KMP_DUPLICATE_LIB_OK=TRUE. It at least works for the dissociation_profile_of_molecule.py sample (I attached a trimmed version of that sample for repro).
qcware-test.zip

from qiskit-aer.

You can use lddtree (part of pax-utils) to see which libraries are being linked by the linker to figure out which are the two versions of omp being linked and remove one. I had this issue with HDF5 when installing PySCF on a server, and lddtree pointed me right at the two installations in different places. It turned out that the issue was that I had appended my anaconda directory to the beginning of my path in my .bash_rc when I first installed it (rather than the end, so linking defaults to the system libraries), so some libraries were using the system default hdf5, and anything new was using the one inside anaconda3/ because that came first in that path. Moving anaconda to the end of the path fixed the issue. That might not be it here, you may need to remove one installation entirely.

from qiskit-aer.

Seeing on mac the tool to use might be otool -L. lddtree was helpful on ubuntu.

~/code/qiskit-aer$ otool -L ./_skbuild/macosx-10.7-x86_64-3.6/cmake-build/qiskit/providers/aer/backends/wrappers/statevector_controller_wrapper.cpython-36m-darwin.so
./_skbuild/macosx-10.7-x86_64-3.6/cmake-build/qiskit/providers/aer/backends/wrappers/statevector_controller_wrapper.cpython-36m-darwin.so:
        /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5)

 ~/code/qiskit-aer$ otool -L ./_skbuild/macosx-10.7-x86_64-3.6/cmake-build/Release/qasm_simulator
./_skbuild/macosx-10.7-x86_64-3.6/cmake-build/Release/qasm_simulator:
        /usr/local/opt/libomp/lib/libomp.dylib (compatibility version 5.0.0, current version 5.0.0)
        /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
        /usr/local/opt/gcc/lib/gcc/8/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.25.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5)
        /usr/local/lib/gcc/8/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

from qiskit-aer.

Yeah, but I'm afraid that the problem is even nastier. Depending on the version of Numpy you are using, it will come linked with one or another library, and there's the possibility to use any other third-party extension that we are not aware of which could be linked with any other version of this library. Basically, we need to have some sort of dynamically check to see whether we need to load or not at runtime the library (the implementation of this last thing is a bit trickier because of the way compilers link with this specific library and the way python dynamically load extensions). I already have a proper fix but still need to polish some details.

from qiskit-aer.

So seems like I could finally came up with a fix for this problem. @chriseclectic is helping me to make sure that it works on every possible sceneario, as the fix it's far from being trivial.
I have reported a bug to LLVM here so we can remove all this machinery soon.

What's going on?

Long story short, OpenMP LLVM runtime (which is the base for other libraries), doesn't allow two copies of the same runtime loaded into the process, so when it's (lazy-) initializing it checks for other already initialized copies in the memory, if there's one, it complains and exits. This is specially problematic because of the fact that we are a Python extension too, and we don't have any control over what other entensions are being loaded by the Python interpreter, so there could be many other than Numpy that could potentially clash with us.

How did we solve it?

Even longer story shorter, I have basically given Aer the capability to detect if there's another OpenMP library already loaded in the process at runtime, and behave "accordingly" (there are many low-level tricky techniques being used to accomplish this, better see the code).

I have found many other Python extensions facing the very same problem, unfortunately their solutions in most of the cases are wrong, as setting the env variable KMP_DUPLICATE_LIB_OK is far from being a safe solution.

from qiskit-aer.