Support for CentOS about fastdup HOT 23 CLOSED

Hi @angelogrosst did you try to
python3.8 -m pip install -U pip
before installing?
So far we tested on the following Linux types: Debian 10 and Ubutnu 18.04 and Ubuntu 20.04.

from fastdup.

If that does not work send us the output of the command python3.8 -m pip debug --verbose

from fastdup.

I did some testing and python3.8 for centos 7 supports cp38-cp38-manylinux_2_17_x86_64 which is old binaries, fastdup currently support cp38-cp38-manylinux_2_27_x86_64 for Ubutnu 18.04 and cp38-cp38-manylinux_2_31_x86_64 for ubuntu 20.04. If you have a specific use case you need support for another platform please join our slack channel to discusss: https://join.slack.com/t/visualdatabase/shared_invite/zt-19jaydbjn-lNDEDkgvSI1QwbTXSY6dlA

from fastdup.

Hi Danny, thanks for the quick reply.

Did I get it right, that it should be possible to install fastdup using an old binary? I tried to install it via
python3.8 -m pip install fastdup-0.33-cp38-cp38-linux_x86_64.whl --force with the .whl file downloaded from https://github.com/visualdatabase/fastdup/releases/download/v0.33/fastdup-0.33-cp38-cp38-linux_x86_64.whl

However when trying to import fastdup the following error message appears:
OSError: libopencv_imgcodecs.so.4.2: cannot open shared object file: No such file or directory

Am I on the right track? What would you suggest me to do?

from fastdup.

hi @angelogrosst sorry for being unclear. I was not talking about old fastdup but talking about old libc version on Centos 7. Your path of download the binary is interesting we never tried it out. The error you see is that because opencv is not installed, try to install opencv 4.2 on your system.
Which centos are you using?

from fastdup.

I tried to install opencv 4.2 it did not work as glibc has old version. I will take a look but it would be easier to switch to a newer OS..

from fastdup.

Hi @dbickson, no problem.
I am using CentOS 7 with kernel 3.10.0-1160.15.2.el7.x86_64.
Unfortunately, I won't be able to switch to a newer OS.

If you find out anything that could help, please let me know.
Thanks for your support!

from fastdup.

HI @angelogrosst I am working on a fix, it may be ready today, I will keep you posted.

from fastdup.

Hi @angelogrosst we have just released a centos 7.9 version. It is in our release page:
https://github.com/visualdatabase/fastdup/releases/tag/v0.118

It can not be installed from pip. you have to download and install from local file. Let us know if it works for you.

from fastdup.

Hi @dbickson,

thank you again for the quick response.

lsb_release -r outputs Release: 7.8.2003

Can you maybe point out how exactly you install fastdup if not from pip?
I downloaded the wheel file https://github.com/visualdatabase/fastdup/releases/download/v0.118/fastdup-0.118-cp38-cp38-linux_x86_64.whl and ran python3.8 -m pip install fastdup-0.118-cp38-cp38-linux_x86_64.whl.

It is downloading and installing opencv_python-4.6.0.66-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

When starting python and running
import fastdup

I obtain the following output
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/data2/angelo.gross/venv/lib/python3.8/site-packages/fastdup/__init__.py", line 41, in <module> dll = CDLL(so_file) File "/usr/local/lib/python3.8/ctypes/__init__.py", line 373, in __init__ self._handle = _dlopen(self._name, mode) OSError: libopencv_imgcodecs.so.4.2: cannot open shared object file: No such file or directory

from fastdup.

Hi @angelogrosst
I found the issue! my bad. Sending a fixed version now

from fastdup.

Okay, although you edited your comment, here is the content of the fastdup site-package folder:

.
├── coco.py
├── galleries.py
├── html_writer.py
├── image.py
├── __init__.py
├── lib
│   └── libboost_program_options.so.1.53.0
├── libfastdup_shared.so
├── __pycache__
│   ├── coco.cpython-38.pyc
│   ├── galleries.cpython-38.pyc
│   ├── html_writer.cpython-38.pyc
│   ├── image.cpython-38.pyc
│   ├── __init__.cpython-38.pyc
│   ├── tensorboard_projector.cpython-38.pyc
│   └── version_check.cpython-38.pyc
├── tensorboard_projector.py
├── UndisclosedFastdupModel.onnx
└── version_check.py

from fastdup.

Can you please try this one?
https://github.com/visualdatabase/fastdup/releases/tag/v0.119

from fastdup.

I did, it gives me the same error message. Thank you for your effort. You can close the issue for now as I will be on vacations until the 8th of August. Then I will check again if I can make it work.

from fastdup.

Apologies this is strange as I look inside the wheel I see those libraries. Can you check in your wheel using unzip -l *.whl and also run find again on the folder under site-packages.

from fastdup.

A workaround would be to install opencv 4.2 on your centos station, which is given by the following command:

 git clone https://github.com/opencv/opencv.git
    cd opencv
    git checkout 4.2.0
    sudo yum -y install python-devel numpy python34-numpy
    sudo yum -y install gtk2-devel
    sudo yum install -y libpng-devel
    sudo yum install -y jasper-devel
    sudo yum install -y openexr-devel
    sudo yum install -y libwebp-devel
    sudo yum -y install libjpeg-turbo-devel
    sudo yum install -y freeglut-devel mesa-libGL mesa-libGL-devel
    sudo yum -y install libtiff-devel
    sudo yum -y install libdc1394-devel
    sudo yum -y install tbb-devel eigen3-devel
    sudo yum -y install boost boost-thread boost-devel
    sudo yum -y install libv4l-devel
    sudo yum -y install gstreamer-plugins-base-devel

    mkdir build
    cd build
    pip3.8 install cmake
    sudo cp ~/.local/bin/cmake /usr/bin/cmake
    cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D OPENCV_GENERATE_PKGCONFIG=ON -D WITH_JPEG=ON -D BUILD_JPEG=ON -D WITH_PNG=ON -D BUILD_PNG=ON -D WITH_TIFF=ON -D BUILD_TIFF=ON -D WITH_1394=OFF -D HAVE_1394=OFF -D WITH_WEBP=OFF -D BUILD_EXAMPLES=OFF -D INSTALL_PYTHON_EXAMPLES=OFF -D INSTALL_C_EXAMPLES=OFF -D PYTHON_EXECUTABLE=$(which python2) -D BUILD_opencv_python2=OFF -D BUILD_SHARED_LIBS=ON -D WITH_OPENEXR=OFF -D PYTHON3_EXECUTABLE=$(which python3.8) -D PYTHON3_INCLUDE_DIR=$(python3.8 -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") -D PYTHON3_PACKAGES_PATH=$(python3.8 -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")  ..
 make -j4
 sudo make install

from fastdup.

Anyway have a nice vacation we will work with you until it is 100% solved!

from fastdup.

Hi @angelogrosst are you back from your vacation? Let me know if you like to continue testing on centos 07, on my side it is working, the error you got last time seems that opencv is not installed, the code should have packaged it but I also gave you directions to try and install it manually.

from fastdup.

Yes, thank you. I will have a look at it tomorrow.

from fastdup.

Alright, so I tried installing opencv manually, as you suggested here

Apparently, the opencv related libraries are loaded now, but there is a new error coming up:

import fastdup
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data2/angelo.gross/venv/lib/python3.8/site-packages/fastdup/__init__.py", line 41, in <module>
    dll = CDLL(so_file)
  File "/usr/local/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libunity_shared.so: cannot open shared object file: No such file or directory

Have you got any idea on that?

from fastdup.

HI @angelogrosst Strange error. Can you please share the output of find /data2/angelo.gross/sst-2956-assess-fastdup/venv/lib/python3.8/site-packages/fastdup/ and also the output of unzip -l fastdup*.whl

Also try to export LD_LIBRARY_PATH=/data2/angelo.gross/venv/lib/python3.8/site-packages/fastdup/lib and then rerun python again from the same terminal.

from fastdup.

Hi @dbickson,

it is now working fine using the export LD_LIBRARY_PATH=/data2/angelo.gross/venv/lib/python3.8/site-packages/fastdup/lib statement you were given. The shared libraries were simply not found...

Thank you so much for your support!

Will there also be a Python 3.10 release someday?

from fastdup.

Hi @angelogrosst great to hear it is working, I have no clue why the packaging failed to find the library path as it should. If you agree I will package again the latest for you to try again since I want to submit a fix on our side.
Do you need python3.10 for centos 7 or for some other system?

from fastdup.