luigifreda / pyslam Goto Github PK

If you get this error using opencv 4.x as found here:

pyslam/thirdparty/orbslam2_features/ORBextractor.h:26:10: fatal error: opencv/cv.h: No such file or directory
   26 | #include <opencv/cv.h>

just go to the file ORBextractor.h and comment out the include using //#include <opencv.h>

If you run into this error:
Building CXX object python/CMakeFiles/g2o.dir/g2o.cpp.o
........
make[2]: *** [python/CMakeFiles/g2o.dir/build.make:63：python/CMakeFiles/g2o.dir/g2o.cpp.o] error 1
make[1]: *** [CMakeFiles/Makefile2:1345：python/CMakeFiles/g2o.dir/all] error 2
make: *** [Makefile:130：all] error 2

The fix is found here by doing this:

        .def("x", (double (Eigen::Quaterniond::*) () const) &Eigen::Quaterniond::x)
        .def("y", (double (Eigen::Quaterniond::*) () const) &Eigen::Quaterniond::y)
        .def("z", (double (Eigen::Quaterniond::*) () const) &Eigen::Quaterniond::z)
        .def("w", (double (Eigen::Quaterniond::*) () const) &Eigen::Quaterniond::w)

in g2opy/python/core/eigen_types.h to:

        .def("x", [](const Eigen::Quaterniond& q) { return q.x(); })
        .def("y", [](const Eigen::Quaterniond& q) { return q.y(); })
        .def("z", [](const Eigen::Quaterniond& q) { return q.z(); })
        .def("w", [](const Eigen::Quaterniond& q) { return q.w(); })

Also you may have issues installing opencv-contrib so you also need to install some stuff via pip manually:
pip3 install scipy opencv-contrib-python termcolor ordered_set matplotlib pygame pyopengl

This got it to build and run except for this error:

(pyslam) ~/pyslam$ python3 -O main_slam.py
 WARNING: cannot import SuperPointFeature2D from feature_superpoint, check the file TROUBLESHOOTING.md
 WARNING: cannot import TfeatFeature2D from feature_tfeat, check the file TROUBLESHOOTING.md
 WARNING: cannot import HardnetFeature2D from feature_hardnet, check the file TROUBLESHOOTING.md
 WARNING: cannot import GeodescFeature2D from feature_geodesc, check the file TROUBLESHOOTING.md
 WARNING: cannot import SosnetFeature2D from feature_sosnet, check the file TROUBLESHOOTING.md
 WARNING: cannot import L2NetFeature2D from feature_l2net, check the file TROUBLESHOOTING.md
 WARNING: cannot import LogpolarFeature2D from feature_logpolar, check the file TROUBLESHOOTING.md
 WARNING: cannot import D2NetFeature2D from feature_d2net, check the file TROUBLESHOOTING.md
 WARNING: cannot import DelfFeature2D from feature_delf, check the file TROUBLESHOOTING.md
 WARNING: cannot import ContextDescFeature2D from feature_contextdesc, check the file TROUBLESHOOTING.md
 WARNING: cannot import LfNetFeature2D from feature_lfnet, check the file TROUBLESHOOTING.md
 WARNING: cannot import R2d2Feature2D from feature_r2d2, check the file TROUBLESHOOTING.md
 WARNING: cannot import KeyNetDescFeature2D from feature_keynet, check the file TROUBLESHOOTING.md
importing pygame
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
initialising pygame
Processing Video Input
num frames:  1206
fps:  29.836205991321275
tracker_config:  {'num_features': 2000, 'num_levels': 8, 'scale_factor': 1.2, 'detector_type': <FeatureDetectorTypes.ORB2: 7>, 'descriptor_type': <FeatureDescriptorTypes.ORB2: 5>, 'match_ratio_test': 0.7, 'tracker_type': <FeatureTrackerTypes.DES_BF: 1>}
num_levels:  8
using opencv  4.4.0
Using Orbslam2Feature2D
 using same detector and descriptor object:  ORB2
num_levels:  8
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/user/pyslam/viewer3D.py", line 84, in viewer_thread
    self.viewer_init(kViewportWidth, kViewportHeight)
  File "/home/user/pyslam/viewer3D.py", line 92, in viewer_init
    pangolin.CreateWindowAndBind('Map Viewer', w, h)
AttributeError: module 'pangolin' has no attribute 'CreateWindowAndBind'

looks like the pangolin didn't build:

make[2]: *** [src/CMakeFiles/_pangolin.dir/build.make:999: src/CMakeFiles/_pangolin.dir/video/drivers/ffmpeg.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:516: src/CMakeFiles/_pangolin.dir/all] Error 2
make: *** [Makefile:152: all] Error 2

Recently I tried to install pyslam on Linux (Python 3.8) with ./install_all.sh.
Unfortunately, it always returns the error :

ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==1.14.0
ERROR: No matching distribution found for tensorflow-gpu==1.14.0

I did install tensorflow and also tensorflow-gpu, but with a new version. When I try to install

pip3 install tensorflow-gpu==1.14

it returns the same error:

ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==1.14
ERROR: No matching distribution found for tensorflow-gpu==1.14

What can I do? It seems, that this exact version of tensorflow-gpu is required, while at the
same time, it cannot be installed.
For any advice, I would be very happy!
Thank you in advance.

I ran main_slam.py for a TUM dataset, I noticed that depth information is not used. I'm using this to estimate extrinsic camera parameters, is there a way to use depth images to increase the accuracy?

Nice work! However I have met some prblems and thanks for helping!

Main problem:
python -O main_vo.py
WARNING: cannot import ORBextractor from orbslam2_features, check the file TROUBLESHOOTING.md
WARNING: cannot import pyslam_utils
WARNING: cannot import Orbslam2Feature2D from feature_orbslam2, check the file TROUBLESHOOTING.md
WARNING: cannot import KeyNetDescFeature2D from feature_keynet, check the file TROUBLESHOOTING.md

I use conda to create pyslam env, anything is fine except that I can not use orbslam2_features.
In pyslam env, I use python3.6, but in thildparty/orbslam2_features/CmakLists, the result of find_package(PythonLibs) is the one in system(python3.5), and the lib is
orbslam2_features.cpython-35m-x86_64-linux-gnu.so.
Even I SET(PYTHON_INCLUDE_DIRS anaconda3/envs/pyslam/include/python3.6m), or set PYTHONPATH, the lib is
orbslam2_features.cpython-35m-x86_64-linux-gnu.so(not the 36m one), and when I do python -O main_vo.py, I still face the problem that I can not use ORBextractor. Can you help me to solve this?

it seems that the url of lfnet is no longer valid, how can i find a pretrained model of lfnet

Hello,

Thank you for this useful repo.
How can I run the vo and slam on videos without ground truth?

Thanks

Many people asked me how to install and test pyslam under Windows.
I would suggest going with Windows WSL2 and install ubuntu under it.
Didn't test it yet but I found this cool install procedure that should provide a fast ubuntu GUI under Windows
https://www.youtube.com/watch?v=IL7Jd9rjgrM

WIP ...

Awesome project! How difficult would it be to bring pyslam into blender?

I've built the environment by running the script install_all.sh and successfully run main_vo.py.
But got TypeError: 'NoneType' object is not callable when running main_slam.py.
Need Help! Please check the error below:

 WARNING: cannot import ORBextractor from orbslam2_features, check the file TROUBLESHOOTING.md
 WARNING: cannot import pyslam_utils
 WARNING: cannot import Orbslam2Feature2D from feature_orbslam2, check the file TROUBLESHOOTING.md
 WARNING: cannot import DiskFeature2D from feature_disk, check the file TROUBLESHOOTING.md
importing pygame
pygame 2.0.1 (SDL 2.0.14, Python 3.6.13)
Hello from the pygame community. https://www.pygame.org/contribute.html
initialising pygame
Processing Video Input
num frames:  1101
fps:  10.0
tracker_config:  {'num_features': 2000, 'num_levels': 8, 'scale_factor': 1.2, 'detector_type': <FeatureDetectorTypes.ORB2: 7>, 'descriptor_type': <FeatureDescriptorTypes.ORB2: 5>, 'match_ratio_test': 0.7, 'tracker_type': <FeatureTrackerTypes.DES_BF: 1>}
num_levels:  8
using opencv  4.5.3
Traceback (most recent call last):
  File "main_slam.py", line 81, in <module>
    feature_tracker = feature_tracker_factory(**tracker_config)
  File "/media/vivi/E/Documents/Kangyun/Projects/pyslam/feature_tracker.py", line 66, in feature_tracker_factory
    tracker_type = tracker_type)
  File "/media/vivi/E/Documents/Kangyun/Projects/pyslam/feature_tracker.py", line 201, in __init__
    descriptor_type=descriptor_type)                     
  File "/media/vivi/E/Documents/Kangyun/Projects/pyslam/feature_manager.py", line 97, in feature_manager_factory
    return FeatureManager(num_features, num_levels, scale_factor, detector_type, descriptor_type)
  File "/media/vivi/E/Documents/Kangyun/Projects/pyslam/feature_manager.py", line 291, in __init__
    self._feature_detector = Orbslam2Feature2D(self.num_features, self.scale_factor, orb2_num_levels) 
TypeError: 'NoneType' object is not callable

Hi Luigi
in camera.py ==> uvs_undistorted = cv2.undistortPoints(np.expand_dims(uvs, axis=1), self.K, self.D, None, self.K) gives this error
in order to fix this, i modified the uvs input :
imagePoints = np.ascontiguousarray(uvs[:, :2]).reshape((uvs.shape[0], 1, 2))
uvs_undistorted = cv2.undistortPoints(imagePoints, self.K, self.D, None, self.K)

Is it possible to use the VO Pipeline with features that all lie in the same plane e.g. capturing the ground in a downward looking camera?

hi folks,

I've ran installed the basic .sh files and while running the vo python file , i'm getting the following error :

(main_vo.py:153596): Gdk-ERROR **: 05:27:39.152: The program 'main_vo.py' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadGC (invalid GC parameter)'.
(Details: serial 790 error_code 13 request_code 60 (core protocol) minor_code 0)
(Note to programmers: normally, X errors are reported asynchronously;
that is, you will receive the error a while after causing it.
To debug your program, run it with the GDK_SYNCHRONIZE environment
variable to change this behavior. You can then get a meaningful
backtrace from your debugger if you break on the gdk_x_error() function.)

Hey @luigifreda,

Thank you for such a comprehensive framework, I am seemingly stuck at the very first step of the process, I am using a system with Ubuntu 18.04 and which has conda installed in it.

However, when I try to install from the install_all_conda.sh, it completes the process, but it fails to load cv2 and just keeps throwing the error there is no cv2 installed. And if I try to check for

pip list

It basically shows me as though there is no pip while I am getting the right output for
which pip
which points to the pip inside the virtual environment pyslam. I am not sure why this is the case. Would appreciate any help in this direction. Also I would request you to put across all the modules that are necessary for the successful running of main_slam.py and main_vo.py so that even if these files don't work as in my case we know what is necessary to manually compile the list of requirements rather than looking through the entire list of things one by one.

Thanks in advance!

Hi Luigi,

Thank you for the nice implementation! I'd like to know if you have any plan to integrate VIO in pyslam. thank you!

Best,

Hi Luigi,

I followed the Mac installation steps on Big Sur 11.2.3 with install_all_mac_venv.sh and everything went okay except for third party orbslam2_features. The following message comes up:

currently used pyenv: pyslam 
-- The C compiler identification is AppleClang 12.0.0.12000032
-- The CXX compiler identification is AppleClang 12.0.0.12000032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:19 (message):
  OpenCV > 2.4.3 not found.

It would seem cmake doesn't find OpenCV but I can successfully run
python3 -c "import cv2; print(cv2.__version__)"
to get 3.4.2.
This issue seems related but I couldn't successfully apply it.

Edit

Found a solution for this issue. Will write down the answer soon.

When I run calibrate.py it gives me a traceback: line 61, in
h, w = cv.imread(img_names[0], cv.IMREAD_GRAYSCALE).shape[:2] # TODO: use imquery call to retrieve results
IndexError: list index out of range.

I found a couple of instructions missing from the readme so I am writing them here, @luigifreda please include them in the readme so that newbies like me don't stay stuck with these errors:

1) SIFT error:

While this is common knowledge to most people that have worked with opencv, for a newcomer it may not be, so I have included the fix here anyway:
(Assuming you are running python>=3.5)

pip3 install opencv-contrib-python==3.4.0.12

this should fix the error.

2) g2o Error:

 File ".../pyslam/optimizer_g2o.py", line 96, in optimization
    edge.fx = f.fx 
AttributeError: 'g2o.EdgeSE3ProjectXYZ' object has no attribute 'fx'

fix:
There is a replacement file in pyslam/lib/g2opy_changes

types_six_dof_expmap.h

you have to replace the file in

g2opy/python/types/sba/

and then install g2o (assuming you have already created the build directory)

cmake ..
make -j8
cd ..
python3 setup.py install

3) ndarray is not C-contiguous

an error as follows will pop-up:

ValueError: ndarray is not C-contiguous

DO NOT uninstall/ downgrade numpy to below 1.10.x! The fix for this is simple:
open pyslam/search_points.py, find this line (for me it was line 79):

projs = f_cur.project_map_points(points)

and replace it with :

projs = f_cur.project_map_points(points)
projs = projs.copy(order='C')

(or just add the second line. I use "replace" so that newbies can just copy-paste it).

Hi, there! Thank you so much for your repo! I have everything set up under Windows10 WSL2 and it's working flawlessly! I'm currently working on a research project related to 6DoF camera pose tracking and I found you're repo very useful for some Non-DNN Visual Odometry methods (feature extraction & comparison).

I'm wondering is adding the dataset support for the Scannet part of your plan? It's a very nice dataset providing a lot of information data for similar research tasks.

Thanks for such a great work!

It is really valuable for me about these supplied feature descriptors, but how can I just use these descriptors and generate a corresponding visualization in my own pics?

I am looking forward for your replay.
Nice work again.

How to start my webcam? whatever I do, your video starts up. I use the main_vo.py file.

In dataset.py i found LiveDataset

To config.ini i added
[LIVE_DATASET] type=live base_path=/dev/video1 name=/dev/video1 cam_settings=settings/KITTI-my.yaml groundtruth_file=auto

End set type=LIVE_DATASET

I modify the function in dataset.py.
`class LiveDataset(Dataset):
def init(self, path, name, associations=None, type=DatasetType.VIDEO):
super().init(path, name, associations, type)
self.camera_num = name # use name for camera number
self.cap = cv2.VideoCapture(self.camera_num)
if not self.cap.isOpened():
raise IOError('Cannot open camera '+self.camera_num)

def getImage(self, frame_id):
    ret, image = self.cap.read()
    if ret is False:
        print('ERROR in reading from camera: ', self.camera_num)
    return self.getImageColor(image)`

because I had error:
provided image has not the same size as the camera model or image is not grayscale

Now I have one error

File "/home/tony/PycharmProjects/pyslam-master/mplot_thread.py", line 206, in setAxis self.ax.axis('equal') File "/home/tony/.local/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 1677, in axis self.set_aspect('equal', adjustable='datalim') File "/home/tony/.local/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 1281, in set_aspect 'It is not currently possible to manually set the aspect ' NotImplementedError: It is not currently possible to manually set the aspect on 3D axes

I have modified backbone of superpoint, and I want to apply my model to this project. But I donnt konw ur node name of "pyslam/thirdparty/superpoint/superpoint_v1.pth". PLZ help me~ thank you!

Is there any possibility to add IMU to this project?

Plzz help me to solve this problem.
Scanning dependencies of target pyslam_utils
[ 33%] Building CXX object CMakeFiles/pyslam_utils.dir/utils_module.cpp.o
[ 66%] Building CXX object CMakeFiles/pyslam_utils.dir/utils.cpp.o
In file included from /home/intelligence/pyslam/pyslam/cpp/utils/utils.cpp:2:0:
/home/intelligence/pyslam/pyslam/cpp/utils/utils.h:20:94: error: ‘WARP_FILL_OUTLIERS’ is not a member of ‘cv’
const int warp_flags= cv::WARP_INVERSE_MAP + cv::INTER_CUBIC + cv::WARP_FILL_OUTLIERS);
^~~~~~~~~~~~~~~~~~
/home/intelligence/pyslam/pyslam/cpp/utils/utils.h:27:94: error: ‘WARP_FILL_OUTLIERS’ is not a member of ‘cv’
const int warp_flags= cv::WARP_INVERSE_MAP + cv::INTER_CUBIC + cv::WARP_FILL_OUTLIERS);
^~~~~~~~~~~~~~~~~~
CMakeFiles/pyslam_utils.dir/build.make:86: recipe for target 'CMakeFiles/pyslam_utils.dir/utils.cpp.o' failed
make[2]: *** [CMakeFiles/pyslam_utils.dir/utils.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /home/intelligence/pyslam/pyslam/cpp/utils/utils_module.cpp:9:0:
/home/intelligence/pyslam/pyslam/cpp/utils/utils.h:20:94: error: ‘WARP_FILL_OUTLIERS’ is not a member of ‘cv’
const int warp_flags= cv::WARP_INVERSE_MAP + cv::INTER_CUBIC + cv::WARP_FILL_OUTLIERS);
^~~~~~~~~~~~~~~~~~
/home/intelligence/pyslam/pyslam/cpp/utils/utils.h:27:94: error: ‘WARP_FILL_OUTLIERS’ is not a member of ‘cv’
const int warp_flags= cv::WARP_INVERSE_MAP + cv::INTER_CUBIC + cv::WARP_FILL_OUTLIERS);
^~~~~~~~~~~~~~~~~~
/home/intelligence/pyslam/pyslam/cpp/utils/utils_module.cpp: In function ‘void pybind11_init_pyslam_utils(pybind11::module&)’:
/home/intelligence/pyslam/pyslam/cpp/utils/utils_module.cpp:37:152: error: ‘WARP_FILL_OUTLIERS’ is not a member of ‘cv’
"image"_a,"kps"_a,"patch_size"_a,"use_orientation"_a=true,"scale_factor"_a=1.0,"warp_flags"_a=cv::WARP_INVERSE_MAP + cv::INTER_CUBIC + cv::WARP_FILL_OUTLIERS
^~~~~~~~~~~~~~~~~~
CMakeFiles/pyslam_utils.dir/build.make:62: recipe for target 'CMakeFiles/pyslam_utils.dir/utils_module.cpp.o' failed
make[2]: *** [CMakeFiles/pyslam_utils.dir/utils_module.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/pyslam_utils.dir/all' failed
make[1]: *** [CMakeFiles/pyslam_utils.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

I'm trying to install pyslam2 in my system.
System Information :
ubuntu 18.04
architecture - x86_64
conda environment
I installed all basic libraries by ./install_basic.sh then when I try to install libs pangolin, g2opy I get CMake Error as follow

[Installing delf ...                                                                                                                                           [118/1615]
================================================ 
Building and installing cpp ... 
-- found OpenCV version: 4.3.0
-- opencv include: /usr/local/include/opencv4
-- opencv lib dirs: /usr/local
-- opencv libs: opencv_calib3d;opencv_core;opencv_dnn;opencv_features2d;opencv_flann;opencv_gapi;opencv_highgui;opencv_imgcodecs;opencv_imgproc;opencv_ml;opencv_objdete
ct;opencv_photo;opencv_stitching;opencv_video;opencv_videoio;opencv_aruco;opencv_bgsegm;opencv_bioinspired;opencv_ccalib;opencv_datasets;opencv_dnn_objdetect;opencv_dnn
_superres;opencv_dpm;opencv_face;opencv_freetype;opencv_fuzzy;opencv_hdf;opencv_hfs;opencv_img_hash;opencv_intensity_transform;opencv_line_descriptor;opencv_optflow;ope
ncv_phase_unwrapping;opencv_plot;opencv_quality;opencv_rapid;opencv_reg;opencv_rgbd;opencv_saliency;opencv_shape;opencv_stereo;opencv_structured_light;opencv_superres;o
pencv_surface_matching;opencv_text;opencv_tracking;opencv_videostab;opencv_xfeatures2d;opencv_ximgproc;opencv_xobjdetect;opencv_xphoto
-- pybind11 v2.3.dev0
-- Configuring done
-- Generating done
-- Build files have been written to: /home/nikhil/Desktop/practice/pyslam/cpp/utils/build
[100%] Built target pyslam_utils
================================================ 
Building Thirdparty 
================================================ 
================================================
Configuring and building thirdparty/Pangolin ... 
================================================================== 
Configuring and building thirdparty/g2o ... 
-- Compiling on Unix
-- Found CHOLMOD and its dependencies
-- Building LGPL code as static library (affects license of the binary)
-- Compiling with OpenGL support
-- Could NOT find QGLVIEWER (missing: QGLVIEWER_INCLUDE_DIR QGLVIEWER_LIBRARY)
-- Compiling with GCC
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
OPENGL_gl_LIBRARY (ADVANCED)
    linked by target "freeglut_minimal" in directory /home/nikhil/Desktop/practice/pyslam/thirdparty/g2opy/EXTERNAL/freeglut
    linked by target "types_slam2d" in directory /home/nikhil/Desktop/practice/pyslam/thirdparty/g2opy/g2o/types/slam2d
    linked by target "types_slam3d" in directory /home/nikhil/Desktop/practice/pyslam/thirdparty/g2opy/g2o/types/slam3d
    linked by target "types_sclam2d" in directory /home/nikhil/Desktop/practice/pyslam/thirdparty/g2opy/g2o/types/sclam2d
    linked by target "types_slam2d_addons" in directory /home/nikhil/Desktop/practice/pyslam/thirdparty/g2opy/g2o/types/slam2d_addons
    linked by target "types_slam3d_addons" in directory /home/nikhil/Desktop/practice/pyslam/thirdparty/g2opy/g2o/types/slam3d_addons

-- Configuring incomplete, errors occurred!
See also "/home/nikhil/Desktop/practice/pyslam/thirdparty/g2opy/build/CMakeFiles/CMakeOutput.log".](url)
](url)

How can I fix this ?

Dear luigifreda,

I'm trying to find in order to find the parameter settings same as "main-slam-kitti-map".

But, even though I tried increasing kTrackingWaitForLocalMappingSleepTime, and kTrackingWaitForLocalMappingToGetIdle=True, a path diverges at 800 frame in kitti 06 seq.

Could you please let me know the parameters used for generating "main-slam-kitti-map" using this program, please?

(I guessed you used SUPERPOINT and some parameter tweak.

But I failed all possible combination I knew)

Thank you.

i am working with ubuntu 20.04
i have successfully executed the following commands:

• git clone --recursive https://github.com/luigifreda/pyslam.git
• cd pyslam
• git checkout ubuntu20
• . pyenv-create.sh
• . install_all_venv.sh

the command 'python3 -O main_vo.py' works well, but the command 'python3 -O main_slam.py' doesn't work:

(pyslam) (base) leequer@leequer-PC:~/pyslam-ubuntu20$ python3 -O main_slam.py
importing pygame
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
initialising pygame
Traceback (most recent call last):
File "main_slam.py", line 41, in
from viewer3D import Viewer3D
File "/home/leequer/pyslam-ubuntu20/viewer3D.py", line 25, in
import pypangolin as pangolin
ImportError: /lib/x86_64-linux-gnu/librsvg-2.so.2: undefined symbol: cairo_tag_end

it seems that there is something wrong with my pangolin. what should i do？

Installation script fails when opencv-contrib-python is not installed. Following error message occured:

Cannot uninstall requirement opencv-contrib-python, not installed

How do I localize with reference to the prebuilt map?

I am testing main_slam.py with a short video that i captured in a parking with a dashcam. It contains one big loop at low speed. I observe big drift and scaling issue especially at the second half. Moreover at the beginning and at each turn (90 degree), I see multiple path in the map viewer. Do you think that it is normal or maybe i do something wrong with the settings ? Here is i am adding two screenshot which shows the trajectory and the map

Thanks

Hi, I am quite new to computer vision based localization and just wanted to try this out. When running the main_slam.py I get the following error.

  File ".../pyslam/optimizer_g2o.py", line 96, in optimization
    edge.fx = f.fx 
AttributeError: 'g2o.EdgeSE3ProjectXYZ' object has no attribute 'fx'

I understand that this error is related to g2opy, but I was wondering how everyone else got around this, since the above object has not been used in any of the examples given in g2opy. It would also be helpful if there was another way to obtain whatever is required here (as in, if there is a member function of that object from which I should get the values).

P.S: I made no changes to the source code of pyslam or g2o

@luigifreda
Thank you for your excellent work!
I want to save image (error, 3D trajectory) when I run main_vo.py
I add this code:

if is_draw_3d:

    if not kUsePangolin:
        **plt3d.ax(data=plt3d.data)
        plt.savefig('3D.png')**
        plt3d.quit()
    else: 
        viewer3D.quit()

And this error occurs:
AttributeError: 'Mplot3d' object has no attribute 'ax'
As I can see in the init function of Mplot3d mplot_thread.py, self.ax is defined as:

    self.fig = plt.figure()

    if kUseFigCanvasDrawIdle:
        self.fig.canvas.draw_idle()         
    self.fig.canvas.mpl_connect('key_press_event', self.on_key_press)       
    self.fig.canvas.mpl_connect('key_release_event', self.on_key_release)             
    self.ax = self.fig.gca(projection='3d')
    if self.title is not '':
        self.ax.set_title(self.title)     
    self.ax.set_xlabel('X axis')
    self.ax.set_ylabel('Y axis')
    self.ax.set_zlabel('Z axis')	

Could you please tell me how I can solve this problem?

Hello,
Thank you for contributing such a great implementation of visual odometry in python.
I'm currently doing research on drones and I want to implement this on my research. I was wondering if this work can be used in real-time tasks. Can you tell me how to use this on a single camera/webcam?

To install pyslam under macOS, I just followed the instructions to run './install_all_mac_venv.sh', however, there is not a file's name is 'install_all_mac_venv.sh'. So, I just wonder that how to launch the macOS install script? Thanks.

Hello, I am eager to run pyslam on my Mac. But when I created the anaconda environment on it, I got the following errors:

./pyenv-conda-create.sh
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:
  - numpy==1.18.1=py36h7314795_1
  - sqlite==3.31.1=h7b6447c_0
  - _tflow_select==2.1.0=gpu
  - tensorboard==1.14.0=py36hf484d3e_0
  - tensorflow-gpu==1.14.0=h0d30ee6_0
  - gnutls==3.6.5=hd3a4fd2_1002
  - gmp==6.2.0=he1b5a44_2
  - gst-plugins-base==1.14.5=h0935bb2_2
  - xorg-libxrender==0.9.10=h516909a_1002
  - pcre==8.44=he1b5a44_0
  - xorg-xextproto==7.3.0=h14c3975_1002
  - cudnn==7.6.5=cuda10.1_0
  - zlib==1.2.11=h7b6447c_3
  - xorg-xproto==7.0.31=h14c3975_1007
  - xorg-libxdmcp==1.1.3=h516909a_0
  - ld_impl_linux-64==2.33.1=h53a641e_7
  - hdf5==1.10.2=hba1933b_1
  - cupti==10.1.168=0
  - jpeg==9c=h14c3975_1001
  - h5py==2.8.0=py36h989c5e5_3
  - tk==8.6.8=hbc83047_0
  - libopenblas==0.3.7=h5ec1e0e_6
  - libffi==3.2.1=hd88cf55_4
  - cudatoolkit==10.1.243=h6bb024c_0
  - protobuf==3.11.4=py36he6710b0_0
  - libgfortran-ng==7.3.0=hdf63c60_5
  - expat==2.2.9=he1b5a44_2
  - xorg-kbproto==1.0.7=h14c3975_1002
  - readline==7.0=h7b6447c_5
  - qt==5.12.5=hd8c4c69_1
  - tensorflow-base==1.14.0=gpu_py36he45bfe2_0
  - openh264==1.8.0=hdbcaa40_1000
  - c-ares==1.15.0=h7b6447c_1001
  - fontconfig==2.13.1=h86ecdb6_1001
  - zstd==1.4.4=h3b9ef0a_2
  - libclang==9.0.1=default_hde54327_0
  - gstreamer==1.14.5=h36ae1b5_2
  - nss==3.47=he751ad9_0
  - dbus==1.13.6=he372182_0
  - xz==5.2.4=h14c3975_4
  - openssl==1.1.1e=h7b6447c_0
  - wrapt==1.12.1=py36h7b6447c_1
  - libtiff==4.1.0=hc3755c2_3
  - xorg-libxau==1.0.9=h14c3975_0
  - pixman==0.38.0=h516909a_1003
  - libuuid==2.32.1=h14c3975_1000
  - harfbuzz==2.4.0=h9f30f68_3
  - lz4-c==1.8.3=he1b5a44_1001
  - freetype==2.10.1=he06d7ca_0
  - libxcb==1.13=h14c3975_1002
  - graphite2==1.3.13=he1b5a44_1001
  - python==3.6.10=h0371630_0
  - libpng==1.6.37=hed695b0_1
  - libiconv==1.15=h516909a_1006
  - libedit==3.1.20181209=hc058e9b_0
  - libgcc-ng==9.1.0=hdf63c60_0
  - jasper==1.900.1=h07fcdf6_1006
  - libllvm9==9.0.1=hc9558a2_0
  - giflib==5.2.1=h516909a_2
  - xorg-libx11==1.6.9=h516909a_0
  - ncurses==6.2=he6710b0_0
  - bzip2==1.0.8=h516909a_2
  - xorg-libxext==1.3.4=h516909a_0
  - xorg-renderproto==0.11.1=h14c3975_1002
  - libwebp==1.0.2=h56121f0_5
  - glib==2.58.3=py36hd3ed26a_1003
  - grpcio==1.27.2=py36hf8bcb03_0
  - xorg-libsm==1.2.3=h84519dc_1000
  - gettext==0.19.8.1=hc5be6a0_1002
  - ffmpeg==4.1.3=h167e202_0
  - lame==3.100=h14c3975_1001
  - x264==1!152.20180806=h14c3975_0
  - libstdcxx-ng==9.1.0=hdf63c60_0
  - cairo==1.16.0=hcf35c78_1003
  - icu==64.2=he1b5a44_1
  - libprotobuf==3.11.4=hd408876_0
  - xorg-libice==1.0.10=h516909a_0
  - libxml2==2.9.10=hee79883_0
  - nspr==4.25=he1b5a44_0
  - nettle==3.4.1=h1bed415_1002
  - scipy==1.4.1=py36habc2bb6_0
  - libxkbcommon==0.10.0=he1b5a44_0
  - pthread-stubs==0.4=h14c3975_1001
  - tensorflow==1.14.0=gpu_py36h3fb9ad6_0

Can you help me with it?

Hi
Thanks a lot for this co;plete VO pipeline. I have followed your installation guide 'Install pySLAM in a Custom Python Virtual Environment (conda)' all the requirements installed successfully including opencv-3.4.1 however when I try to run the main_vo.py in the environment, I have the no module found cv2 error. do you have any idea how i can solve this ?

Thanks in advance

Hi,
I got following error when run the main_slam

File "/home/dam/.local/lib/python3.5/site-packages/ordered_set.py", line 134, in copy
return self.class(self)
File "/home/dam/.local/lib/python3.5/site-packages/ordered_set.py", line 69, in init
self |= iterable
TypeError: unsupported operand type(s) for |=: 'OrderedSet' and 'OrderedSet'

Issue solved by installing lower version of OrderedSet

pip install ordered-set==3.1.1

When I run calibrate.py it gives me a traceback: line 61, in
h, w = cv.imread(img_names[0], cv.IMREAD_GRAYSCALE).shape[:2] # TODO: use imquery call to retrieve results
IndexError: list index out of range.

Hello!
How to run vo on my own data without groundtruth?

I try to run pyslam on a series of images, using a FolderDataset. But I get the following error:

# python3 main_vo.py
fps:  None
Traceback (most recent call last):
  File "main_vo.py", line 59, in <module>
    dataset = dataset_factory(config.dataset_settings)
  File "/root/pyslam/dataset.py", line 67, in dataset_factory
    dataset = FolderDataset(path, name, associations, DatasetType.FOLDER)      
  File "/root/pyslam/dataset.py", line 184, in __init__
    self.Ts = 1./self.fps 
TypeError: unsupported operand type(s) for /: 'float' and 'NoneType'

When looking at dataset.py, the reason seems obvious to me: The constructor is defined as :

class FolderDataset(Dataset): 
    def __init__(self, path, name, fps=None, associations=None, type=DatasetType.VIDEO): 
        super().__init__(path, name, fps, associations, type)  
        print('fps: ', self.fps)  
...

but it's called in the following way:

    if type == 'folder':
        dataset = FolderDataset(path, name, associations, DatasetType.FOLDER)   

The fps argument is missing here.

Great repo! How do I perform relocalization based on the prebuilt map?

Hi, when I tried to run main_slam.py, it shows AttributeError: module 'g2o' has no attribute 'Flag'.

Thank you!

Hi Luigi,

Thanks again for this great slam toolbox. Considering the capabilities of the pyslam, I would like to know if you have any more detailed docs/tutorials that explain main parts of the toolbox. It would be great for my better understanding of slam process while using the toolbox.

Best

Ferit

There are multiple fuzzy full image matching https://github.com/thorn-oss/perception
When making it Transform-Invariant some has used triangle meshes https://pippy360.github.io/transformationInvariantImageSearch/
But as seen with this repo, there is an alternative https://github.com/luigifreda/pyslam#-supported-local-features
e.g.

• https://all4win.github.io/projects/cv_proj2/index.html
• https://cclaassen3.github.io/computer-vision-local-feature-matching/

Are there any algorithms that are included in this repo that does something similar?

Greetings,

I am trying to install and run pyslam in a shared university environment, for a course.

I have the ability to create python virtual environments, install packages in them, the ability to compile software, as well as basic operations like downloading from git repos. This environment is also the only place I have access to computational/GPU resources to run pytorch and tensorflow.

However, this environment is not running ubuntu 18.04 nor am I allowed to install system packages. As a result, the install scripts cannot be run.

These scripts are nested and very complex, and I am finding it difficult to trace many of the actions they perform in order to satisfy pyslam's dependencies.

Is it possible you could provide an equivalent list of all the manual steps required to set up pyslam's dependencies? Which steps may not be required, depending on the desired feature detector and descriptor?

Thank you very much for your help,
-Matthew Bradley

Hi, I am new to robotics and SLAM in general. I was hoping for some guidance in using this tool. I want to be able to generate a map from a self propelling robot with one camera and have the bot localize itself. I think this is the right toolkit to use. I just don't know where to start. Could I just try this with a webcam and start generating a map of a room?

@luigifreda Thanks for this great work!

I am trying to use tracker_config = FeatureTrackerConfigs.SIFT in main_vo.py

I have noticed that this will create a DescriptorFeatureTracker that uses DES_BF = 1 # descriptor-based, brute force matching with knn right ?

My question is about the matcher:

self.matcher = feature_matcher_factory(norm_type=self.norm_type, ratio_test=match_ratio_test, type=self.matching_algo)

it seems that norm_type=self.norm_type is always set to norm_type=cv2.NORM_HAMMING
but for SIFT it should be NORM_L2?

Thank you for your excellent work!

I have run main_slam.py with tum dataset and want to generate trajectory file as tum given groundtruth.txt.

Is the slam.tracking.tracking_history.relative_frame_poses the history of estimated trajectory of slam?
If yes, I have tried to print the last element of slam.tracking.tracking_history.relative_frame_poses and its timestamp, but it's different from the groundtruth.
In ground truth:
1341845986.0403 -0.2093 -2.9924 1.3790 -0.7683 0.0502 -0.1037 0.6296
my result:
1341845986.0421 0.02307 0.0387 0.0204 -0.0093 0.0049, -0.0108 0.9998

I use this code to print my result:

from scipy.spatial.transform import Rotation as R
print(timestamp)
print(R.from_matrix(slam.tracking.tracking_history.relative_frame_poses[-1].orientation().matrix()))
print(slam.tracking.tracking_history.relative_frame_poses[-1].position())

Do I misunderstand something?
Do I need to calibrate camera for tum dataset?
Could you give me some advice to generate this estimated trajectory file?

This can be added to the ReadME for some of the descriptors not in this repo: https://gist.github.com/natowi/240120ea2e6b6a1b9e45b0f77f2b0b7e
https://github.com/OpenSLAM/awesome-SLAM-list

Dear Luigi,
When I use the TUM dataset and I have to edit the file about the camera settings, I'm not sure about some parameters.
What is the meaning of fps, IR and depth? Where can I find these values?

Thank you in advance