shichaoy / pop_up_slam Goto Github PK

The following two The two pictures below show the error and the result of running the program on my computer. Many details are not shown. And what should I do to make it work properly?
And I've run sh install_dependenices.sh many times and failed at sudo pip install -r py_requirements.txt, so I've skipped over it.

hello,
I have run the code successfully before. And saw the expected result.
But, several weeks later, I run the code again, error occured.

Warning: TF_OLD_DATA ignoring data from the past for frame odom_init at time 1.34184e+09 according to authority unknown_publisher
Possible reasons are listed at http://wiki.ros.org/tf/Errors%20explained
at line 277 in /tmp/binarydeb/ros-kinetic-tf2-0.5.20/src/buffer_core.cpp

processing kf 0 1341836537.655661616
*** Error in `/home/zp/0pop_up_slam-master_good/devel/lib/pop_planar_slam/planar_test': double free or corruption (out): 0x0000000005047a00 ***

[planar_test-2] process has died [pid 11236, exit code -6, cmd /home/zp/0pop_up_slam-master_good/devel/lib/pop_planar_slam/planar_test /image:=/mono/image_mono /camera_info:=/mono/camera_info __name:=planar_test __log:=/home/zp/.ros/log/9c4ddef6-72ba-11e9-8104-107b4490cde3/planar_test-2.log].

And there is no label images in the RVIZ, It seems that the label image haven't transported into the system.
I have done nothing to the code, and my linux system also haven't changed.
Then, I download your code again, and compile it. Error also occurs. It just so weird.
Looking forward to your reply.

sorry to interrupt.
I had this problem running this program：pop_polygon in rviz can not received
I have no ideal how to solve this problem.
Thank you for your helping

• images.
  
  

Hello :)
When I ran the system with your example, I received the same result each time.
However when I tried the system on a bag file which I built using a video of my own I receive different result each run.
Here are a few examples that display the issue:

I am unsure what might be the issue that is causing this. Can you help me understand what might be the cause?

In addition to the issue above, I also noticed the following issues, which might indicate on the cause of the issue above:

1. There are multiple occasions where the segmentation and pop polyogn windows freeze for a few moments.
2. I received the following warning: "[ WARN] [1549441953.616061157, 1341836538.688953038]: Dual match, add new landmark 1 at frame 11"

My first guess is that the format of the bag file I provide is incorrect.
Therefore I also wanted to ask a few questions regarding the expected format of the bag file.

I noticed that, in the example bag file you supplied, the number of messages in the camera info topic is different than the number of messages in the rgb and segmentation topics, where the latter is the number of frames. Can you explain the reason for this? Or, equally, given a video of N frames, how many messages should the camera info topic should contain?

In addition I noticed that in the example video there are 82 frames supplied while it seems the system indentifies that the length of the video is 48 seconds. Therefore I concluded that the sample rate of the video you supplied might be low (around 2 fps), while my video was filmed at a standard camera rate of around 30 fps.
Does your system assume anything on the rate sample of the video?

And finally, I would like to provide a link to the bag file I used in case you would like to try to reproduce and examine the issue yourself

https://drive.google.com/file/d/1ytnL2h7qheFo8mp025N-ZDzZEWuN4OSv/view?usp=sharing

Thank you very much in advance!

May I ask if there are many small faces in my corridor scene, instead of long ones pieced together

Hi,
I am trying to download the params and dataset using the sh download_data.sh but seems the link to the data is down. Can you check or reupload it.

Thanks a lot.

error: invalid conversion from ‘int’ to ‘cv::AccessFlag’ [-fpermissive]
return stdAllocator->allocate(u, accessFlags, usageFlags);

error: cannot declare variable ‘g_numpyAllocator’ to be of abstract type ‘NumpyAllocator’
NumpyAllocator g_numpyAllocator;

Hi, I am reading your paper "Pop-up SLAM: Semantic Monocular Plane SLAM for Low-texture Environments" and I find it hard to understand how you compute the plane equation in the camera coordinate. I don't know how to estimate the initial camera pose in the world coordinate to get Tw,c, then using the equation (6) to get pi_c. There is a section that you start with "Then we show how to compute the plane equation pi_c", while I think it is explaining how to compute the plane equation of the wall.

Would you please tell me how to understand this?

Best regards

Hello
What does this code mean
“vertical_3d_world.col(0) = temp_vertex; //.row(line_ind).segment<3>(pt_ind*3);
vertical_3d_world.col(1) = temp_vertex + Vector3f(0, 0, 2);”
When searching for the direction of hitting upwards or downwards in the find_2d_3d_closed_polygon.

Hello!The link of the dataset does not work,Can you provide me another valid dataset link？Thank you !!!

/popup_ws/src/pop_up_slam/dependency/numpy_opencv/src/np_opencv_converter.cpp:12:2: error: return-statement with a value, in function returning 'void' [-fpermissive]
import_array();

/popup_ws/src/pop_up_slam/dependency/numpy_opencv/src/utils/conversion.cpp:200:4: error: return-statement with a value, in function returning 'void' [-fpermissive]
import_array();

hello, thanks for your great work.
I run the pop_up_slam, and the Rviz came out successfully, but the terminal showed error like:

[ERROR] [1542117455.104213710, 1454368534.284882313]: publish all cloud
[rosbag-5] process has finished cleanly
log file: /home/zp/.ros/log/0255265a-e74c-11e8-b148-107b4490cde3/rosbag-5*.log
[ERROR] [1542117457.879948980, 1454368534.889248398]: finish pub cloud num 106

Have you ever met this problem?
Thanks for your reply!

catkin_make --only-pkg-with-deps odom_tf_utils

hello, shichaoy.
I have run the codes with tum_far_preproc_tf.bag successfully, but when I change the .bag file into others, such as rgbd_dataset_freiburg3_structure_notexture_far.bag, the whole codes won't give any error, but the RVIZ won't show anything. There is nothing out. So, I'm writing to ask whether the tum_far_preproc_tf.bag actually is the .bag file that has been pre-processed one? And how can I make some tests on other data series?
Thanks for you reply.

Using intervaltree version 3,I meet such error:"AttributeError: 'IntervalTree' object has no attribute 'search'".It can be fixed by replacing "search" with "overlap" or "envelop".Which one is right?

hello, shichaoy
I was meant to print the pose of every keyframe, but as a result, when the terminal screen showed that we have processed more than 200 keyframes, the final pose.txt only includs 82 poses, which is the exact number of the dataset you used. However, I have changed the dataset.
So, I'm writing to ask about this problem...
Looking forward to your reply.

started core service [/rosout]
process[world_to_odominit-2]: started with pid [8793]
process[planar_test-3]: started with pid [8794]
process[tf_to_odom-4]: started with pid [8801]
process[rosbag-5]: started with pid [8802]
process[rviz-6]: started with pid [8804]
[planar_test-3] process has died [pid 8794, exit code -11, cmd /home/zhang/workspace/popup_ws/devel/lib/pop_planar_slam/planar_test /image:=/mono/image_mono /camera_info:=/mono/camera_info __name:=planar_test __log:=/home/zhang/.ros/log/dc7a796c-2431-11e9-885b-b083feb6fccb/planar_test-3.log].
log file: /home/zhang/.ros/log/dc7a796c-2431-11e9-885b-b083feb6fccb/planar_test-3*.log
Warning: TF_OLD_DATA ignoring data from the past for frame odom_init at time 1.34184e+09 according to authority unknown_publisher
Possible reasons are listed at http://wiki.ros.org/tf/Errors%20explained
at line 277 in /tmp/binarydeb/ros-indigo-tf2-0.5.18/src/buffer_core.cpp
[rosbag-5] process has finished cleanly
log file: /home/zhang/.ros/log/dc7a796c-2431-11e9-885b-b083feb6fccb/rosbag-5*.log

Excuse me, the two bags downloaded from the cloud storage are not the same set of data ？I want to run a complete effect. How should I operate it

Hello!
Can you please elaborate more on the steps needed to implement pop up SLAM with a point SLAM system (ORB-SLAM2 or LSD).

I read in your paper that you proposed to integrate the output of LSD SLAM as odometry constarints to planar SLAM. Can you please elaborate more on how this can be done practically and what needs to be adjusted in your package.

Also, can your system be implemeted with a stereo-based point SLAM system?

Your help is highly appreciated.

Hello :)
First of all, thank you for publishing this github repository!

I am trying to follow the installation guide that you posted and I was stuck in the step called 'Download data'.

When running the command sh download_data.sh I got the following output:

mkdir: cannot create directory ‘pop_planar_slam/dataset’: File exists
--2019-01-12 20:55:19--  http://www.frc.ri.cmu.edu/~syang/dataset/iros_pop_slam_16/tum_far_preproc_tf.bag
Resolving www.frc.ri.cmu.edu (www.frc.ri.cmu.edu)... 128.2.205.32
Connecting to www.frc.ri.cmu.edu (www.frc.ri.cmu.edu)|128.2.205.32|:80... connected.
HTTP request sent, awaiting response... 401 Unauthorized

Username/Password Authentication Failed.

It seems that the script attempts to access two links from the site www.frc.ri.cmu.edu and both of them require a username and password to enter. Therefore the data in inaccessible.

Can you provide a way to access the data somehow? :)
Thank you very much!

which vision of Eigen is it？

Hello,

I am trying to select the ground edges directly from the CNN segmentation image.
I changed this line in main_3d.cpp:

to:
pop_up_obj->get_ground_edges(label_img,label_img);

However, I am getting the following error:

Any idea why this might be happening?

Your help is very much appreciated. Thank you.

Hi,

I've successfully built the code and run the following line for a test.

roslaunch pop_planar_slam pop_planar_tum_far_example.launch

Then I got the error here.

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/jason/SqX/popup_ws/devel/lib/python2.7/dist-packages/pop_up_python/__init__.py", line 35, in <module>
    exec(__fh.read())
  File "<string>", line 1, in <module>
  File "/home/jason/SqX/popup_ws/src/pop_up_slam/pop_up_python/python/pop_up_python/pop_up_fun.py", line 7, in <module>
    from intervaltree import Interval, IntervalTree  # to select not overlapping edges
ImportError: No module named intervaltree
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'pop_up_fun' is not defined

I'm sure I've installed "intervaltree". But when I test it in python, it does not work and outputs the same ImportError.
I wonder if anyone has an idea what else I can do to make it work properly.

Thanks very much for helping.

cheers,
Elinor

Hello @shichaoy , thank you and your team for great work on exploring combination of SLAM with Segmentation Network, I have successfully run your pop_up_image but some error "pop up" with this SLAM though.

I have built DBoW2 successfully and install python requirements with pip2 as:

sudo -H pip2 install -r py_requirements.txt

My system pip2 is 9.0.3 and it installed without any error.

After download data, I make the workspace with catkin_make -j8 but it resulted in following failure:

/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/Mapping.cpp: In member function ‘void Mapper_mono::update_plane_measurement()’:
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/Mapping.cpp:600:6: error: ‘update_plane_equation_from_seg’ is not a member of ‘popup_plane’
      popup_plane::update_plane_equation_from_seg(one_frame->ground_seg2d_lines,inv_calib,pose, all_planes_sensor_new);
      ^
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp: In constructor ‘plane_slam::plane_slam(ros::NodeHandle&)’:
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:179:14: error: ‘class popup_plane’ has no member named ‘walllength_threshold’
  pop_up_obj->walllength_threshold = 3.0;
              ^
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp: In member function ‘void plane_slam::img_callback_cnn_crf(const ConstPtr&, const ConstPtr&)’:
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:417:14: error: ‘class popup_plane’ has no member named ‘walllength_threshold’
  pop_up_obj->walllength_threshold = 3.0;
              ^
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:418:14: error: ‘class popup_plane’ has no member named ‘simple_polygon_mode’
  pop_up_obj->simple_polygon_mode = false;
              ^
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:428:55: error: ‘class popup_plane’ has no member named ‘ground_seg2d_lines_actual’
  MatrixX4f utilized_ground_seg2d_lines  = pop_up_obj->ground_seg2d_lines_actual;     // whether include connected segments
                                                       ^
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:434:57: error: no matching function for call to ‘is_finite(Eigen::MatrixXf&)’
  if (!is_finite<MatrixX4f>(pop_up_obj->all_planes_sensor))  //check nan
                                                         ^
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:62:13: note: candidate: template<class Derived> bool is_finite(const Eigen::MatrixBase<Derived>&)
 inline bool is_finite(const Eigen::MatrixBase<Derived>& x)
             ^
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:62:13: note:   template argument deduction/substitution failed:
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:434:40: note:   cannot convert ‘((plane_slam*)this)->plane_slam::pop_up_obj->popup_plane::all_planes_sensor’ (type ‘Eigen::MatrixXf {aka Eigen::Matrix<float, -1, -1>}’) to type ‘const Eigen::MatrixBase<Eigen::Matrix<float, -1, 4> >&’
  if (!is_finite<MatrixX4f>(pop_up_obj->all_planes_sensor))  //check nan
                                        ^
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:441:66: warning: ISO C++ says that these are ambiguous, even though the worst conversion for the first is better than the worst conversion for the second:
      if (check_element_in_vector(i,pop_up_obj->good_plane_indices))  // good lines in red, bad lines in blue
                                                                  ^
In file included from /home/buiminh/Workspace/PopIm/src/pop_up_image/pop_up_wall/include/pop_up_wall/popup_plane.h:52:0,
                 from /home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:14:
/home/buiminh/Workspace/PopIm/src/pop_up_image/pop_up_wall/include/pop_up_wall/matrix_utils.h:93:6: note: candidate 1: bool check_element_in_vector(const float&, const VectorXf&)
 bool check_element_in_vector(const float& element, const VectorXf &vec_check);
      ^
/home/buiminh/Workspace/PopIm/src/pop_up_image/pop_up_wall/include/pop_up_wall/matrix_utils.h:94:6: note: candidate 2: bool check_element_in_vector(const int&, const VectorXi&)
 bool check_element_in_vector(const int& element, const VectorXi &vec_check);
      ^
[ 98%] Linking CXX executable /home/buiminh/Workspace/PopSLAM/devel/lib/pop_up_wall/pop_up_main_node
In file included from /usr/include/eigen3/Eigen/Core:297:0,
                 from /usr/include/eigen3/Eigen/Dense:1,
                 from /home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/Thirdparty/isam/include/isam/Slam.h:32,
                 from /home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/Thirdparty/isam/include/isam/isam.h:32,
                 from /home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/Mapping.h:9,
                 from /home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:10:
/usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h: In instantiation of ‘void Eigen::internal::call_assignment_no_alias(Dst&, const Src&, const Func&) [with Dst = Eigen::Matrix<int, -1, 1>; Src = Eigen::Matrix<float, -1, 1>; Func = Eigen::internal::assign_op<int>]’:
/usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h:712:27:   required from ‘void Eigen::internal::call_assignment(Dst&, const Src&, const Func&, typename Eigen::internal::enable_if<(! Eigen::internal::evaluator_assume_aliasing<Src>::value), void*>::type) [with Dst = Eigen::Matrix<int, -1, 1>; Src = Eigen::Matrix<float, -1, 1>; Func = Eigen::internal::assign_op<int>; typename Eigen::internal::enable_if<(! Eigen::internal::evaluator_assume_aliasing<Src>::value), void*>::type = void*]’
/usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h:693:18:   required from ‘void Eigen::internal::call_assignment(Dst&, const Src&) [with Dst = Eigen::Matrix<int, -1, 1>; Src = Eigen::Matrix<float, -1, 1>]’
/usr/include/eigen3/Eigen/src/Core/PlainObjectBase.h:682:32:   required from ‘Derived& Eigen::PlainObjectBase<Derived>::_set(const Eigen::DenseBase<OtherDerived>&) [with OtherDerived = Eigen::Matrix<float, -1, 1>; Derived = Eigen::Matrix<int, -1, 1>]’
/usr/include/eigen3/Eigen/src/Core/Matrix.h:225:24:   required from ‘Eigen::Matrix<_Scalar, _Rows, _Cols, _Options, _MaxRows, _MaxCols>& Eigen::Matrix<_Scalar, _Rows, _Cols, _Options, _MaxRows, _MaxCols>::operator=(const Eigen::DenseBase<OtherDerived>&) [with OtherDerived = Eigen::Matrix<float, -1, 1>; _Scalar = int; int _Rows = -1; int _Cols = 1; int _Options = 0; int _MaxRows = -1; int _MaxCols = 1]’
/home/buiminh/Workspace/PopSLAM/src/pop_up_slam/pop_planar_slam/src/main_3d.cpp:494:36:   required from here
/usr/include/eigen3/Eigen/src/Core/util/StaticAssert.h:32:40: error: static assertion failed: YOU_MIXED_DIFFERENT_NUMERIC_TYPES__YOU_NEED_TO_USE_THE_CAST_METHOD_OF_MATRIXBASE_TO_CAST_NUMERIC_TYPES_EXPLICITLY
     #define EIGEN_STATIC_ASSERT(X,MSG) static_assert(X,#MSG);
                                        ^
/usr/include/eigen3/Eigen/src/Core/util/XprHelper.h:707:3: note: in expansion of macro ‘EIGEN_STATIC_ASSERT’
   EIGEN_STATIC_ASSERT((internal::functor_is_product_like<BINOP>::ret \
   ^
/usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h:745:3: note: in expansion of macro ‘EIGEN_CHECK_BINARY_COMPATIBILIY’
   EIGEN_CHECK_BINARY_COMPATIBILIY(Func,typename ActualDstTypeCleaned::Scalar,typename Src::Scalar);
   ^
[ 98%] Built target pop_up_main_node
pop_up_slam/pop_planar_slam/CMakeFiles/planar_test.dir/build.make:86: recipe for target 'pop_up_slam/pop_planar_slam/CMakeFiles/planar_test.dir/src/Mapping.cpp.o' failed
make[2]: *** [pop_up_slam/pop_planar_slam/CMakeFiles/planar_test.dir/src/Mapping.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
pop_up_slam/pop_planar_slam/CMakeFiles/planar_test.dir/build.make:62: recipe for target 'pop_up_slam/pop_planar_slam/CMakeFiles/planar_test.dir/src/main_3d.cpp.o' failed
make[2]: *** [pop_up_slam/pop_planar_slam/CMakeFiles/planar_test.dir/src/main_3d.cpp.o] Error 1
CMakeFiles/Makefile2:2265: recipe for target 'pop_up_slam/pop_planar_slam/CMakeFiles/planar_test.dir/all' failed
make[1]: *** [pop_up_slam/pop_planar_slam/CMakeFiles/planar_test.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2
Invoking "make -j8" failed

It seems that the popup_plane not having some members function. And I don't have idea to fix it. Did you encounter any error like this?
I will looking for your reply, if you have any suggestion with this type of error.

Thank in advance.