vis4rob-lab / ccm_slam Goto Github PK

Hi,
I'm interested in Multi-robot SLAM system,and your repo is very helpful to me.Thank you very much.

I did some changes,and I get my own KF trajectory.But I don't know how to compute the error between my KF trajectory and Euroc MAV groundtruth when I set two agent,could you please give me some guidance?

Thanks a lot.

A slight error in the config files resulting in the output trajectories not being written correctly.

From #11 it is apparent that the T_SC matrix in the config.yaml files should be set to the identity matrix if there is no conversion between the camera and sensor frame.

The T_SC matrix in the kitti.yaml files is not correctly set resulting in the following matrix:

[1 0 0 0]
[0 1 0 0]
[0 0 0 1]
[0 0 0 1]

As this matrix has no inverse, the resulting trajectory files (KF_GBA.csv) will contain only nan values.

Fix:
#T_SC
Cam0.T00: 1.0
Cam0.T01: 0.0
Cam0.T02: 0.0
Cam0.T03: 0.0
Cam0.T10: 0.0
Cam0.T11: 1.0
Cam0.T12: 0.0
Cam0.T13: 0.0
Cam0.T20: 0.0
Cam0.T21: 0.0
Cam0.T22: 1.0
Cam0.T23: 0.0
Cam0.T30: 0.0
Cam0.T31: 0.0
Cam0.T32: 0.0
Cam0.T33: 1.0

We are told to run

roslaunch ccmslam Client0_kitti_half_res.launch

But this just waits on this message

[ INFO] [1574114392.427121734]: Started CSLAM client node...

What exactly are we supposed to do?

Where do I actually input the paths of the KITTI dataset?

Hi! @patriksc

I've been trying to change ccm-slam to rgbd version, I changed some codes according to ORB2-SLAM.

I think I changed it right. But now a strange thing happend, I can successfully run this rgbd version on the terminal side, however I have nothing visualized on the Rviz side.(As shown below)

I don't know why is this hapening.

Can you please help me on this, thank you so much! Really looking forward to your relpy!

Hey,

I ran two clients one after the other, using the same dataset(MH_01_easy). After the two maps on the server are merged, a lot of loops will be generated. As a result, the pose of the keyframes will be greatly disordered after the loop optimization. What could be the problem? Is there a solution?

Thank,
LinYayun

The launch file for the EuroC Dataset includes the following static transformation-
<node pkg="tf" type="static_transform_publisher" name="linkS0_broadcaster" args="0 0 5 -1.571 0 -2 world odomS0 100" />

What is the reason for keeping the Roll as -2? Shouldn't it also be -1.571 since the camera is front-facing in the dataset? I am assuming that this will hardly affect the result unless comparing to the ground truth which is collected in the world frame. (for a custom dataset)

@patriksc

hi！I test ccm on kitti and I noticed that when dataset run for 3/4 it need memory about 9g while orb-slam2 only cost 3g.I have no idea why it need so much memory(It should not be related to the number of KF ,because it has almost the same number of KF as orb-slam2).And it need more than 20g when I change it to stereo-version which is to large for my computer.
Is there any reason may result in this?Looking forward to your reply!

I noticed that you use an additional module,which you called "communication controller", in your paper CCM‐SLAM ,to measure the communication bandwidth for different ploss and delay settings. How did you implement this part, or is it included in your open source code? thanks!

First thanks for your excellent work. I run Server backend on my own desktop to merge three agents trajectory on EuRoC dataset. However, sometimes the graph is distorted. Maybe the loop closure edge is wrong? The RMSE ATE is around 0.20 for MH_01 and MH_02, respectively.

I am trying to perform 3D reconstruction using the depth maps of the TUM datasets in conjunction with the poses of CCMSLAM.
I know that the camera poses of CCMSLAM are not scaled correctly but I take care of this separately.

Now to perform this reconstruction correctly from my understanding I would need the Twc (camera pose in world reference) to transform the point clouds resulting from the depth maps. But how would I get the camera pose so that it aligns correctly with the typical xyz coordinate frame?

Currently, the poses are coherent to a somewhat twisted coordinate frame and then transformed by this static transformer:
#44

I found this issue in the ORBSLAM2 repo describing how to correctly visualize the poses:
raulmur/ORB_SLAM2#832

While the approach from the ORBSLAM2 repo seems to help the poses are still not fully aligned.

Could someone give a hint about the transformation that is needed or some explanation on the transformation between the ORB frame and world frame?

Hi,
My environment is ubuntu 18.04 with ros-melodic.When i try to compile use catkin build,it always crash at 58%.I don't know what happened.I have tried several times.

hi!
I use d435i camera to test ccm-slam on one Xavier NX.
And I found it's easy to get traking lost if I move d435i a little bit fast.
I want to know if there is any reason to lead to this problem?
Is this related to fps ?(my d435i's fps is 30)
Thank you very much!

hi,
Nowadays, we are trying to build multi-robot SLAM system.
Our usage is based on RGBD sensor, which is different from ccm_slam(mono slam).
Do you think is it easy to convert CCM SLAM from monoslam to RGBD slam?

Thanks.
I really appreciate your affort.

Hello, i have a question: "how can i use your ccm slam for collaborative work of uav and ugv?"
Does it matter to use only UAV, like in your work, or not?
Thanks for your answer

Hi, I am interested in acquiring the commercial version of CCM-SLAM so I sent an inquiry to the email listed in the License section of the ReadMe. However the email server cannot find the address. I verified that I was writing to the correct email with the papers listed. I tried with 2 different domains, one through Outlook and one through Gmail and got the same error code both times. Is there an alternate address I can contact to inquire about the commercial version? Thanks.

hi!
I test euroc dataset on one PC and I play rosbag at a half speed.
And I got result like this.
I find that if I play only two dataset at the same time,I got a map with two diffenrent color.

But If I play four rosbags at the same time,I got a point cloud map with only white point and MarkerC0,MarkerC1,MarkerC2 and MarkerC3 got an error like below.Is this necessary to fix?

And I got an error like Tracking: lost in client2.
Could you tell me how to fix it?
Thank you so much!

I am trying to understand why my trajectory evaluation results differ significantly from the ones reported in the publication.

I am using the evaluation procedure as outlined in the readme (i.e. using evo_ape with KF_GBA_0.csv and gt.csv)

evo_ape output from EuRoc MH01:
max 0.352780
mean 0.154116
median 0.133981
min 0.026507
rmse 0.180358
sse 12.621255
std 0.093687

How come these results differ so much from the trajectory error reported in the publication (0.061 RMSE [m])?

Edit:
When adding code to save the frame trajectory (tracking thread) the trajectory error aligns with the results of the publication. This still leaves the question of why the KF trajectories result in a larger errors.

Hello,
I built this project successfully. But when I try to launch the server I get the error below:

This error does not tell anything much about the issue. I don't understand where to look to debug it.
I am using Ubuntu 18.04 and the processor is AMD Ryzen 7. I have 12 GBs of memory.

Hii

It is mentioned in the JFR publication that you are utilizing the Essential graph as done in ORB SLAM implementation for quicker GBA and for losing the redundant edges. I interpreted that changing the Opt.EssGraphMinFeats will change the criteria for building the Essential Graph from the Covisibilty Graph. The default value is 100. I noticed a problem with this parameter. It seems that it is not being used.

An experiment was performed with 2 agents for 2 different runs and it was observed as follows-

Run 1. Opt.EssGraphMinFeats = 100: Here it was kept as default value. As attached in the screenshot "GBA_100.png", the no. of Keyframes (Vertices) were 617 and the no. of edges were pretty high = 201005. The no. of edges were relatively very high for 617 KFs and thus the optimization time was a whopping 57.188s.

Run 2. Opt.EssGraphMinFeats = 200: I increased the value to 200. As seen in the attached screenshot "GBA_200_rviz.png", a very few grey lines (Viewer.CovGraphMinFeats was also set to 200 which represent the grey lines) can be seen. But when we see the screenshot "GBA_200.png", the no. of Keyframes were 590 and the no. of edges were still high = 192063. Thus, the optimization time was again very high.

Thus, I feel that there is some bug which is not causing the correct building of essential graph and thus it is not utilizing the Essential graph for GBA.

It would be great if you can look into this issue.
@patriksc

Attachments-

1. GBA_100.png
   

2. GBA_200.png
   

3. GBA_200_rviz.png
   

Hi,
I am getting below error when I run the server

[ccmslam/ccmslamServerNode-6] process has died [pid 18211, exit code -11, cmd /home/arindam/ccmslam_ws/devel/lib/ccmslam/ccmslamServerNode /home/arindam/ccmslam_ws/src/ccm_slam/cslam/conf/ORBvoc.txt __name:=ccmslamServerNode __log:=/home/arindam/.ros/log/6456d802-8699-11e9-a8c0-283a4d456d6f/ccmslam-ccmslamServerNode-6.log].
log file: /home/arindam/.ros/log/6456d802-8699-11e9-a8c0-283a4d456d6f/ccmslam-ccmslamServerNode-6*.log

And getting below error in case of client

[ccmslam/ccmslamClientNode0-2] process has died [pid 14051, exit code -11, cmd /home/arindam/ccmslam_ws/devel/lib/ccmslam/ccmslamClientNode /home/arindam/ccmslam_ws/src/ccm_slam/cslam/conf/ORBvoc.txt /home/arindam/ccmslam_ws/src/ccm_slam/cslam/conf/vi_euroc.yaml __name:=ccmslamClientNode0 __log:=/home/arindam/.ros/log/b6f85bfa-868d-11e9-a8c0-283a4d456d6f/ccmslam-ccmslamClientNode0-2.log].
log file: /home/arindam/.ros/log/b6f85bfa-868d-11e9-a8c0-283a4d456d6f/ccmslam-ccmslamClientNode0-2*.log

Is this issue with ccm-slam code or ROS problem? Can you please help me on this?

Hi,
I really thank you for your reply.
I have one more question.

ccm slam can save map on server?
and can we load map on client?

I want to make a system which can save map on server and load map on client.

I reallly thank you for your reply.
thanks.

I would like to add the number of agents to 8 to compare CCM-SLAM with our methods, but I found it was difficult to do this. Is there a close source edition CCM-SLAM that support 8 or more Agent?
By the way, I noticed your latest work on COVINS, but it need IMU information and I need a monocular algorithm.
Looking forward your reply, Thanks!

Hey,
What are the #T_SC matrix parameters stand for?
They do not exist in the method of calibration I use, not in the original orb slam implementation.

Hello @mchli , @patriksc

Is there any way of knowing which set of keypoints correspond to which image frame?

According to my understanding, some of the image frames are treated as keyframes, and from these keyframes, the keypoints are being generated.

I am working on an image stitching (panorama building) problem. For this purpose, I want to get keypoints and descriptors from your repository and stitch images based on these keypoints and descriptors.
At the moment, I am subscribing to topic '/ccmslam/MapOutClient0' and extracting. This topic has type Map (your builtin) message. From this message when I extract:
your_map_message.Keyframes[0].mnId,
the mnIds I get apparently do not correspond to keyframes sent.
So, if mnids = 13, this does not correspond to image 13.

In short, If let's say I extract all the keypoints from the message 'map.msg', how can I know which set of keypoints correspond to which image frame? Can you please tell me how do I extract this information?

Also, inside 'map.msg' sometimes the 'KF keyframe' array comes out to be empty? What does this 'emptiness' indicate?

when i run 'roslaunch ccmslam Server.launch' , the error is:

[ccmslam/ccmslamServerNode-6] process has died [pid 24549, exit code -11, cmd /home/swj/ccmslam_ws/devel/lib/ccmslam/ccmslamServerNode /home/swj/ccmslam_ws/src/ccm_slam/cslam/conf/ORBvoc.txt __name:=ccmslamServerNode __log:=/home/swj/.ros/log/ca1aa532-2f4d-11ed-be68-38f3ab349edb/ccmslam-ccmslamServerNode-6.log].
log file: /home/swj/.ros/log/ca1aa532-2f4d-11ed-be68-38f3ab349edb/ccmslam-ccmslamServerNode-6*.log

Hi

I am facing the following errors while running the devel branch code

When I use more than one agent, which have a lot similar trajectory as seen in the attached picture, the server is unable to close the loop after some loop closures and stops to process (the htop command shows very low load and no thread is above 20-30%) with the last print screen message being -

Map: GBA finished - continue
!!! LOOP FOUND !!!
--- Lock threads
--- transform KFs/MPs
--- update matched MPs
--- optimize Ess grapgh
--- Launch GBA thread
LoopFinder: Wait for GBA to finish
-> Starting Global Bundle Adjustment
--> Optimizing Map 0 -- Contains Agents 0;1;
----> Loop KF: 235|1
----- Add KFs
----- Add MPs
----- Optimize
------> max Iterations: 20
------> vertices|edges: 1159|307065
------> Optimization Time: 162.947 [s]
----- Recover KFs
----- Recover MPs
----- done.

The whole terminal output has been attached in the attached txt file. When this occurs, the clients start having the print screen message as follows-

+++++ Enforcing KF upper limit +++++1.59585e+09
+++++ Enforcing KF upper limit +++++1.59585e+09

This problem is not faced while running the same bag files in the master branch (though the problem is that each of the loop closure is followed by the GBA instead of interruption which takes a lot of time)

Moreover the Essential Graph also doesn't seem to work as mentioned in #25 which is causing a lot of time for optimization(going up to several minutes(10+) for high KFs)

Rviz - visualization of the 2 trajectories
#Note - There is a very high ATE since the final GBA are not performed and the server stops the processing

Full Server print screen txt-
server_print.txt

Hi,
I am very impressed in ccm-slam. I have a question: in my point of view, clients do local mapping and then send local keyframes and key local mappoints to server. The server doesn't need to do local mapping. But in your system, local mapping is separately executed in client and server. I am very curious about the reason.
Thanks for your attention. I am looking forward to your response!
Best regards,
Ouyang

hi!
I test ccm-slam in realworld,and I found a problem.
When I move camera to a new scene,I found sometime it can not extracting feature points.I will track lost if I continue to move camera.
Is this related to nFeatures in configure.yaml?or other reasons?
thank you for your help!

(ps:in the left side of picture,it can extract feature points,because it's the scene it has experienced.I move camera from left to right)

Hi,

Have you tested ccm_slam running with Ubuntu18.04(ROS Melodic)? which parts should be modified(like CMakeLists.txt)?

Thanks!

Hello, I have successfully run the program, and I am more concerned about the closed-loop module between multiple machines. I found at runtime that the system always performs MapMatch first, and then MapMerge. If there are two clients, it will only be performed once, and if there are three clients, it will only be performed twice. Isn't the closed loop between multiple agents being detected all the time? , it should be more than that little.
From the run log it looks like all agents only did one MapMatch.
@mchli @patriksc
Hope to get your help！

Hi, when I ran this code on the kitti dataset, it always failed to initialize. And I had adjust conf/vi_euroc.yaml correctly.

when using docker ros:melodic-ros-base, I try to compile the ccmslam project with "catkin build ccmslam --cmake-args -DG2O_U14=0 -DCMAKE_BUILD_TYPE=Release", then there is an error "c++: internal compiler error: Killed(program cc1plus)".
is any body try to compile this project in docker?

update date: 20/7/9: I use jetson nano to complie this project, I tried three method.

1. install ros directly in jetson nano, and compile failed. (stuck in 17%）
2. install ros in ros:melodic-ros-base docker at jetson nano, and compile failed. (c++ internal compiler error)
3. install ros in ubuntu:18.04 docker and install ros in this docker at jetson nano, and compile failed. (stuck in17%)

many other errors have fixed already, but I really don't know how to fixed this situation above.
I guess arm64 is uncompatiable for ccm-slam project, so who have compiled successfully in jetson nano, Can anybody help me?

Is there any function hided in open source version?

hi!I noticed in your paper the number of KFs in local map is set to 50 and parameters k (number of KFs sent from server to agent to augment local map) is set to 5.
However,in config.yaml ,I noticed that Comm.Server.KfsToClient is set to 0.
Should this parameter set to 5?
Looking forward to you reply!

hi!
I notice that there is a static transform publisher parameter. I think when map merges,we can get every agent pose which means these is no need to know their initial pose set by static transform publisher .
Whether this parameter is only for visualization in rviz?And when map merge,this parameter become useless?
thank you for your help~

Hi,

I tried enabling the network disturbance via the communication controller (mentioned Sec 5.3) however it doesn't seem to be included in the repository?

I changed the line in the server launch file (src/ccm_slam/cslam/launch/Server.launch) to:

<arg name="dist" default="1"/>

However it appears there's no such node:

ERROR: cannot launch node of type [ccmslam/ccmslamCommTestDisturber]: Cannot locate node of type [ccmslamCommTestDisturber] in package [ccmslam]. Make sure file exists in package path and permission is set to executable (chmod +x)

And grep'ing ccmslamCommTestDisturber does not return anything, making me wonder if this part is simply not in the repository at all?

Thanks!

Hello, I have run this project on the Euroc dataset. First of all, I would like to express my praise and admiration for your work. It is awesome! Then I have a few questions to ask:

First, the keyframes and mappoints output by this project code are binary files, can they be directly loaded into the system afterwards?

Second, are the KF_GBA_0.txt, KF_GBA_1.txt, KF_GBA_2.txt, and KF_GBA_3.txt output by the system the result of the server's global optimization of the pose? When I used the evo evaluation tool to analyze the accuracy, I found that the RMSE is generally above 0.25m, which is seriously inconsistent with the results in the paper, and is more like the results obtained by the agent's real-time tracking.

Looking forward to your reply, thank you very much！

Hello,
I wanted to test the client ability to navigate in the map held by the server.
For that I created a small experiment in which I fly two drone clients in different parts of the room, and make them merge on a certain area. After that I wanted to test the ability of one client to navigate in the regions mapped by the other client.

I add some modifications in the code that when the merge occurs, the client stops his local mapping logic. I looked in the code and read the article, and it seems as if the client should receive MPs and KFs from the server, that are locally connected to his current one, so I though that might be a nice expirement to show that the server map is really usefull for "blind tracking".

However, my client didn't seem to learn new KFs and MPs from the server (although, by judging the communication logic in the code, it should receive the locally connected MPs and KFs and the messages), and quickly lost tracking without the local mapping actually gaining the new KFs from the Tracking thread.

My question is, do you think the experiment should work? should one client, after being merged in the server, be able to "get" KFs and MPs from the server and using them perform tracking?
If not, how do you recomment to test the usefullness of the data in the server after the merge?

Thank,
Matan

Hello, I would like to ask how the coordinate system of CCM-SLAM is determined. Is the pose information of all agents based on the first agent coordinate system? Then merge the maps by closing the loop? Is my understanding right? Looking forward to your reply.

Hello.
I was trying to execute ccm slam on my local machines. I was successfully able to connect client with server. I made all necessary updates required to use my pc's usb cam with the code. No when I launch my usb cam it doesn't show any thing.

Kindly guide me in solving this issue.

These are my client and server in running state:

Then as soon as I open my web cam I get this issue:

This is my client file:

@patriksc please if you can help.

I would like to measure the running time of the whole system. What should I do, please !

After seeing #15, I did some tests, and running something such as:

rosbag play MH_01_easy.bag --start 55
rosbag play MH_01_easy.bag --start 45 /cam0/image_raw:=/cam0/image_raw1

Consistently crashes on agent 1 (i.e. the one who starts earlier in the sequence, and thence re-visits what agent 0 has seen)

The crashes occur in g2o because of inconsistencies in the graph, but there are at least two places that create such inconsistencies

1. src/MapPoint.cpp:263
   I don't know where the crash occurs exactly here (has to be in src/Optimizer.cpp I imagine)

+++++ Enforcing KF upper limit +++++1.5904e+09
+++++ Enforcing KF upper limit +++++1.5904e+09
+++++ Enforcing KF upper limit +++++1.5904e+09
!!!!! FATAL ERROR !!!!! EstablishInitialConnectionsClient263Constructing MP & is already associated to KF
KF-ID: 56|1 --- this MP-ID: 8168|0 --- comp. MP-ID: 8168|0
(pMP = shared_from_this()): 0
!!!!! FATAL ERROR !!!!! EstablishInitialConnectionsClient263Constructing MP & is already associated to KF
KF-ID: 54|1 --- this MP-ID: 8168|0 --- comp. MP-ID: 8168|0
(pMP = shared_from_this()): 0
!!!!! FATAL ERROR !!!!! EstablishInitialConnectionsClient263Constructing MP & is already associated to KF
KF-ID: 55|1 --- this MP-ID: 7753|0 --- comp. MP-ID: 7753|0
(pMP = shared_from_this()): 0
!!!!! FATAL ERROR !!!!! EstablishInitialConnectionsClient263Constructing MP & is already associated to KF
KF-ID: 55|1 --- this MP-ID: 7968|0 --- comp. MP-ID: 7968|0
(pMP = shared_from_this()): 0
!!!!! FATAL ERROR !!!!! EstablishInitialConnectionsClient263Constructing MP & is already associated to KF
KF-ID: 54|1 --- this MP-ID: 7968|0 --- comp. MP-ID: 7968|0
(pMP = shared_from_this()): 0
!!!!! FATAL ERROR !!!!! EstablishInitialConnectionsClient263Constructing MP & is already associated to KF
KF-ID: 54|1 --- this MP-ID: 7781|0 --- comp. MP-ID: 7781|0
(pMP = shared_from_this()): 0
+++++ Enforcing KF upper limit +++++1.5904e+09
addVertex: FATAL, a vertex with ID 48 has already been registered with this graph
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
[ccmslam/ccmslamClientNode1-2] process has died [pid 15723, exit code -6, cmd /ccmslam_ws/devel/lib/ccmslam/ccmslamClientNode /ccmslam_ws/src/ccm_slam/cslam/conf/ORBvoc.txt /ccmslam_ws/src/ccm_slam/cslam/conf/vi_euroc.yaml __name:=ccmslamClientNode1 __log:=/ros/log/63b491ee-9e69-11ea-aa72-1c1b0da2ba18/ccmslam-ccmslamClientNode1-2.log].
log file: /ros/log/63b491ee-9e69-11ea-aa72-1c1b0da2ba18/ccmslam-ccmslamClientNode1-2*.log

2. In other cases, g2o throws a std::bad_alloc without the FATAL ERROR shown above. In this case, the crash occurs at src/Optimizer.cpp:423 (addVertex: FATAL, a vertex with ID 63 has already been registered with this graph).

I am trying to understand the values used in the static transformers to adjust them to my own case.

From #21 I understand that the roll is set deliberately higher (2 > 1.571) to account for a slightly downward-facing camera. But why is the yaw and roll in general rotated by 90deg?

In the "Using your own Data" section example of the readme, such a rotation is left out. So I would interpret that I would mainly need to account for the camera angle (roll) when using my own data?

Hi,
I am trying to convert ccm slam to work with stereo camera. I have done all (i think so) necessary changes to work for stereo camera and I was testing on EuRoC data set, but I am getting frequently the below error message to the client terminal.

!!!!! FATAL ERROR !!!!! EstablishInitialConnectionsClient309Constructing MP & is already associated to KF
KF-ID: 105|0 --- this MP-ID: 6468|0 --- comp. MP-ID: 6468|0
(pMP = shared_from_this()): 0

Can you please provide me the probable reasons for getting such error.

Another problem I am also facing is that the client gets track lost very frequently. Is there any specific reason for such case?

Thanks,
Arindam

Hi, I have followed the steps and it's OK to perform both client and server on one PC. After I use two PCs and execute the ROS_MASTER_URI in terminal, client performs well. But server cannot show the map. Do I miss something? I'm sure that two PCs connect to the same wifi.
Thanks for your help!

Hello. I wanted to mention an error we were encountering while installing ccm slam on rasberry pi 3b using ubuntu 18. We have been trying continuously but every time this issue occurs and disrupts the installation process.

Kindly guide me on how we can solve this problem.