Tello_ROS_ORBSLAM
In this project we provide a full and whole framework for using Drones in general, and the DJI Tello specifficaly.
In this project you will find a GUI that will allow you to control the Tello and command it to move in the x,y,z,pitch,roll,yaw plane.
Using this GUI will allow fast development of SLAM algorithms and integrate them with real Tello hardware.
The coordinates are derived from a pose that is published in one of the 2 slam algorithms (currently, orbslam and ccm slam, but you can easily add your own)
Inside the files, you will find a joystick/keyboard to control the tello from within ROS, instead of using your android phone.
ORBSLAM
In this video we can see a brief explenation about how the framework looks like when using the ORBSLAM. The Tello sends video stream to the ORBSLAM, the ORBSLAM provides position and orientation to the control, and the control controls the Tello to navigate it to the wanted location.
Tello UI (User Interface):
The Tello UI is conssited of multiple tools that allows you to control the Tello using a SLAM algorithm or using raw RPY (Roll, Pitch, Yaw).
Brief Explenation of the UI
Command Position Section
Takeoff, Land
Well, it goes without saying - this buttons will make the tello takeoff/land.
Calibrate Z!
Pressing this button will make the Tello elevate ~0.5 meters and then descent ~0.5 meters.
During this time the tello's internal height sensor will be sampled among with the height that is being published by the SLAM algorithm.
At the end of the movement the control calculates:
The control will be using that factor to go from the SLAM coordinate system to the real world coordinate system.
Publish Command!
Pushing this button will send the coordinates in the boxes (x, y, z, yaw) to the control, and the control will navigate the tello to those coordinates.
note that the control will control the Tello only if the "Toggle Slam Control!" Button is pressed!
Stay In Place!
This button will command the Tello to stay in the current place using the control.
Real World Position Section
In this section we can observe the tello's current coordinates in the real world coordinates. The Altitude Scale is the factor that was calculated during the Calibrate Z! process.
Rotated SLAM Coordinates Section
In this section we can observe the tello's current coordinates in the SLAM coordinates (after rotating the coordinate system to compensate for the angle of the camera of the Tello).
Delta Between Command and Real World Section
In this section we can observe the difference between the Tello's current pose, to the desired pose we commanded.
Speed Section
In this section we can observe the current speed in all the axes (pitch, roll, throttle, yaw) of the Tello. Also, we can see some side information about the Tello, likt the Altitude and the remaining Battery[%].
Toggle Slam Control!
This is a protection button. Pressing this button will allow the control to take over the Tello's speed control.
Manual Control Section
In this section we can manually control the speeds of the Tello in all axes. Just insert the wanted speed (0-1) in each axis and press Manual Control Set! To stop press the Manual Control Clear! button.
Trajectory Control Section
This section allows to command the Tello to go to many points one after the other. You can either insert the coordinates manually in the boxes, or load the using Load File button. Eventually press the Publish Trajectory! button to start the trajectory.
note that the control will control the Tello only if the "Toggle Slam Control!" Button is pressed!
Toggle Use Merged Coordinates
This button is used in the ccmslam algorithm. After the two drones have merged a map, our modification to ccmslam algorithm allows to control the drones using the same coordinate system. In other words, after the map was merged, pressing this button will make the controller of each drone to use the same coordinate system.
Tello Viewers:
In the Viewer we can observe the video stream received from the SLAM algorithm, among with the cloud map in the X-Y plane.
Tello Client 0:
Tello Client 1:
Usage
orbslam2
roslaunch flock_driver orbslam2_with_cloud_map.launch
ccmslam
Server:
roslaunch ccmslam tello_Server.launch
Client0:
roslaunch ccmslam tello_Client0.launch
Client1:
roslaunch ccmslam tello_Client1.launch
Installation Guide
Installing ROS melodic
Following this page: http://wiki.ros.org/melodic/Installation/Ubuntu
Setup your computer to accept software from packages.ros.org. :
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
Set up your keys
sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
Installation
First, make sure your Debian package index is up-to-date:
sudo apt update
Desktop-Full Install: (Recommended) : ROS, rqt, rviz, robot-generic libraries, 2D/3D simulators and 2D/3D perception
sudo apt install ros-melodic-desktop-full
Initialize rosdep
Before you can use ROS, you will need to initialize rosdep. rosdep enables you to easily install system dependencies for source you want to compile and is required to run some core components in ROS.
sudo rosdep init
rosdep update
Environment setup
It's convenient if the ROS environment variables are automatically added to your bash session every time a new shell is launched:
echo "source /opt/ros/melodic/setup.bash" >> ~/.bashrc
source ~/.bashrc
Dependencies for building packages
Up to now you have installed what you need to run the core ROS packages. To create and manage your own ROS workspaces, there are various tools and requirements that are distributed separately. For example, rosinstall is a frequently used command-line tool that enables you to easily download many source trees for ROS packages with one command. To install this tool and other dependencies for building ROS packages, run:
sudo apt install python-rosinstall python-rosinstall-generator python-wstool build-essential
Install Prerequisites
Please install all the prerequisites, no matter which algorithm you want to use.
Easy Install Prerequisites
catking tools
First you must have the ROS repositories which contain the .deb for catkin_tools:
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu `lsb_release -sc` main" > /etc/apt/sources.list.d/ros-latest.list'
wget http://packages.ros.org/ros.key -O - | sudo apt-key add -
Once you have added that repository, run these commands to install catkin_tools:
sudo apt-get update
sudo apt-get install python-catkin-tools
Eigen3
Required by g2o. Download and install instructions can be found here. Otherwise Eigen can be installed as a binary with:
sudo apt install libeigen3-dev
ffmpeg
sudo apt install ffmpeg
Python catkin tools (probably already installed)
sudo apt-get install python-catkin-tools
Joystick drivers
Tested it only on melodic.
sudo apt install ros-melodic-joystick-drivers
Python PIL
sudo apt-get install python-imaging-tk
Github based Prerequisites
Pangolin (used in orbslam2)
Based on https://github.com/stevenlovegrove/Pangolin
cd ~/ROS/
git clone https://github.com/stevenlovegrove/Pangolin.git
sudo apt install libgl1-mesa-dev
sudo apt install libglew-dev
sudo apt-get install libxkbcommon-dev
cd Pangolin
mkdir build
cd build
cmake ..
cmake --build
h264decoder
Baed on https://github.com/DaWelter/h264decoder
cd ~/ROS/
git clone https://github.com/DaWelter/h264decoder.git
Inside h264decoder.cpp replace PIX_FMT_RGB24 with AV_PIX_FMT_RGB24
mkdir build
cd build
cmake ..
make
now copy it to python path
sudo cp ~/ROS/h264decoder/libh264decoder.so /usr/local/lib/python2.7/dist-packages
Installing Our Repository
Cloning Our repo from github
cd ~
mkdir ROS
cd ROS
git clone https://github.com/tau-adl/Tello_ROS_ORBSLAM.git
Installing our version of TelloPy
based on https://github.com/dji-sdk/Tello-Python and https://github.com/hanyazou/TelloPy
cd ~/ROS/Tello_ROS_ORBSLAM/TelloPy
sudo python setup.py install
Installing dependencies for ROS
cd ~/ROS/Tello_ROS_ORBSLAM/ROS/tello_catkin_ws/
sudo rosdep init
rosdep update
rosdep install --from-paths src --ignore-src -r -y
Installing orbslam2
based on https://github.com/appliedAI-Initiative/orb_slam_2_ros and https://github.com/rayvburn/ORB-SLAM2_ROS First - if using Melodic version of ROS, change the ~/ROS/Tello_ROS_ORBSLAM/ROS/tello_catkin_ws/src/orb_slam_2_ros/CMakeLists.txt To the CMakeLists_melodic.txt
Build the code:
cd ~/ROS/Tello_ROS_ORBSLAM/ROS/tello_catkin_ws/
catkin init
catkin clean
catkin build
If it doesn’t work, make sure you changed the makefile to the wanted version of ROS
Add the enviroment setup to bashrc
echo "source $PWD/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc
Installing ccm_slam
based on https://github.com/VIS4ROB-lab/ccm_slam
Compile DBoW2:
cd ~/ROS/Tello_ROS_ORBSLAM/ROS/ccmslam_ws/src/ccm_slam/cslam/thirdparty/DBoW2/
mkdir build
cd build
cmake ..
make -j8
Compile g2o:
cd ~/ROS/Tello_ROS_ORBSLAM/ROS/ccmslam_ws/src/ccm_slam/cslam/thirdparty/g2o
mkdir build
cd build
cmake --cmake-args -DG2O_U14=0 ..
make -j8
Unzip Vocabulary:
cd ~/ROS/Tello_ROS_ORBSLAM/ROS/ccmslam_ws/src/ccm_slam/cslam/conf
unzip ORBvoc.txt.zip
Build the code:
cd ~/ROS/Tello_ROS_ORBSLAM/ROS/ccmslam_ws/
source /opt/ros/melodic/setup.bash
catkin init
catkin config --extend /opt/ros/melodic
catkin build ccmslam --cmake-args -DG2O_U14=0 -DCMAKE_BUILD_TYPE=Release
If Gives error - ROS distro neither indigo nor kinetic - change the makefile, use CmakeFile_changed2.
Add the enviroment setup to bashrc
echo "source $PWD/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc
Notes
Flock
we have used code from https://github.com/clydemcqueen/flock