Giter VIP home page Giter VIP logo

ppo-based-autonomous-navigation-for-quadcopters's Introduction

PPO-based Autonomous Navigation for Quadcopters

license

This repository contains an implementation of Proximal Policy Optimization (PPO) for autonomous navigation in a corridor environment with a quadcopter. There are blocks having circular opening for the drone to go through for each 4 meters. The expectation is that the agent navigates through these openings without colliding with blocks. This project currently runs only on Windows since Unreal environments were packaged for Windows.

🛠️ Libraries & Tools

Overview

The training environment has 9 sections with different textures and hole positions. The agent starts at these sections randomly. The starting point of the agent is also random within a specific region in the yz-plane.

Observation Space

  • State is in the form of a RGB image taken by the front camera of the agent.
  • Image shape: 50 x 50 x 3

Action Space

  • There are 9 discrete actions.

Environment setup to run the codes

#️⃣ 1. Clone the repository

git clone https://github.com/bilalkabas/PPO-based-Autonomous-Navigation-for-Quadcopters

#️⃣ 2. From Anaconda command prompt, create a new conda environment

I recommend you to use Anaconda or Miniconda to create a virtual environment.

conda create -n ppo_drone python==3.8

#️⃣ 3. Install required libraries

Inside the main directory of the repo

conda activate ppo_drone
pip install -r requirements.txt

#️⃣ 4. (Optional) Install Pytorch for GPU

You must have a CUDA supported NVIDIA GPU.

Details for installation

For this project, I used CUDA 11.0 and the following conda installation command to install Pytorch:

conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0 -c pytorch

#️⃣ 4. Edit settings.json

Content of the settings.json should be as below:

The setting.json file is located at Documents\AirSim folder.

{
    "SettingsVersion": 1.2,
    "LocalHostIp": "127.0.0.1",
    "SimMode": "Multirotor",
    "ClockSpeed": 20,
    "ViewMode": "SpringArmChase",
    "Vehicles": {
        "drone0": {
            "VehicleType": "SimpleFlight",
            "X": 0.0,
            "Y": 0.0,
            "Z": 0.0,
            "Yaw": 0.0
        }
    },
    "CameraDefaults": {
        "CaptureSettings": [
            {
                "ImageType": 0,
                "Width": 50,
                "Height": 50,
                "FOV_Degrees": 120
            }
        ]
    }
  }

How to run the training?

Make sure you followed the instructions above to setup the environment.

#️⃣ 1. Download the training environment

Go to the releases and download TrainEnv.zip. After downloading completed, extract it.

#️⃣ 2. Now, you can open up environment's executable file and start the training

So, inside the repository

python main.py

How to run the pretrained model?

Make sure you followed the instructions above to setup the environment. To speed up the training, the simulation runs at 20x speed. You may consider to change the "ClockSpeed" parameter in settings.json to 1.

#️⃣ 1. Download the test environment

Go to the releases and download TestEnv.zip. After downloading completed, extract it.

#️⃣ 2. Now, you can open up environment's executable file and run the trained model

So, inside the repository

python policy_run.py

Training results

The trained model in saved_policy folder was trained for 280k steps.

Picture2

Test results

The test environment has different textures and hole positions than that of the training environment. For 100 episodes, the trained model is able to travel 17.5 m on average and passes through 4 holes on average without any collision. The agent can pass through at most 9 holes in test environment without any collision.

Author

License

This project is licensed under the GNU Affero General Public License.

ppo-based-autonomous-navigation-for-quadcopters's People

Contributors

bilalkabas avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

ppo-based-autonomous-navigation-for-quadcopters's Issues

Unreal Engine Environment

Thank you very much for providing the UE environment, but when we downloaded and opened it, it turned out to be a packaged UE engine. May I ask if you can upload an unpackaged environment (i.e. a folder with .uproject files)? We would like to know the coordinate design of the hole, which is a crucial issue. Thank you for your work and time!

env page not found

hello i want to learn this model and env. but, your env release page wasn't found. can i get some env file?

TestEnv

Do we need to rewrite the env_config ["TrainEnV"] in the policy run. py file we run during testing to env_config ["TestEnV"]? I didn't see TestEnv in config.yml, do I need to redefine it myself? Do I need to correspond to the holes in the UE environment in TestEnv.zip?

A warning I met during I perform "python policy_run.py"

I have followed each step as suggested by the readme. However, I encounter the problem as follow:

WARNING:tornado.general:Connect error on fd 336: WSAECONNREFUSED
WARNING:tornado.general:Connect error on fd 336: WSAECONNREFUSED
WARNING:tornado.general:Connect error on fd 336: WSAECONNREFUSED
WARNING:tornado.general:Connect error on fd 336: WSAECONNREFUSED
WARNING:tornado.general:Connect error on fd 336: WSAECONNREFUSED
Traceback (most recent call last):
File "policy_run.py", line 14, in
env = DummyVecEnv([lambda: Monitor(
File "E:\Anaconda\envs\PPO_drone\lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py", line 25, in init
self.envs = [fn() for fn in env_fns]
File "E:\Anaconda\envs\PPO_drone\lib\site-packages\stable_baselines3\common\vec_env\dummy_vec_env.py", line 25, in
self.envs = [fn() for fn in env_fns]
File "policy_run.py", line 15, in
gym.make(
File "E:\Anaconda\envs\PPO_drone\lib\site-packages\gym\envs\registration.py", line 235, in make
return registry.make(id, **kwargs)
File "E:\Anaconda\envs\PPO_drone\lib\site-packages\gym\envs\registration.py", line 129, in make
env = spec.make(kwargs)
File "E:\Anaconda\envs\PPO_drone\lib\site-packages\gym\envs\registration.py", line 90, in make
env = cls(
_kwargs)
File "E:\Project\PPO_based_ANfQ\PPO-based-Autonomous-Navigation-for-Quadcopters\scripts\airsim_env.py", line 169, in init
super(TestEnv, self).init(ip_address, image_shape, env_config)
File "E:\Project\PPO_based_ANfQ\PPO-based-Autonomous-Navigation-for-Quadcopters\scripts\airsim_env.py", line 19, in init
self.setup_flight()
File "E:\Project\PPO_based_ANfQ\PPO-based-Autonomous-Navigation-for-Quadcopters\scripts\airsim_env.py", line 174, in setup_flight
super(TestEnv, self).setup_flight()
File "E:\Project\PPO_based_ANfQ\PPO-based-Autonomous-Navigation-for-Quadcopters\scripts\airsim_env.py", line 36, in setup_flight
self.drone.reset()
File "E:\Project\PPO_based_ANfQ\PPO-based-Autonomous-Navigation-for-Quadcopters\scripts\airsim\client.py", line 26, in reset
self.client.call('reset')
File "E:\Anaconda\envs\PPO_drone\lib\site-packages\msgpackrpc\session.py", line 41, in call
return self.send_request(method, args).get()
File "E:\Anaconda\envs\PPO_drone\lib\site-packages\msgpackrpc\future.py", line 43, in get
raise self._error
msgpackrpc.error.TransportError: Retry connection over the limit

I would be grateful if anyone could tell me how to fix this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.