Comments (7)
Hi @hnsyzjianghan, RL can be really sensitive and finicky. I havn't tried the older version of gymfc with gz9 and ROS but if the PID example is working fine than that shouldnt cause any problems. The graph you are showing me isn't awful, its learning the step just not quite there. Some things to check/try,
- Make sure the world step size is 0.001 or smaller
- Make sure you train at least 3 independent trials with different seeds and take the best
- When doing the validation average at least 5 different steps to get a good idea of the progress. Usually I'll watch as checkpoints are created and validate them on the fly.
- Try some hyperparameter tuning, in our follow up neuroflight paper we got better results with step size of 1e-4, you can check that paper for all the hyperparameters we used. Also try hidden nodes of size 32.
- If from your validation, rewards are still increasing then increase the number of simulation steps to 50 million.
from gymfc.
Dear william,
Thank you very much for your answer and I'll try your advice. I am a novice with RL and gymfc, I still have others doubts:
- You suggest me that trials with different seeds, but it seems not used in your demo run_gymfc.py. I believe that your demo is written in imitation of /baselines/ppo1/run_humanoid. py. However, make_mujoco_env() is used to build the environment in run_humanoid.py, and therefore seed's parameter input, but gym.make() is used to build the environment in your demo, without seeds. The parameter input of seed. What's the difference between these two functions in your project? What is the effect of seed in your environment?
- In your neuroflight paper, I see the angular velocity error and delta angular velocity error are used in reward function. Do you use angular error (integral of angular velocity)? In gymfc version 0.1.0, it seems that the FDMPacket class in gazebo_env.py returns to the environment, among them unpacked [7:11] is like a quaternion form returning to the attitude angle of the aircraft. I tried to output them, but found that they were fixed to [1, 0, 0, 0]
- In your newly updated version of gymfc 0.2.0, I see that the explanation possibly removing Gym portions. Does this version support the gym environment? If so, how can I use the new version to train the openai baseline RL algorithm? Can you give me some cases to refer to?
Thank you again for your answer and look forward to your recovery.
from gymfc.
So that script is just a demo script for interfacing with gymfc using openai baselines, it isn't the scripts used for the manuscript. For your own work its going to involve you implementing your own script to interface with the gym.
The gym source for the neuroflight paper was never published, just the neuroflight firmware code.
gymfc2 is a huge implementation change, its now a general flight control tuning framework. gymfc2 core it self an openai gym any more however this is extremely easy to do as you just need to inherit the FlightControlEnv class and provide your own state and reward functions. Will be porting over the openai gym interfaces to the new architecture when I have time. An example of creating a subclass can be found here. Further details will be in my thesis which will be completed in the next couple weeks, and when I have time I'll add more to the readme and additional examples. In the mean time these are new motor models that I've ported over for gymfc2 https://github.com/wil3/gymfc-aircraft-plugins.
from gymfc.
Sorry, I went on a business trip the other day. Thank you for your reply. I will use your model to finish my project.
from gymfc.
Hi william,
I got another problem,when I run the Simulator,sometimes the gazebo report (especially when I use mpiexec python(mpi4py MPI) in the terminal to accelerate the simulation.)
Broken Quadcopter connection,count 1~5/5.
and when count=5,it will report
Broken Quadcopter connection, resetting motor control.
It looks like the connection between gazebo and Python is broken. I find this report comes from /gymfc/envs/assets/gazebo/plugins/QuadcopterWorldPlugin.cpp.
I want to know that does this problem affect the simulation results? And how can i avoid this problem.
from gymfc.
Hi @hnsyzjianghan,
What you are seeing is the error reported by parts of the Arducopter plugin that were used in the first version of GymFC. That is caused by a network communication issue where packets sent from python to gazebo get dropped. If you cant communicate with the quadcopter then yes it will affect the results because the quadcopter isn't receiving the commands. The version of GymFC you are using is outdated and no longer supported, I suggest migrating to GymFCv2 on the master branch which is far more stable with significant enhancements. However if you still want to use the Iris quadcopter model we need help migrating it over to GymFCv2.
from gymfc.
Okay, I'll change it in the future work, thank you.
from gymfc.
Related Issues (20)
- Ask for help: Attitude controller instead of angle_rate controller HOT 1
- Bug: there are some bugs in continous.py HOT 1
- error: Failed to compile Gazebo plugin HOT 6
- Failing to run the simulation HOT 1
- Failed to launch quadcopter HOT 2
- Aircraft SDF file at location examples doesnot exist.
- Failing to train without internet connection. HOT 1
- the empty world model HOT 1
- libcurl: (6) Could not resolve host: fuel.ignitionrobotics.org HOT 3
- Feature Request: migrate from gym to gymnasium HOT 1
- terminate called after throwing an instance of 'sad::runtime_error' HOT 1
- How does neuroflight deal with gravity? HOT 1
- PID tunning for pid_example. HOT 4
- iPython HOT 4
- AWS-robomaker compatibility HOT 2
- Gymfc v0.1.0 HOT 1
- Motor output usage larger than simulation HOT 1
- Protobuf exception when running tests/test_step_sim.py HOT 2
- help with running pid example HOT 4
- the ppo_baselines train problem HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gymfc.