Giter VIP home page Giter VIP logo

data-driven-dynamics's Introduction

data-driven-dynamics

SysID Pipeline Test

A dynamics model is a valuable tool for a variety of applications, including simulations and control. However, traditional methods for obtaining such models for an unmanned aerial vehicle (UAV), such as wind tunnel testing or the installation of additional sensors, can be time-consuming, inaccesible and costly. In this project, we aim to address this challeng through a data-driven dynamics modelling pipeline that uses the PX4 unified flight log (ULog) format. The pipeline can be used for obtaining a dynamics model solely by minimizing the prediciton error of the model with respect to the flight data collected by the by default on the vehicle installed sensors. The modular approach of the pipeline aims to accomodate different model structures, optimizers and various UAV airframes, including multirotors and vertical take-off and landing (VTOL) UAVs. Specifically, for the example of multirotors, the pipeline automates the joint estimation of a parametric model and the quality of the fit and integrates the former into the PX4 Gazebo flight simulator.

You can get an overview of the pipelines functionalities on the project description website or in the following video presentation:

Watch the video

More detailed information can be found in the student paper Data-Driven Dynamics Modelling Using Flight Logs.

Setting up the environment

This plugin depends on PX4 SITL. Therefore custom messages of PX4 SITL needs to be linked. Therefore, prior to building this package, PX4 and corresponding SITL package needs to be built.

In case you have not cloned the firmware repository

git clone --recursive https://github.com/PX4/PX4-Autopilot.git ~/src/PX4-Autopilot

For internal use, our internal firmware repository is as the following.

git clone --recursive https://github.com/ethz-asl/ethzasl_fw_px4.git ~/src/PX4-Autopilot
cd <Firmware Dir>
DONT_RUN=1 make px4_sitl gazebo

The build directory of PX4 can be linked by setting the environment variable PX4_ROOT to the root of the firmware directory. Set the environment variable export PX4_ROOT=~/src/PX4-Autopilot or add it to your bashrc

echo "export PX4_ROOT=~/src/PX4-Autopilot" >> ~/.bashrc
source ~/.bashrc

The use the parametric model structure you need to install python 3.8 and the needed python libraries. It is strongly advised to install the pip packages in a virtual enviroment setup for this project.

Install the dependencies using:

make install-depdencies

Build

After the environment has been setup as described in the previous section, build the package as the following.

mkdir build
cd build
cmake ..
make

Generating a Parametric Model from Log File

Link the latest log files to your local logs folder using:

source setup.bash

Generate the parametric model using a log file (ulog or csv):

make estimate-model [model=<modeltype>] [config=<config_file_path>] [data_selection=<none|interactive|setpoint|auto>] [selection_var=topic_name/variable_name] [plot=<True/False>] [log=<log_file_path>]

Pipeline Arguments

Model Choice

The chosen vehicle model class determines what physical effects are modelled and what parameterts need to be regressed in the system identification process. Current vehicle model choices are:

  • quadrotor_model (default config for quadrotor)
  • fixedwing_model (default config for cruise flight of fixed-wings and VTOLs, documented here)

Config File

The config file allows to configure the intra class vehicle variations, used log file topics, data processing and other aspects of the pipeline. The default location is in Tools/parametric_model/configs. The path can be passed in the make target through the config=<config_file_path> argument. If no config is specified the default model config is used.

Log File

The Log file contains all data needed for the system identification of the specified model as defined in its config file. Next to the ULog file format it is also possible to provide the data as a csv file. An example of the required formating can be seen in the resources folder.

Data Selection

The data_selection argument is optional (per default none) and can be used to visually select subportions of the data.

  • none(default): Data selection is disabled, and the whole section of the log is used
  • interactive: Data is selected interactively using the Visual Dataframe Selector, before running the model estimation. It is also possible to save the selected subportion of data to a csv file in order to use this exact dataset multiple times.
  • setpoint: Data is selected based on a certain topic value, which has to be specified with the variable selection_var through the command line. The variable has to be provided in the format topic_name/variable_name to be recognized and loaded correctly. For example, to select data based on the aux1 value of the manual_control_setpoint topic, the variable manual_control_setpoint/aux1 has to be specified.
  • auto: Data is selected automatically (Beta)

Results

The resulting parameters of the model estimation together with additional report information will be saved into the model_results folder as a yaml file.

Getting Started

As an example to get started you estimate the parameters of a quadrotor model with the reference log_files:

make estimate-model model=quadrotor_model log=resources/quadrotor_model.ulg

Generating a Model Prediction for Given Parameters and Log

It is also possible to test the obtained parameters for a certain model on a different log using:

make predict-model [model=<modeltype>] [config=<config_file_path>] [data_selection=<none|interactive|auto>] [log=<log_file_path>] [model_results=<model_results_path>]

Testing the functionality of Parametric model

To ensure that the parametric model works as expected you can perform a set of pytests, which are stored in Tools/parametric_model/tests. To start the tests you have to run the shell script:

Tools/parametric_model/test_parametric_model.sh

Currently only the transformation from body to intertial frame and vise versa are checked. This should be expanded in the future.

Running the Simulation

To run the simulation,

source setup.bash
Tools/sitl_run.sh -m iris -s iris_aerodynamics

The custom Gazebo quadrotor model will always read the model parameters from the file model_results/quadrotor_model.yaml. You can simply rename your desired model results file to fly your estimated model in Gazebo.

Credits

This project was done in collaboration between the Autonomous Systems Lab, ETH Zurich and Auterion AG

To cite this work in a academic context:

@article{galliker2021data,
  title={Data-Driven Dynamics Modelling Using Flight Logs},
  author={Galliker, Manuel Yves},
  year={2021}
}

data-driven-dynamics's People

Contributors

jaeyoung-lim avatar manumerous avatar nrjl avatar pascalau avatar sfuhrer avatar sjschlapbach avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

data-driven-dynamics's Issues

Fixing Computation of Optimization Parameters

Fixing the Symmetric Sigmoid Function

While validating the model I realized, that the scaling factor of the symmetric sigmoid function, used to interpolate between the linear and plate region of the aerodynamics model used to compute lift and drag force in the x-z body plane, was not yet adapted and therefore resulted in both the linear and plate region coefficients contributing to the complete range of present angle of attack.

This can be seen in the plots below. A function output of 1 means purely the plate model is used and 0 means purely the linear/quadratic model for lift/drag respectively is used.

The scale factor is used as follows:
image

Old function: scale_factor = 1

Screenshot from 2021-05-04 11-29-04

New function: scale_factor = 30

Screenshot from 2021-05-04 11-29-31
Screenshot from 2021-05-04 11-28-24

Worse Force Prediciton min Y-Direction

After implementing the changes (that can be seen here) it was observed that the acceleration prediction in the y-direction actually got quite a bit worse when testing a pure hover flight log. This was expected to be the case in the x and z directions, where the new changes actually reduce the model fidelity. However, since the changes should not affect the features in the y direction i would have not expected a change there.

All following logs are generated using the standard gazebo vtol vehicle from the simulator.

With scale_fac = 1

Screenshot from 2021-05-04 12-08-45

c_d_wing_y_offset: -0.00188112081996518

With scale_fac = 30

Screenshot from 2021-05-04 11-43-53
c_d_wing_y_offset: -0.011940344476886905

As can be seen the The second drag coefficient for the wing in y direction is much larger than in the first case and actually performs worse than just setting the coefficient to zero:
Screenshot from 2021-05-04 12-04-46

This last version contains only the gravity contribution to acceleration in y direction.

Question about propeller rolling moment

In gazebo_motor_model.cpp there is a variable called rolling_moment which is calculated as follows:

rolling_moment = -std::abs(real_motor_velocity) * rolling_moment_coefficient_ * velocity_perpendicular_to_rotor_axis;

Does rolling_moment here denote the moment that is an effect of the "advancing blade" generating more lift than the "retracting" blade? If yes why is its sign not dependent on the turning direction of the rotor? I guess the formula does not really make sense for this but that was the first effect i could imagine acting as a torque in that direction.

The only thing I could find on that topic was from the RD lecture, where it was mentioned but not really explained.
Screenshot from 2021-04-08 22-37-11

Also the formula there looks quite different. So I am a bit confused here.

Adapt motor normalization

The gazebo simulation uses actuator inputs between [0, 1] for motors and between [-1, 1] for flaps as explained here. The current normalization should be adapted to this format for an easy interface with gazebo.

No gazebo model for tiltwing

Problem Description
Although there is a parameter estimation pipeline for tiltwings, there is no gazebo model for the tiltwing aircraft where we can generate logs and fly the vehicle.

It is also a problem that PX4 doesn't support tiltwings, so we can't fly the vehicle in either case.

Proposed Solution

  • Add a tiltwing model configuration using control allocation in PX4
  • Create a gazebo tiltwing model for this repository

Additional Context

  • In case this is not addressed before the dev summit, I think it would be more sensible to remove the model from this repo before making it public.

Include options for separate moment and force estimation in config

Depending on the need of the user and the data and knowledge present it could be beneficial to beeing able to for example only estimate the forces but not the moments.

currently the delta_quad_plane_model.py and simple_multirotor_model.py estimate only forces while the quad_plane_model.py estimates both.

Here a unified approach should be chosen at some point.

Choosing SW license

Dear Jay,

Today I had a closer look at the problem with the software licenses you mentioned last week in the meeting using your link. I agree that GPL is probably not the best option here. I think I just used it since I was most familiar with that license from the Linux environment.

Both the MIT and BSD license seem fine to me. Do you have any strong preference? I guess BSD could be better to stay consistent with the PX4 ecosystem.

Actuator Outputs for Standart VTOL

Currently I try to reverse engineer the actuator output data for the flaps of the standart VTOL used in gazebo.

image

When looking at the actuator_outputs topic in the log files it seems that there are 3 outputs active for servo outputs.
However, the standard VTOL only has two elevons and the mixer file of the model as specified here only generates two outputs for the servos by mixing pitch and roll command as expected.

Could it be that for this model the code somehow logs the roll command (left /right or vise versa) separate from the pitch command instead of the actual actuator outputs? At least this seems to be the case when looking at the model.

I tried to find the place in the PX4-Autopilot where this message is published but didn't find anything yet. Maybe the uorb topic is called differently than the topic name in the log?

read min and max PWM from params

This is a reminder to read min and max pwm signal from the parameters to enable flight time selection for real flight logs and not just the simulator.

Choosing best attitude data

To implement the attitude into my model I wondered which of the different attitude data sets should be used. In particular:

  • There are several quaternions in the estimator ( estimator_attitude.00, estimator_attitude.01, ..). Why are there 6 different attitude estimates?

  • Are those estimates the same? They seem to be from the log coming out of the simulation.

  • What is the difference between the estimator attitudes and the vehicle_attitude? The latter seems to be slightly different than the estimator attitudes.

Direction of Attitude Quaternion

Out of the log file I get a quaternion q = [q0, q1, q2, q3]. If I convert this quaternion to a rotation matrix do i get the rotation from the body to the world frame or vise versa?

Is there some documentation about that somewhere? I Somehow didn't find any.

Moment coefficients missing from quadrotor model results

Problem Description
While the moment coefficient estimation seems to be existing in the code, the model results are missing the outputs of the moment coefficients:

  • c_m_leaver_quad
  • c_m_leaver_lin
  • c_m_drag_z_quad
  • c_m_drag_z_lin
  • c_m_rolling

Additional Context

  • Just as a side note, I am not sure if we are estimating redundant parameters here, since we are inputting the position of the rotors, we might as well calculate the moment using the information.
  • Blocker for #89

Adding mass to vehicle models.

Currently only the multirotor_model (outlined in #107) and tiltwing_model were considering the mass of the vehicle. All the other models should also add the mass of the vehicles at some point.

Help options do not display possible options

Problem Description
When trying to generate a model through the generate_parametric_model.py, the help option doesn't display the possible options

  • simple_quadrotor_model
  • quad_plane_model
python3 Tools/parametric_model/generate_parametric_model.py logs/586524-01-/07_26_36.ulg 
usage: generate_parametric_model.py [-h] model log_path
generate_parametric_model.py: error: the following arguments are required: log_path

Also, the script shouldn't fail like this. It should use default values if not specified.

Log path uses absolute paths

Problem Description
When running trying to process a log, the log is passed as an absolute path, which just adds inconvenience of typing more of the path

$ python3 Tools/parametric_model/generate_parametric_model.py quad_plane_model logs/586524-01-/07_26_36.ulg 
Traceback (most recent call last):
  File "Tools/parametric_model/generate_parametric_model.py", line 38, in <module>
    start_model_estimation(arg_list)
  File "Tools/parametric_model/generate_parametric_model.py", line 21, in start_model_estimation
    quadPlaneModel = QuadPlaneModel(rel_ulog_path)
  File "/home/jaeyoung/dev/data-driven-dynamics/Tools/parametric_model/src/models/quad_plane_model.py", line 33, in __init__
    super(QuadPlaneModel, self).__init__(rel_ulog_path, req_topic_dict)
  File "/home/jaeyoung/dev/data-driven-dynamics/Tools/parametric_model/src/models/dynamics_model.py", line 19, in __init__
    self.ulog = load_ulog(rel_ulog_path)
  File "/home/jaeyoung/dev/data-driven-dynamics/Tools/parametric_model/src/tools/ulog_tools.py", line 15, in load_ulog
    ulog = core.ULog(log_file_path)
  File "/home/jaeyoung/.local/lib/python3.8/site-packages/pyulog/core.py", line 134, in __init__
    self._load_file(log_file, message_name_filter_list)
  File "/home/jaeyoung/.local/lib/python3.8/site-packages/pyulog/core.py", line 512, in _load_file
    self._file_handle = open(log_file, "rb")
FileNotFoundError: [Errno 2] No such file or directory: '/home/jaeyoung/logs/586524-01-/07_26_36.ulg'

Expected Behavior
Be able to use relative paths since the logs directory is a symbolic link inside the repository

Tiltwing model flooded with debug outputs

Problem Description
When running the tiltwing model, the terminal is being flooded with debug outputs

Evaluating objective function.
acceleration prediction RMSE:  1.2792108617237172
Evaluating objective function.
acceleration prediction RMSE:  1.2792108636238253
Evaluating objective function.
acceleration prediction RMSE:  1.2792108616957063
Evaluating objective function.
acceleration prediction RMSE:  1.2792108623663425
Evaluating objective function.
acceleration prediction RMSE:  1.2792108616133975
Evaluating objective function.
acceleration prediction RMSE:  1.2792108619574944
Evaluating objective function.
acceleration prediction RMSE:  1.2792108624657261
Evaluating objective function.

Air density data in Aero mini tilt wing log

To incorporate the effect of air density into my model I was looking into the ulog file for our tilt wing model. Unfortunately, I did not find anything related to the air density measured by the barometer.

Could it maybe be in one raw_data channel of the adc_Report?

@tstastny

Question about applying forces to links in gazebo

While looking through the gazebo_motor_model.cpp I had a few questions coming up.

Am I right to assume that link_ will specify the propeller as an actual physical body which is linked through the rest through the revolute joint joint_?
What is the difference between adding relative and normal forces?

e.g. link_->AddRelativeForce(ignition::math::Vector3d(0, 0, force * scalar));

e.g. link_->AddForce(air_drag);

How do we know the direction the force is applied to in both cases? Is the first one relative to a local bodyframe of link and the second one represented in the "main body frame"?

I guess it would be possible to find that somewhere in the gazebo repo but i hope it is okay to ask, since you might already know part of the answers I guess.

Log creation when running gazebo with shell script

Hi

Today i tried to generate a log file with the tech-pod using the sitl_run.sh script. Unfortunately, i did not find any log created from the flown mission when using the script to start it. Are the logs saved in a different location than usual when run like this? Where would they be?
I searched in
~/src/PX4-Autopilot/build/px4_sitl_default/logs

Thanks for your help.

body rotation influence on angular acceleration

The influence of the body angular velocity on the angular acceleration is not yet taken into account:
image

There is already a function computing features for omega x omega in the dynamics model:

    def compute_body_rotation_features(self, angular_vel_topic_list):
        """Include the moment contribution due to rotation body frame:
        w x Iw = X_body_rot * v
        Where v = (I_y-I_z, I_z-I_x, I_x- I_y)^T
        is comprised of the inertia moments we want to estimate
        """
        angular_vel_mat = (self.data_df[angular_vel_topic_list]).to_numpy()
        X_body_rot = np.zeros((3*angular_vel_mat.shape[0], 3))
        X_body_rot_coef_list = ["I_yy-I_zz", "I_zz-I_xx", "I_xx- I_yy"]
        for i in range(angular_vel_mat.shape[0]):
            X_body_rot[3*i, 0] = angular_vel_mat[i,
                                                 1]*angular_vel_mat[i, 2]
            X_body_rot[3*i + 1, 0] = angular_vel_mat[i, 2] * \
                angular_vel_mat[i, 0]
            X_body_rot[3*i + 2, 0] = angular_vel_mat[i, 0] * \
                angular_vel_mat[i, 1]
        return X_body_rot, X_body_rot_coef_list

To really take this effect into account it would be necessary to define I_xx, .., I_zz as optimization variables which would rend the problem nonlinear. I guess this is long-therm the way to go. What do you think @Jaeyoung-Lim ?

Name mismatch between pipeline model and gazebo models

Problem Description
The names between the pipeline model / gazebo model don't match.

To keep track, the names are as the following

Model in pipeline / Gazebo model / PX4 airframe config name

  • quadrotor_model / iris_aerodynamics / iris
  • quadplane_model / standard_vtol_aerodynamics / standard_vtol
  • standardplane_model / techpod_aerodynamics / plane or techpod

Proposed Solution

  • I don't think it makes much sense to align everything with the PX4 airframe configs since this is vehicle specific
  • probably switch model names to quadrotor, standard_vtol, tiltwing_vtol

Repeated code in model classes

Problem Description
This is just a note while looking into the architecture

While dynamics_model is a parent class of different models, there seems to be quite some code that are identical, but unshared between models.

e.g. estimate_model seems almost identical across models, but not quite.

def estimate_model(self):
print("Estimating quad plane model using the following data:")
print(self.data_df.columns)
self.data_df_len = self.data_df.shape[0]
print("resampled data contains ", self.data_df_len, "timestamps.")
X, y = self.prepare_regression_matrices()
self.reg = LinearRegression(fit_intercept=False)
self.reg.fit(X, y)
print("regression complete")
metrics_dict = {"R2": float(self.reg.score(X, y))}
self.coef_name_list.extend(["intercept"])
coef_list = list(self.reg.coef_) + [self.reg.intercept_]
self.generate_model_dict(coef_list, metrics_dict)
self.save_result_dict_to_yaml(file_name="quad_plane_model")
return

def estimate_model(self):
print("Estimating quad plane model using the following data:")
print(self.data_df.columns)
self.data_df_len = self.data_df.shape[0]
print("resampled data contains ", self.data_df_len, "timestamps.")
X, y = self.prepare_regression_matrices()
self.reg = LinearRegression(fit_intercept=False)
self.reg.fit(X, y)
print("regression complete")
metrics_dict = {"R2": float(self.reg.score(X, y))}
self.coef_name_list.extend(["intercept"])
coef_list = list(self.reg.coef_) + [self.reg.intercept_]
self.generate_model_dict(coef_list, metrics_dict)
self.save_result_dict_to_yaml(file_name="quad_plane_model")
return

def estimate_model(self):
print("Estimating quad plane model using the following data:")
print(self.data_df.columns)
self.X, self.y = self.prepare_regression_mat()
self.reg = LinearRegression(fit_intercept=False)
self.reg.fit(self.X, self.y)
print("regression complete")
metrics_dict = {"R2": float(self.reg.score(self.X, self.y))}
self.coef_name_list.extend(["intercept"])
coef_list = list(self.reg.coef_) + [self.reg.intercept_]
self.generate_model_dict(coef_list, metrics_dict)
self.save_result_dict_to_yaml(file_name="simple_quadrotor_model")
return

Could not install dependencies

Problem Description
I could not install the python dependencies as specified in this package

Steps to reproduce

$ cd Tools/parametric_model
$ pip3 install -r requirements.txt 
Collecting h5py>=3.2.1 (from -r requirements.txt (line 1))
  Could not find a version that satisfies the requirement h5py>=3.2.1 (from -r requirements.txt (line 1)) (from versions: 2.2.1, 2.3.0b1, 2.3.0, 2.3.1, 2.4.0b1, 2.4.0, 2.5.0, 2.6.0, 2.7.0rc2, 2.7.0, 2.7.1, 2.8.0rc1, 2.8.0, 2.9.0rc1, 2.9.0, 2.10.0, 3.0.0rc1, 3.0.0, 3.1.0)
No matching distribution found for h5py>=3.2.1 (from -r requirements.txt (line 1))

Update simple drag model for quadcopter.

It was forgotten to finalize the simple drag model used by the simple_quadrotor_model to a meaningfull final result. It now still contains lift and drag parameters the same way as the fixed wing models do.

Before the code is released this should be adapted to a model of choice. Maybe as the name suggest we could for example just define a drag parameter for each axis.

Code review comments for meeting

@manumerous Since I wasn't sure if I could make it to the meeting, I am writing down some points that I would like to discuss in the architecture review meeting today.

Minimum set of features
In my opinion, the minimum set of features for this repo to be useful is as the following.

  • Needs to work for multirotors/fixed wing (reliably)
  • Predict moments
  • Working with the Gazebo simulation
  • Minimal set of diagnostics

While it is important to get something working with VTOLs, I think VTOLs have many different configurations and other complexities that make it hard to generalize. Moreover most people that will be using this(in case this gets public) will only be using it for multirotors/fixedwing) Therefore, IMHO it is important that this pipeline works well for multirotors and fixedwing vehicles.

Architecture Comments
These are some comments on the structure of this repo

  • Currently everything is written inside a dynamics model class. However, the class contains many functionalities that are not related to models such as visaulziation, optimization and data processing. The repo needs to be broken down into modules, so that prototyping for other methods can be easier. We could first start to breakdown the pipeline into the following modules.
    • Data processing / handling (#84, #87)
    • Visualization
    • Diagnostics (including visualization)
  • In general, diagnostics of the pipeline is missing.
  • Installing the package vs absolute path imports
  • Configuration management for common vehicle types. (e.g. quadrotor /hexacopter should only differ a config file, not code)
  • Currently the code plots a few dozen figures in the UI. However, this needs to be written into a report (e.g. pdf) so that the figures are persistent.

Multirotor identification with better input shaping

Problem Description
When trying to run the identified dynamics for multirotors in the simulation with #89 I have not been able to simulate the rotational dynamics.

When looking at the residual errors, the prediction doesn't seem to accurately capture the dynamics anyway, therefore I think not being able to fly is a expected result.

Therefore I have been looking into how we can generate better data for multirotors that are informative enough for the rotational dynamics. I have been reading that some work for online parameter identification were using Lissajous Trajectories in order to excite all axis of the vehicle dynamics for multirotors.

Method
The vehicle is following a Lissajous trajectory that is shown below. Trajectory tracking is done using a nonlinear geometric controller implemented in https://github.com/Jaeyoung-Lim/mavros_controllers
Hovering drone

The Lissajous curves are in the following form.
image
From: Böhm, Christoph, Martin Scheiber, and Stephan Weiss. "Filter-Based Online System-Parameter Estimation for Multicopter UAVs.", Robotic Science and Systems, 2021

The coefficient c_j is randomly selected in the range of 3~8 so I chose 3, 4, 8 for each axis.

The generated log can be found in: flightreview

Results
The rotational predictions seem to match slightly better
With Previous Reference Log

  • Dataset Visualization
    Processed_Data_Visualization

  • Acceleration Prediction
    Figure_3
    Figure_4

  • Residual Distribution
    Residual_Visualization

With Lissajous

  • Dataset Visualization
    Processed_Data_Visualization

  • Acceleration Prediction
    Figure_3
    Figure_4

  • Residual Distribution
    Residual_Visualization

Generalize quad_plane_model.py

At some point it would be beneficial to further generalize the structure of the quad plane model to be suitable for all kind of quad plane configurations. These could include :

  • AAER
  • AAE
  • Delta with two ailevons

Since only the aerodynamics module and the input to it change for the different types this could be handled on a case basis.

Currently there is a delta quad plane model that only estimates forces to work with logs from logs.px4.io. This file could then be eliminated.

input scaling in log vs simulation

Problem Description
I am somewhat confused with the input scaling values in the logs while trying to link this into simulation.

  • The input seems to be normalized with min and max pwm values here in the pipeline
  • For the simulation, there is a separate input scaling mechanism being used here

They don't seem to be consistent and it is introducing problems when rescaling inputs to the sim and multiplying coefficients for the input values

Additional Context

  • Blocker for #89

Submodule update with make

Submodule update with make #74

I just wanted to update the submodule to the latest commits using the make submodulesupdate command.

Unfortunately, it did not work and I think we are missing the following command to update the submodules to the latest commits:

submodulesupdate:
	git submodule update --remote --merge
	git submodule update --init --recursive

I would also suggest to rename submodulesupdate to update-submodules. This seems more consistent with the other options such as for example estimate-model.

What do you think @Jaeyoung-Lim? If you agree I will adress this in #70

Rework user terminal interface

Currently the project contains a quite large amount of printouts. Some of them are helpful, others are artifacts from debugging and development and are less needed.

Before releasing the project it would make sense to redesign the terminal output user interface with the intend of presenting only the valuable information to the user and unify the printouts between models.

[FeatureRequest] Using multiple flight logs

Problem Description
The current implementation runs on available data from a single flight log.

While this makes sense if you can learn your dynamics model from a single flight, this constraint actually is over constraining the problem since the observations are only based on single time steps.

Proposed solution
Allow the model to populate data frames from multiple flight uLog files

Model results file missing information of model description

Problem Description
The yaml output file of the pipeline only outputs the value of parameters and is missing most of the information that is present in the model config. List of Missing information is as the following.

  • rotor position and direction
  • Control surfaces channel mapping

These information are needed when integrating the model output into the sim.

Possible Alternatives
Possible alternatives maybe that the plugin reads the model outputs file as well as the model config file.

  • This would make the pipeline error prone introducing a possibility of wrong model config / model outputs to the plugin.
  • This would start to mix pipeline configs and simulation configs. A clean separation (model configs for pipeline configuration, model outputs as a interface for interacting with the sim) seems like a better implementation.

Additional Context

  • This needs to be fixed for continuing #89

Estimating Parametzers of continuous time system using discrete data

Lately I have been a bit confused and thinking about how to obtain the parameters from the discrete data, but in the papers I have and on the internet I did not yet find a solution or actual description how this can be done. For this reason I decided to post the question here. I guess there should already be plenty of solutions around for that problem.

To my understanding we need to do the following:

  1. Define a CT model of our actual dynamics
  2. Discretize it using some system discretization method (e.g. 4th order Runge-Kutta)
  3. Estimate the model parameters using the data.

The problem I now have is that for solving the discretization in 2 i would already need the parameters estimated using the DT model of 3. How is this chicken and egg problem usually solved? Should we solve 2 and 3 iteratively?

If you already have time for an answer that would be great. Else we can discuss this in the afternoon meeting.

Setting rotor diameter results in huge motor constants

Problem Description
Setting the rotor diameters to diameter: 0.2032 results in huge motor constants.

I think the values are unreasonable, (probably it makes sense for it to increase about 25 times?)

To reproduce: #106

Before PR
quadrotor_model.yaml

coefficients:
  c_d_wing_xy_lin: 0.022751869230557692
  c_d_wing_xy_offset: 0.005741662399349513
  c_d_wing_xy_quad: 0.0630041928511851
  c_d_wing_xz_stall_90_deg: -0.04128438467976083
  c_d_wing_xz_stall_min: 0.0057110318294756185
  c_d_wing_y_offset: 0.007906539273005816
  intercept: 0.0
  vertical_c_m_drag_z_lin: -10.324851252250626
  vertical_c_m_drag_z_quad: 6.0213939854338685
  vertical_c_m_leaver_lin: -8.33722923229799
  vertical_c_m_leaver_quad: 32.623014913712176
  vertical_c_m_rolling: -1.467193735480539
  vertical_rot_drag_lin: 0.12166778752250448
  vertical_rot_thrust_lin: -0.1425458591033593
  vertical_rot_thrust_quad: 4.003013943813853
metrics:
  R2: 0.9781734959524644
model:
  vertical_:
  - dataframe_name: u0
    description: front right rotor
    position:
    - 0.13
    - 0.22
    - -0.023
    rotor_0: null
    rotor_axis:
    - 0
    - 0
    - -1
    rotor_type: RotorModel
    turning_direction: -1
  - dataframe_name: u1
    description: back left rotor
    position:
    - -0.13
    - -0.2
    - -0.023
    rotor_1: null
    rotor_axis:
    - 0
    - 0
    - -1
    rotor_type: RotorModel
    turning_direction: -1
  - dataframe_name: u2
    description: front left rotor
    position:
    - 0.13
    - -0.2
    - -0.023
    rotor_2: null
    rotor_axis:
    - 0
    - 0
    - -1
    rotor_type: RotorModel
    turning_direction: 1
  - dataframe_name: u3
    description: back right rotor
    position:
    - -0.13
    - 0.2
    - -0.023
    rotor_3: null
    rotor_axis:
    - 0
    - 0
    - -1
    rotor_type: RotorModel
    turning_direction: 1
numper of samples: 5650

After PR
quadrotor_model.yaml

coefficients:
  c_d_wing_xy_lin: 0.02275186923057826
  c_d_wing_xy_offset: 0.005741662398774494
  c_d_wing_xy_quad: 0.06300419285146497
  c_d_wing_xz_stall_90_deg: -0.041284384679179054
  c_d_wing_xz_stall_min: 0.005711031829141788
  c_d_wing_y_offset: 0.007906539272867441
  intercept: 0.0
  vertical_c_m_drag_z_lin: -29803.363896948467
  vertical_c_m_drag_z_quad: 17381.150752720474
  vertical_c_m_leaver_lin: -4890.203454982687
  vertical_c_m_leaver_quad: 19135.035849196163
  vertical_c_m_rolling: -1.4671937354805777
  vertical_rot_drag_lin: 0.12166778752301603
  vertical_rot_thrust_lin: -16.989613335289164
  vertical_rot_thrust_quad: 2347.968620383804
metrics:
  R2: 0.9781734959524644
model:
  vertical_:
  - dataframe_name: u0
    description: front right rotor
    diameter: 0.2032
    position:
    - 0.13
    - 0.22
    - -0.023
    rotor_0: null
    rotor_axis:
    - 0
    - 0
    - -1
    rotor_type: RotorModel
    turning_direction: -1
  - dataframe_name: u1
    description: back left rotor
    diameter: 0.2032
    position:
    - -0.13
    - -0.2
    - -0.023
    rotor_1: null
    rotor_axis:
    - 0
    - 0
    - -1
    rotor_type: RotorModel
    turning_direction: -1
  - dataframe_name: u2
    description: front left rotor
    diameter: 0.2032
    position:
    - 0.13
    - -0.2
    - -0.023
    rotor_2: null
    rotor_axis:
    - 0
    - 0
    - -1
    rotor_type: RotorModel
    turning_direction: 1
  - dataframe_name: u3
    description: back right rotor
    diameter: 0.2032
    position:
    - -0.13
    - 0.2
    - -0.023
    rotor_3: null
    rotor_axis:
    - 0
    - 0
    - -1
    rotor_type: RotorModel
    turning_direction: 1
numper of samples: 5650

Check for Model Class and Config Combatibility

Each config file contains a specified model file (e.g. model_file: "delta_quadplane_model.py"). Currently it is not checked if the config is used with its corresponding model class. We should implement a check here in order to prevent mistakes as presented in here.

Protect master branch

I accidentally pushed onto master since I forgot that I was not on my dev branch. I suggest protecting the master again as we had in the old repository.

However, I don't think I have the power to do so.

Enabling moment estimation results in failure

Problem Description
Enabling moment estimation results in a failure

Log
To reproduce,

$ make estimate-model model=quadrotor_model
python3 Tools/parametric_model/generate_parametric_model.py --model quadrotor_model --config /home/jaeyoung/dev/data-driven-dynamics/Tools/parametric_model/configs/quadrotor_model.yaml --data_selection False /home/jaeyoung/dev/data-driven-dynamics/resources/quadrotor_model.ulg 
Visual Data selection enabled:  False
Initializing of configuration succesfull. 
Resample frequency:  100.0 Hz
Starting data resampling of topic types:  dict_keys(['actuator_outputs', 'vehicle_local_position', 'vehicle_attitude', 'vehicle_angular_velocity', 'sensor_combined', 'vehicle_angular_acceleration'])
Resampling |################################| 6/6
Initializing of configuration succesfull. 
Resample frequency:  100.0 Hz
Traceback (most recent call last):
  File "Tools/parametric_model/generate_parametric_model.py", line 72, in <module>
    start_model_estimation(arg_list)
  File "Tools/parametric_model/generate_parametric_model.py", line 40, in start_model_estimation
    model = MultiRotorModel(config_file, model_name="quadrotor_model")
  File "/home/jaeyoung/dev/data-driven-dynamics/Tools/parametric_model/src/models/multirotor_model.py", line 47, in __init__
    False), "Estimation of moments is not yet implemented in DeltaQuadPlaneModel. Disable in config file to estimate forces."
AssertionError: Estimation of moments is not yet implemented in DeltaQuadPlaneModel. Disable in config file to estimate forces.
Makefile:33: recipe for target 'estimate-model' failed
make: *** [estimate-model] Error 1

Additional Context

  • This seems to have been caused by #94

Estimated local acceleration is logged at low rate

@manumerous I have looked into the python pipeline during the weekend, and also played around with #43

Here is what I found

  • (Correct me if I am wrong) When you are estimating the body acceleration, I think gravity is not being accounted for. This is probably why you see large side forces on your vehicle, since you see gravity in the body frame whenever the vehicle tilts.

  • The acceleration from vehicle_local_position message is being used, which the logging rate is very low. (10Hz )

This is a comparison of the topics logged for acceleration. You can see that the topic is logged at a very low rate. Therefore might be better to use the sensor_combined topic which is also already in the body frame

image

@tstastny FYI

Forces in y direction

image

After finding a few more mistakes in my model I get now quite nice predictions in the z direction. However, the force prediction in the x and y directions looks less good. What kind of forces could cause the gravity compensated

Choosing a valid wind speed from Log

In the log there are two topics containing the wind in north and east direction. The two topics are airspeed_wind.00 and airspeed_wind.01

Somehow both give drastically different values Which one is the real wind speed? Why are there two topics with so different data? Is Px4 using different estimators?

Expanding tests for config files

Currently there is only a basic set of tests of config files covering everything on the side of the dynamics model.

At some point the tests could be expanded by:

  • testing that all actuator topics in the model_config are also contained in the required ulog topics.
  • testing the data types and dimensions of each rotor parameter

Furthermore, it would be nice to have new separate test that validates all config files that are uploaded to the repo. This way we can ensure that the syntax is correct in all of them.

Remove yaml files from generated models from git history.

Currently every generated dynamics model saves all its parameters and metrics into a yaml file. The changes in this file are then tracked whenever a different model is used or the parameters slightly change, which is not desirable.

I therefore propose to make a separate folder to which the yaml files are saved using the model name of the model they were used to generate and a timestamp. Like this it is easier to track progress over time and this folder can just be added to the .gitignore.

Any other suggestions?

Plate wing model for flow from behind

For any vehicle in hover it can occur, that the airspeed vector comes from behind (negative x direction in body frame) when moving backwards or turning the back towards the wing.

For a quad plane this flow from behind will flow from the reverse direction over the wing and should in theory create a small lift force pointing downwards.

Since this is an aerodynamics configuration very specific to quad planes which should never occur in normal fw cruise flight i did not find any literature or references on that subject.

Currently the model just assumes the wing to be a plate when airflow comes from behind. Therefore, it does not take the effect of the reversed flow over the curved airfoil into account. I would assume that this additional effect due to the curvature of the airfoil would be less significant, since the reverse airflow due to moving backwards would usually be smaller than the cruise airspeed. However for higher wind that assumption might not hold well.

Is that additional downward force significant enough to include in our parametric model? Or should we maybe try to capture the effect with the residual dynamics? I would be happy for some second opinion or literature here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.