Hi!

I was updating numpy in Python 3.8 and I got the following warning:

DEPRECATION: mpc 0.0.4 has a non-standard dependency specifier numpy>=1<2. pip 23.3 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of mpc or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at pypa/pip#12063

Can I solve the problem somehow? Thank you in advance!

/Users/rossfisher/PycharmProjects/wea/venv/bin/python /Users/rossfisher/PycharmProjects/wea/mpcccc.py
Initial mean(cost): 3.9041e+01
/Users/rossfisher/PycharmProjects/wea/venv/lib/python3.7/site-packages/mpc/pnqp.py:18: UserWarning: torch.btrifact is deprecated in favour of torch.lu and will be removed in the next release. Please use torch.lu instead.
H_lu = H.btrifact()
Traceback (most recent call last):
File "/Users/rossfisher/PycharmProjects/wea/mpcccc.py", line 38, in
)(x_init, QuadCost(C, c), LinDx(F))
File "/Users/rossfisher/PycharmProjects/wea/venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/Users/rossfisher/PycharmProjects/wea/venv/lib/python3.7/site-packages/mpc/mpc.py", line 267, in forward
x_init, C, c, F, f, cost, dx, x, u)
File "/Users/rossfisher/PycharmProjects/wea/venv/lib/python3.7/site-packages/mpc/mpc.py", line 362, in solve_lqr_subproblem
x, u = _lqr(x_init, C, c, F, f if f is not None else e)
File "/Users/rossfisher/PycharmProjects/wea/venv/lib/python3.7/site-packages/mpc/lqr_step.py", line 112, in forward
Ks, ks, self.back_out = self.lqr_backward(C, c_back, F, f_back)
File "/Users/rossfisher/PycharmProjects/wea/venv/lib/python3.7/site-packages/mpc/lqr_step.py", line 303, in lqr_backward
x_init=prev_kt, n_iter=20)
File "/Users/rossfisher/PycharmProjects/wea/venv/lib/python3.7/site-packages/mpc/pnqp.py", line 19, in pnqp
x_init = -q.btrisolve(*H_lu) # Clamped in the x assignment.
TypeError: btrisolve() takes 3 positional arguments but 4 were given

This is useful if the controller consistently does not hit a fixed point

Minor issue, but worth mentioning: this library has no explicit requirements for FFMPEG, but recordings of the videos seem to depend pretty heavily on the -crf command (example: cell:4;line:6 in the CartPole example). It seems to presume a recent FFMPEG build and "It assumes you have ffmpeg compiled with --enable-libx264" (from here ). I tried to tinker with it and get it to work with the encoders listed in ffmpeg -codecs | grep 264, by I had no luck (I'm not a heavy user of ffmpeg). Do you have a suggestion for getting around this without recompiling ffmpeg (aka using the precompiled ffmpeg from apt, or the precompiled version included with conda/pip python libraries ?

My motivation for trying out box-DDP was the failure of iLQR on the pendulum swing up task. It tries to apply more torque than allowed and always swings back and forth on one side. This project has working pendulum swing ups with the mpc/env_dx/pendulum.py environment, but I haven't been able to get it to work on the gym pendulum environment: https://github.com/LemonPi/mpc.pytorch/blob/master/examples/gym_pendulum.py

It would also swing back and forth on one side and exit some iterations in from not converging. Any ideas why (I can silence the exit with exit_uncoverged=False but I meant why it's not working as the existing pendulum)?

I try the task of Pendulum-v0. Here is an interesting scene: the moment switches the direction when the stick is at the bottom other than when the stick falls. But, a better solution is that the direction of the moment is always the same as the speed direction of the stick. When the stick falls, the moment should change the direction immediately. Thus, I have a question about the optimal property of this MPC solution.

Heyo!

I'm a user and a layman contributor to OpenPilot, an open source driving agent. With some python and a consumer phone, I've added enhanced lane keep assist to my Toyota Prius for under $500 USD.
https://github.com/commaai/openpilot

I spend my time tuning lateral control and have found their plant model (vehicle_model.py) to be insufficient to properly estimate the path of the vehicle with their MPC solution they've written in Acado Toolkit.

You can find their MPC here:
https://github.com/commaai/openpilot/tree/devel/selfdrive/controls/lib/lateral_mpc

And the vehicle model here:
https://github.com/commaai/openpilot/blob/devel/selfdrive/controls/lib/vehicle_model.py

PID or INDI control (user selectable) is used to send torque to the vehicle's EPS motor. The input to PID is desired angle, which sends torque commands to drive the wheels to the desired angle. (PID) https://github.com/commaai/openpilot/blob/devel/selfdrive/controls/lib/pid.py

The existing MPC controller puts these arrays out every cycle:

Lastly, the inputs/outputs of MPC are as follows:
self.libmpc = libmpc_py.libmpc
self.libmpc.init(MPC_COST_LAT.PATH, MPC_COST_LAT.LANE, MPC_COST_LAT.HEADING, steer_rate_cost)

self.mpc_solution = libmpc_py.ffi.new("log_t *")
self.cur_state = libmpc_py.ffi.new("state_t *")
self.cur_state[0].x = 0.0
self.cur_state[0].y = 0.0
self.cur_state[0].psi = 0.0
self.cur_state[0].delta = 0.0

self.angle_steers_des = 0.0
self.angle_steers_des_mpc = 0.0
self.angle_steers_des_prev = 0.0
self.angle_steers_des_time = 0.0

In which they reference these values:
states[0].x = v_ego (speed)
states[0].psi = v_ego (speed) * curvature_factor (how curvy is the road center) * math.radians(steer_angle) / steer_ratio
states[0].delta = math.radians(steer_angle) / steer_ratio

Long story short, and what I'm assuming is clearly out of your scope as this project's maintainer, can mpc.pytoch be used as a replacement for MPC exported from acado toolkit? If I'm understanding the readme correctly,
"and non-linear system transition dynamics that can be defined by hand if you understand your environment or a neural network if you don’t."

Does this mean that mpc.pytorch can, um, use a neural network to learn the plant model?

Apologies if I'm off track in any of this, I'm not formally educated and knew nothing about MPC, self driving or any of this a few months ago :) I think it could be a good real world use case for your library and I'm beginning to learn just enough to be dangerous!

On the CartPole and Swingup examples getting the following error

Tmp dir: /tmp/tmpsasrsxq7

  0%|          | 0/100 [00:00<?, ?it/s]

---------------------------------------------------------------------------

RuntimeError                              Traceback (most recent call last)

[<ipython-input-2-cc854f287de7>](https://localhost:8080/#) in <module>()
     40         grad_method=GradMethods.AUTO_DIFF,
     41         eps=1e-2,
---> 42     )(x, QuadCost(Q, p), dx)
     43 
     44     next_action = nominal_actions[0]

3 frames

[/usr/local/lib/python3.7/dist-packages/torch/autograd/function.py](https://localhost:8080/#) in __call__(self, *args, **kwargs)
    314     def __call__(self, *args, **kwargs):
    315         raise RuntimeError(
--> 316             "Legacy autograd function with non-static forward method is deprecated. "
    317             "Please use new-style autograd function with static forward method. "
    318             "(Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)")

RuntimeError: Legacy autograd function with non-static forward method is deprecated. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)

Any clue about what this relates to?

Hello,

Thanks for sharing your this project. I had some issues running the examples and seem to be related to torch version. I am currently using torch 1.2.0 and CUDA10.0, could you please tell me the latest torch and cuda version used with this project?

I am getting errors like this:

(.mpc) user@userlabpc:~/git_clone/mpc.pytorch/examples$ python pendulum.py 
Tmp dir: /tmp/tmp7hokco9j
  0%|                                                                                      | 0/100 [00:00<?, ?it/s]/pytorch/torch/csrc/autograd/python_function.cpp:638: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
  0%|                                                                                      | 0/100 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "pendulum.py", line 80, in <module>
    )(x, QuadCost(Q, p), dx)
  File "/home/user/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/user/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/mpc-0.0.3-py3.5.egg/mpc/mpc.py", line 265, in forward
  File "/home/user/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/mpc-0.0.3-py3.5.egg/mpc/mpc.py", line 362, in solve_lqr_subproblem
  File "/home/user/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/mpc-0.0.3-py3.5.egg/mpc/lqr_step.py", line 112, in forward
  File "/home/user/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/mpc-0.0.3-py3.5.egg/mpc/lqr_step.py", line 303, in lqr_backward
  File "/home/user/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/mpc-0.0.3-py3.5.egg/mpc/pnqp.py", line 31, in pnqp
  File "/home/user/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/torch/tensor.py", line 325, in __rsub__
    return _C._VariableFunctions.rsub(self, other)
RuntimeError: Subtraction, the `-` operator, with a bool tensor is not supported. If you are trying to invert a mask, use the `~` or `bitwise_not()` operator instead.

Traceback (most recent call last):
  File "/home/fer/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/fer/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/mpc-0.0.3-py3.5.egg/mpc/mpc.py", line 265, in forward
  File "/home/fer/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/mpc-0.0.3-py3.5.egg/mpc/mpc.py", line 362, in solve_lqr_subproblem
  File "/home/fer/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/mpc-0.0.3-py3.5.egg/mpc/lqr_step.py", line 114, in forward
  File "/home/fer/git_clone/mpc.pytorch/.mpc/lib/python3.5/site-packages/mpc-0.0.3-py3.5.egg/mpc/lqr_step.py", line 344, in lqr_forward
AttributeError: module 'torch' has no attribute 'any'

Thank you very much.

Hi,

I was trying to run your scripts on a Jetson Nano, which has an ARMv8 cpu and runs Ubuntu 18.04LTS, so all the following relates to this platform only. I found that it will work in this platform only when installing under python3 and using pip install mpc. If installation is done using python setup.py install under python3 the installation does not work, and both pip install mpc and python setup.py install do not work under python2.7. The error found is always with importing mpc:

Traceback (most recent call last):
  File "examples/MassSpringDamper_Example.py", line 41, in <module>
    from mpc import mpc
  File "build/bdist.linux-aarch64/egg/mpc/mpc.py", line 17, in <module>
  File "build/bdist.linux-aarch64/egg/mpc/lqr_step.py", line 13, in <module>
ImportError: cannot import name mpc

However, even using python3 and pip install mpc, the scripts are unable to locate the mpc/env_dx folder, presumably because it is only available when cloning your github repo and installing using setup.py. In order to make it work one has to move files in env_dx folder to the appropriate location in order for python to find them.

I was wondering if you have found this issue before and how did you solve it, or any pointers on a solution. This does not happen in my desktop (Intel Core i5, Ubuntu16.04).

Thanks!

Hi I'm trying to run the following code

x_lqr, u_lqr, objs_lqr = controller(state, QuadCost(Q, c), model_dynamics)

where controller is initialized as

controller = mpc.MPC(
        n_state=n_state,
        n_ctrl=n_ctrl,
        T=T,
        u_lower=action_low, 
        u_upper=action_high,
        lqr_iter=20,
        verbose=1,
        backprop=True,
        exit_unconverged=False,
        grad_method=GradMethods.AUTO_DIFF)

and I'm getting the following error output

Any idea what might be causing this ? From the source code pinned below Ic is clearly a boolean and I would expect this to always be a boolean and thus for Line 33 (also pinned below) to always fail.

Also worrying to me is the fact that the error output has If = Ic - 1 on line 31 and not 33. I've updated my package with pip install mpc --upgrade and it claims it's up to date. Any help on this would be super appreciated.

When running the example "Time Varying Linear-Quadratic Control" with Pytorch 1.5.1, the following error appears,

"Legacy autograd function with non-static forward method is deprecated. "

RuntimeError: Legacy autograd function with non-static forward method is deprecated. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)

Running the example in the Jupyter Notebook generates the same error.

should import empc be changed to import mpc (or rename the mpc directory)?

self.length is not defined in cartpole.py

I'm not sure if it is leaking memory, but the forwards pass may leave some objects around that the gc may not be able to clear.

Hello,

I was wondering if you had any examples in which the GradMethods.ANALYTIC option is used. I would like to see how to implement the calculation of the jacobian for your library to use it. Looking at the code I see one needs to implement the grad_input method, but would like to know more details. Right now I am having an issue in which when I try to use GradMethods.ANALYTIC the size of the matrices do not match in the forward method of my dynamics scripts (inside env_dx folder).

Your feedback will be very appreciated.

• Juan

Following Example: Time-Varying Linear Control on project page I failed importing mpc, QuadCost, LinDx from mpc package.
I installed mpc with pip install mpc and the version is 0.0.2.

Is there a way to improve the speed performance of the solver by doing the computation on GPU? If so, which tensors should be assigned to 'cuda'?

Is somewhat involved since LqrStep uses a lot of class attributes for the forward and backward pass

Dear all,

Running the example notebooks (Time-Varying Linear Control or Pendulum), I get the following error:

RuntimeError: Legacy autograd function with non-static forward method is deprecated. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)

Does anybody know how to resolve this issue?

Thank you in advance!

Hi,
I am quite new to RL. I found the example of gym_pendulum.py very useful and I was trying to do the same with continuous mountain car. My code is available here. It does not converge. I am not sure which hyper parameter to tune or what the value of downward_start start to use.

Any help would be really appreciated. Thanks

If I understand correctly, 206bd93 does the LQR step with the slew rate penalty in the quadratic cost approximation, but does the line search on the unmodified cost function without the slew rate penalty.

This is non-trivial to fix because we currently have a time-invariant non-quadratic cost function and no easy way to access the nominal control trajectory within it. If we did, the clean solution would be to write a new cost function with the slew rate penalty. Instead, perhaps the easiest way of fixing this would be to add the slew rate to the get_cost function.

• Forward pass without slew rate
• Backward pass without slew rate
• Forward pass with slew rate
• Backward pass with slew rate

Undefined names have the potential to raise NameError at runtime.

flake8 testing of https://github.com/locuslab/mpc.pytorch on Python 3.7.0

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./mpc/lqr_step.py:140:16: F821 undefined name 'lqr'
        _lqr = lqr.LQR(
               ^
./mpc/lqr_step.py:239:38: F821 undefined name 'Vtp1'
                Qt = C[t] + Ft_T.bmm(Vtp1).bmm(Ft)
                                     ^
./mpc/lqr_step.py:241:42: F821 undefined name 'vtp1'
                    qt = c[t] + Ft_T.bmm(vtp1.unsqueeze(2)).squeeze(2)
                                         ^
./mpc/lqr_step.py:244:42: F821 undefined name 'Vtp1'
                    qt = c[t] + Ft_T.bmm(Vtp1).bmm(ft.unsqueeze(2)).squeeze(2) + \
                                         ^
./mpc/lqr_step.py:245:34: F821 undefined name 'vtp1'
                        Ft_T.bmm(vtp1.unsqueeze(2)).squeeze(2)
                                 ^
5     F821 undefined name 'lqr'
5

Dear developers,

I tried to run the example 'gym_pendulum_approximate.py'. The solver did not manage to control the pendulum to the upright position as planned. The file was not modified at all. I assumed that you have testified it, just to make sure whether this is due to some mismatch in the versions of some python modules. Thanks.

July

• Current: Fixed number of iters or a threshold on the full action step norm
• It would also be nice to stop based on the absolute or relative change in cost if the user doesn't care about hitting a fixed point.

Hi,
The mpc package (version 0.0.4) specifies its dependency on numpy in a non-standard way, which will cause problems with future versions of pip. The current dependency specifier is written as numpy>=1<2, which is not a valid format.

The standard way to specify the dependency should be numpy>=1.0,<2.0.

Collecting mpc
  Using cached mpc-0.0.4.tar.gz (17 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [3 lines of output]
      error in mpc setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers; Expected end or semicolon (after version specifier)
          numpy>=1<2
               ~~~^
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Currently there is a backprop option that defaults to True that does nothing. It would be good if this actually controlled for whether gradients are computed throughout. My intended use case is replacing an iLQR implementation with one that handles control constraints. If differentiability is too baked into the code, is there an alternative control-constrained (box constraints are sufficient) DP based control implementation you would recommend (numpy or pytorch are fine)?

Alternatively QuadCost(q, c, diag=True)

I also don't like having the x.requires_grad conditional in detach_maybe since it hides if that tensor is detached or not.

\cc @ivandariojr

I get the following error. Probably a pytorch version issue. Any suggestions?

TypeError: FunctionBackward.forward: expected Tensor or tuple of Tensor (got lqrBackOut) for return value 2

Hi @bamos !
I'm trying to run the pendulum's example but in the MPC constructor, I think there is an error. The output is the following:

`
Traceback (most recent call last):

File "", line 54, in
)(x, QuadCost(Q, p), pend)

File "/home/carlos/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)

File "/home/carlos/anaconda3/lib/python3.7/site-packages/mpc/mpc.py", line 265, in forward
x_init, C, c, F, f, cost, dx, x, u)

File "/home/carlos/anaconda3/lib/python3.7/site-packages/mpc/mpc.py", line 362, in solve_lqr_subproblem
x, u = _lqr(x_init, C, c, F, f if f is not None else e)

File "/home/carlos/anaconda3/lib/python3.7/site-packages/mpc/lqr_step.py", line 112, in forward
Ks, ks, self.back_out = self.lqr_backward(C, c_back, F, f_back)

File "/home/carlos/anaconda3/lib/python3.7/site-packages/mpc/lqr_step.py", line 303, in lqr_backward
x_init=prev_kt, n_iter=20)

File "/home/carlos/anaconda3/lib/python3.7/site-packages/mpc/pnqp.py", line 45, in pnqp
g_[Ic.bool()] = 0.

AttributeError: 'Tensor' object has no attribute 'bool'
`

I really appreciate if you could help me out!

Hello,

I'm running into an error using the "mpc" package, installed via Pip, but in a Conda environment.

When trying to export the environment ("conda env export > env.yaml"), I get this error:

InvalidVersionSpec: Invalid version '1<2': invalid character(s)

The manual fix is to edit the <ENV_BASE_DIR>/lib/python3.8/site-packages/mpc-0.0.4.dist-info/METADATA file, changing:

Requires-Dist: numpy (>=1<2)

to

Requires-Dist: numpy (>=1,<2)

It looks like this might be fixed in the code by changing line 14 of setup.py from 'numpy>=1<2', 'torch' to 'numpy>=1,<2', 'torch'.

Would you be willing to make this change and publish a new version?

Thank you,
Seth

hello mpc.pytorch developer,
first of all, mpc.pytorch is an amazing and fast mpc solver. however, i found there is no constraints for states for input of this mpc solver. although solving was fast, the solution did not satisfy state constraint conditions.
did i miss this feature or it is just not developed yet?
PS: the mpc expression in code does not include state constraints either

 min_{tau={x,u}} sum_t 0.5 tau_t^T C_t tau_t + c_t^T tau_t
                        s.t. x_{t+1} = f(x_t, u_t)
                            x_0 = x_init
                            u_lower <= u <= u_upper

sincerely asking,
sichao

I am trying to design a custom MPC to follow a trajectory. The cost function has the form of

(x-x_i)^T Q_i (x-x_i) + (u-u_i)^T R (u-u_i)

where the x_i is desired value at that point. Is this possible using this framework? if so how would I go about setting up such a cost function?

In (LQRStep.lqr_forward) https://github.com/locuslab/mpc.pytorch/blob/master/mpc/lqr_step.py, we have the following:

if t < T-1:
    if isinstance(true_dynamics, mpc.LinDx):
        F, f = true_dynamics.F, true_dynamics.f
        new_xtp1 = util.bmv(F[t], new_xut)
        if f is not None and f.nelement() > 0:
            new_xtp1 += f[t]
    else:
        new_xtp1 = true_dynamics(
            Variable(new_xt), Variable(new_ut)).data

    new_x.append(new_xtp1)
    dx.append(new_xtp1 - x[t+1])

(line 218 onwards)

This effectively removes the last state from the trajectory. If there is a cost associated with it, however, doesn't that make the gradient computation incorrect? We won't have dx (or the final x) in the backward function of the same class.

Note that further down the same function, the cost is in fact updated:

if isinstance(true_cost, mpc.QuadCost):
    C, c = true_cost.C, true_cost.c
    obj = 0.5*util.bquad(new_xut, C[t]) + util.bdot(new_xut, c[t])
else:
    obj = true_cost(new_xut)

which means the terminal state contributes to the cost, but is dropped from the list of states used elsewhere.

• Jupyter notebook examples of the pendulum and cartpole
• Examples -> Website
• Pendulum control
• Arbitrary gym environment control

Describe:

• Gradient computation modes
• Slew rate
• {detach,exit}_unconverged
• The verbose output columns, with an example