nslyubaykin / relax Goto Github PK

Hello Nikita,

first of all I would like to acknowledge that this is probably the cleanest ml code I have srsly ever seen. Thank you for that!

So I am using FRWR and Random Shooting and I have a question about the horizon.
I am running on limited resources in real-time (gtx1060+win+cuda, overall I would say RS is quite fast, CEM slow, FRWR a bit slow) and I see that the length of the horizon scales worse in regards to the overall performance. So I experimented to lower the iterations/s and reduced f.i. ensemble size to 3, candidate_sequences = 300, horizon = 3-5. So I thought, it would be nicer, to spread the sampling into the future by 2^x (so horizon[1,2,4,8,16] ), maybe also weighting the later even higher - and discarding the intermediate steps for calculations of candidates. So I could span a bigger time horizon with the same number of steps. (Maybe this approach is already used in the mpc community, but I am fairly new to it, so I don't know.)
I see you have a very interesting scheduler system, but I am confused how to initialize an exponential scheduler that lags the horizon in its tensor dimensions. I made a drawing for this.

mpc horizon mod

Update: I tried in the data utils get_next_lag_obs

    lag_obs_split = np.split(lag_obs, 
                             indices_or_sections=1+nlags**2,   # pow  or   =1+int(nlags**1.25)
                             axis=concat_axis)

I am also thinking to convert all lists to np.arrays and jax jit-ing the buffers (and trying hugginface accelerate), but not sure

Also a short quesstion, would it be possible to replace RS/CEM by (NN-approximated) MPPI)? Because they say so. (A combination of MPPI/FRWR)

Anyways, thank you for your beautiful and very instructive codebase! (I am wondering at what / what instance taught this way of ('pure pythonic'?) coding?)

Best

Lee
Living Computation Foundation Member