johbrust / simulink_gym Goto Github PK

Compatibility with Stable-Baselines3 and rllib required Gym version 0.21.0. Once (most of) these libraries are compatible with newer version of Gym, this wrapper will also be updated.

Quire task #113

Hello,

I have been working with the simulink_gym project, specifically the cartpole_simulink example, to train a reinforcement learning agent using the PPO strategy. However, I observed that the training process is approximately 20 times slower compared to the standard Gym implementation of CartPole-v1. I don't know if this is normal.

Is this level of performance discrepancy expected due to the nature of the integration between Python and Simulink, or is there a potential optimization issue that could be addressed? Additionally, are there any recommended approaches or modifications to enhance the performance of Python+Simulink setups for reinforcement learning applications?

Any insights or suggestions you could provide would be greatly appreciated as I continue to explore this integration for control systems training.

Thank you!

Hello,
I am trying to run a model which uses a variable step solver.
The problem is that at certain time steps, the simulation results are numerically very high/low and then they get back to the expected level in the following simulation steps.
I think that this might have something to do with the fact that python receives states at the exactly same simulation-timestamp and therefore the variable step solving procedure gets mixed up.
Furthermore, I noticed that the output time steps are also mixed up sometimes.

I already set the priority of TCP/In to 1 and TCP/Out to 2.
Running the model with a constant input just within Simulink works well.
Furthermore, I set the sample time of the TCP/In and TCP/Out to inherit, i.e. -1

Do you have any idea how this issue can be solved?

I have successfully run the cartpole_simulink.ipynb file. Now I have two questions:

1. In the step method in cartpole_simulink.py, I find you define action = int(action) in line 119. Is it due to just the action.space requirentment or the TCP/IP sending requirements. If I train a RL model with float type action, will it have any problem?
2. In the cartpole example, the dimension of action is only one. If I have action with multiple dimensions, will there be any fix in simulink_block_lib? Can you help to specify how to fix it? Or suggest to add an example with more than one dimension with float dtype.
   Thanks.

Thank you for this gym wrapper for simulink. However, when I python setup.py install it, and runs the cartpole example, it shows:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 from cartpole_simulink import CartPoleSimulink

File /media/lk/disk1/MATLAB/matlab_py/simulink_gym/examples/envs/cartpole_simulink/cartpole_simulink.py:1
----> 1 from simulink_gym import SimulinkEnv, Observation, Observations
      2 from gym.spaces import Discrete
      3 from pathlib import Path

File /media/lk/disk1/MATLAB/matlab_py/simulink_gym/simulink_gym/__init__.py:7
      5 from gym import logger
      6 from gym import spaces
----> 7 from .environment import SimulinkEnv
      8 from .observations import Observation, Observations
      9 from .utils import BlockParam

File /media/lk/disk1/MATLAB/matlab_py/simulink_gym/simulink_gym/environment.py:4
      2 import matlab.engine
      3 import gym
----> 4 from simulink_gym import logger, SIMULINK_BLOCK_LIB_PATH
      5 import threading
      6 import numpy as np

ImportError: cannot import name 'SIMULINK_BLOCK_LIB_PATH' from partially initialized module 'simulink_gym' (most likely due to a circular import) (/media/lk/disk1/MATLAB/matlab_py/simulink_gym/simulink_gym/__init__.py)

I can successfully run the cartpole_simulink.ipynb. However, I find the sampling interval is 0.02s by checking info['simulation time [s]']. Does the interval depend on the tcp/ip sending frequency or be possible to set manually ? How can set another interval, like 1s. Thank you.