Giter VIP home page Giter VIP logo

microgridrlsimulator's Introduction

Installation

Be sure you have python 3.6 installed. Run install.sh. This creates the python environment mgenv with the necessary requirements to run the application.

To install as a python package, run pip install -e .

Gym wrapper

You can use the simulator as a Gym environment, see examples/env_demo.py

Running the application

Be sure you are using the python environment mgenv (source menv/bin/activate).

Get help::

python -m microgridRLsimulator -h

Run the application::

python -m microgridRLsimulator --from_date 20150101T00:00:00 --to_date 20150102T00:00:00 --agent DQN --agent_options agent_options.json case1

This runs the case1 examples located in folder examples/data using a DQNAgent, that is a agent that uses Deep Q learning.

Documentation

You can generate this documentation yourself:

::

cd <to the root of the project>
cd docs; make html; cd ..

The html doc is in _build/html

microgridrlsimulator's People

Contributors

omardrwch avatar bcornelusse avatar jcboukas avatar

Stargazers

 avatar  avatar Jamal Aldahmashi  avatar Kang avatar  avatar Shubham Pachori avatar  avatar Carlos Silva avatar  avatar  avatar Rafael C.P. avatar Henrique B Morari avatar Code_in_process avatar Sergio A. Dorado-Rojas avatar David Domínguez Barbero avatar Sayan Goswami avatar  avatar  avatar Noé Malais avatar  avatar d3sm0 avatar Mohammed Alabdullah avatar  avatar  avatar  avatar Damien A. avatar  avatar

Watchers

James Cloos avatar d3sm0 avatar Castronovo Michael avatar  avatar  avatar Michal Valko avatar  avatar

microgridrlsimulator's Issues

Range of state space

Check definition of the observation space in microgridRLsimulator/gym_wrapper/microgrid_env.py:

high = 1e3*np.ones(2 + len(self.simulator.grid.storages))
self.observation_space = spaces.Box(-high, high, dtype=np.float32)

I put all variables between -1e3 and 1e3 but I'm not sure if this is ok.

bug in `state_formatting` in microgridRLsimulator/gym_wrapper/microgrid_env.py

Hi !
In my code I create an instance of MicrogridEnv:

from microgridRLsimulator.gym_wrapper import MicrogridEnv
env = MicrogridEnv(from_date,to_date, case)

Then, no matter which action I take, I get this error

  File "./abstract/manager/manager.py", line 100, in _train_simulate
    o_r_d_i = env.step(action)
  File "/home/da/Documents/emploi/postdoc/2018_2019/dev/rl/venv/lib/python3.6/site-packages/gym/core.py", line 304, in step
    observation, reward, done, info = self.env.step(action)
  File "/home/da/Documents/emploi/postdoc/2018_2019/dev/rl/venv/src/microgridrlsimulator/microgridRLsimulator/gym_wrapper/microgrid_env.py", line 62, in step
    state_formatted = self.state_formatting(state)
  File "/home/da/Documents/emploi/postdoc/2018_2019/dev/rl/venv/src/microgridrlsimulator/microgridRLsimulator/gym_wrapper/microgrid_env.py", line 91, in state_formatting
    delta_t = state_array[n + 2]
IndexError: list index out of range

It seems that it comes from these functions:

    def state_refactoring(self, state):
        """
        Convenience function that flattens the received state into an array

        :param state: State of the agent as a list
        :return: Flattened representation of the state as an array
        """
        consumption = state[0]
        storages_soc = state[1]
        production = state[2]
        delta_t = state[3]
        state_array = np.concatenate((np.array([consumption]), np.array(storages_soc).reshape(-1), np.array([production]), np.array([delta_t])),
                                     axis=0)
        return state_array

    def state_formatting(self, state_array):
        """
        Inverse of state_refactoring
        """
        n= len(self.simulator.grid.storages)
        consumption = state_array[0]
        storages_soc = list(state_array[1:1+n])
        production = state_array[n + 1]
        delta_t = state_array[n + 2]
        state = [consumption, storages_soc, production, delta_t]
        return state

Indeed, here is the print of state_array

[3.52440585725, [6.0, 6.0], 0.0, 0.0]

which is a list of : a float, a list of two floats and two other floats. Since in state_formatting function, n=2, it is not possible to access to delta_t = state_array[n + 2]. That's probably because next_state is not flatten in step function

    def step(self, action, state=None):
        """
        Step function, as in gym.
        May also accept a state as input (useful for MCTS, for instance).
        """
        assert self.action_space.contains(action), "%r (%s) invalid" % (action, type(action))

        if state is None:
            state = self.state
        state_formatted = self.state_formatting(state)
        next_state, reward, done = self.simulator.step(high_level_action = action, state = state_formatted)
        self.state = self.state_refactoring(next_state)

        return np.array(self.state), reward, done, {}

A possible fix could be to modify state_formatting function into:

    def state_formatting(self, state_array):
        """
        Inverse of state_refactoring
        """
        consumption = state_array[0]
        storages_soc = list(state_array[1])
        production = state_array[2]
        delta_t = state_array[3]
        state = [consumption, storages_soc, production, delta_t]
        return state

Any idea ? :) Thanks !

Request for your code

Hello, I recently read your article "Lifelong control of off-grid microgrid with model-based reinforcement learning" , I benefit a lot from it. I wonder that the "D-dyna" agent is not include in the code?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.