tum-ens / hamlet Goto Github PK
View Code? Open in Web Editor NEWHAMLET: Hierarchical Agent-based Markets for Local Energy Trading
License: GNU General Public License v3.0
HAMLET: Hierarchical Agent-based Markets for Local Energy Trading
License: GNU General Public License v3.0
Meters doesnt show the power consumption of the Heat Pump. The Meters.ft file which is one of the Results form the Executor doesnt provide the power consumption of the heat pump
Meters show the power consumption of the Heat Pump.
No response
No response
import sys
sys.path.append("..") # Add the parent directory to the Python path for execution outside of an IDE
sys.path.append("./") # Add the current directory to the Python path for execution in VSCode
from hamlet import Creator
from hamlet import functions as f
import os
# Path to the scenario folder (relative or absolute)
path = "../02_config/scenario_pv_60_hp_60_ev_60_transition"
# Create the creator object
sim = Creator(path=path)
# Create the scenario
sim.new_scenario_from_grids()
# get all agents in this scenario
sfh_agents = f.get_all_subdirectories("../04_scenarios/scenario_pv_60_hp_60_ev_60_transition/agents/sfh")
# now find the agent at grid bus 20136, since it has 3 EVs with different time series files according to electricity.xlsx
for sfh_agent in sfh_agents: # iterate each agent to find agent at bus 20136, it has multiple EVs
agent_path = os.path.join("../04_scenarios/scenario_pv_60_hp_60_ev_60_transition/agents/sfh", sfh_agent)
agent_config = f.load_file(os.path.join(agent_path, "account.json")) # load agent config to get the agent's bus
if agent_config['general']['bus'] == 20136: # find agent at bus 20136, print config for EVs of this agent
print("Agent at bus 20136 has agent id " + sfh_agent + ", it has multiple EVs: \n")
agent_plants = f.load_file(os.path.join(agent_path, "plants.json")) # load plants config for this agent
for plant in agent_plants.values(): # loop plants of this agent to find EVs
if plant['type'] == 'ev':
print(plant)
break
If an agent has multiple plants of the same type (e.g. EV), the time series files for all these EV plants are always the same as the first plant when creating a scenario from grid, even though they have different time series files according to the grid file (electricity.xlsx).
The time series files should be individual for each plant as defined in the grid file (electricity.xlsx).
No response
No response
Tool does not contain an analyzer yet. Therefore, an implementation starting with an analyzer architecture is necessary.
Agents with only inflexible load will get in the executor this error message in the executor : (hope this is not only for me the case)
File "C:\Users\claud\PycharmProjects\HAMLET-develop\hamlet\executor\utilities\database\database.py", line 195, in get_bids_offers
bids_offers_table = self.filter_market_data(market=bids_offers_table, by=by, values=values, inclusive=True)
File "C:\Users\claud\PycharmProjects\HAMLET-develop\hamlet\executor\utilities\database\database.py", line 288, in filter_market_data
dtype = datetime_index.dtypes[0]
File "C:\Users\claud\anaconda3\envs\hamlet\lib\site-packages\polars\lazyframe\frame.py", line 672, in dtypes
return self._ldf.dtypes()
exceptions.ColumnNotFoundError: timestep
Some of the columns are still using the old naming format in the creator.
Change all column names accordingly, e.g. "cop_file" to "file_add".
Currently, the aggregator is included in the config file but not yet implemented. For the implementation it needs to be checked how the config needs to look like to actually implement it, it needs to be considered in the scenario creation and execution. And of course the analysis once this exists.
the path separator is now hard-coded in creator so the code is not working when running on other OS, should be adjusted correspondingly
Based on #1
Timezone is not computed based on config_general time info but hard-coded (see code snippet below). Change so that it uses config_general instead.
weather.index = dtime.tz_convert("Europe/Berlin")
Bug in:
setup.py: __execute_agents_parallel() and __execute_agents()
Line where bug is found:
# Get the data of the agents that are part of the tasklist
agents = self.database.get_agent_data(region=tasklist[0, 'region'])
Issue
The following line returns a dict with the keys being the agent IDs. The dict should contain as keys the agent types and as subkeys the agent IDs.
Implement chp. Lemlab's implementation can be used for inspiration.
This is lemlab's config setup for chp:
"chp_fraction": 0 # fraction of prosumers with chp
"chp_sizing_power": 1 # thermal sizing of chp plant
# 1 = thermal power of chp equals ceiled max heat demand
# note: if value <1 without other heating devices, the simulation will crash
"chp_heat_elec_ratio": 2 # ratio of heating to electric power (p_th/p_el)
"chp_efficiency": 0.9 # efficiency of chp plant: (e_th + e_el)/e_tot
"chp_capacity": [2, 3] # storage capacity in hours of chp power
# note: value randomly chosen from list
"chp_soc_init": 0 # initial soc of the storage (0-1)
"chp_tes_efficiency": 0.98 # charging and discharging efficiency of thermal energy storage
"chp_fcast": "perfect" # heating demand forecasting technique
# "perfect" - perfect forecast of heating demand
# "nn" - neural network based on the included weather data
# "mr" - multiple regression based on the included weather data
"chp_fcast_retraining_period": 86400 # models should be periodically retrained based on new data
# how many seconds between retraining periods?
"chp_fcast_update_period": 3600 # forecasts are periodically updated and saved to file where the
# most current forecast can be retrieved by model predictive controllers
# and market agents. How many seconds between updates?
They are only applied to the current timestamp settlement but previous market clearings for the same timestep need to be also considered and feed and levies only applied to the net energy amount.
Create separate files for the subclasses of grids, agents and markets in the creator.
Currently the task list contains the type of coupling, e.g. above_l. This needs to be replaced with just above when it is appropriate and in other instances with None. This way the market knows when coupling is necessary and when not without having to check it for itself. It only retrieves the other bids and offers if the row of the coupling column contains some other than None.
Currently the forecaster class inits the forecaster to train all methods. However, it does so only for the plants and not the markets. This needs to be added.
Furthermore, the class requires a function that creates all the forecasts for each plant and market. I have added it to the class with some comments but feel free to change it however you like. I also added more variable to the init. Just let me know if you don't need some of them.
def make_forecasts(self):
"""Make forecasts for all plants."""
# TODO: @Jiahe, please implement this method
# Loop through all forecast columns (probably best to sort this by plant ID and market type)
# Check if the model needs to be re-fitted
# Make the forecast
# Return the forecasts dataframe (i.e. self.forecasts)
return self.forecasts
Each forecast method for each device gets its own entry in the config file.
Change the creator so that it can work with the new feature.
Example:
fcast:
method: naive_average # household load forecasting method
# options:
# - perfect: perfect knowledge of the future
# - naive: today will be the same as yesterday
# - naive_average: today will be the same as the average the last n days
# defined in naive_average_days
# - smoothed: prediction value is a moving mean of the future values
# with the window width defined in smoothed_timesteps
# - sarma: full sarma model with the order defined in sarma_order
perfect:
naive:
naive_average:
days: 2
smoothed:
timesteps: 9
sarma:
order: [2, 0, 2, 2, 0, 0, 96, 2, 0, 0, 672]
# order of double seasonal arma model
# order: [s2_ar, 0, s2_ma, s2]
I'm not sure if the error is caused by MacOS but I got the following error while creating scenario (both multi-market and single-market):
Let me know if you get the same error while calling 01 - examples/01_creating_scenarios.py @TUM-Doepfert . If not, I'll check if it's a MacOS problem.
When running the Executor, following progress bars appears:
But actually there should be only one progress bar. If I delete from hamlet.creator.setup import Creator
in HAMLET\hamlet\__init__.py
, I got the right version of the progress bar:
My assumption is that it's because the progress bar in class Creator is initialized outside of the __init__()
function of the Creator:
But if I move progress_bar = tqdm()
into the def __init__()
as self.progress_bar = tqdm()
, I got the following error while creating the scenario:
All entries are added twice to the market transactions.
Currently, the tool uses concurrent.futures thread-functionality to run processes in parallel. However, this does not speed up the process as the GIL hinders the processes to actually be executed in parallel. Therefore another approach needs to be used.
Include the functionality to create the generation time series for PV and wind using the spec json files. The function to be used are labeled with @jiahe and are found in creator/agents/agents.py class Agents.
This issue contains all the forecasting methods and to which plants they apply:
(TBD)
The tables that are defined in the scenario creator still need to be assigned their dtype as currently they are still floats and objects while they can easily be turned into ints.
Include the creation of a cop and/or power timeseries based on the specifications json file. The function should return a pandas dataframe with the index as unix timestamp and the columns named after what they contain, e.g. 'cop'.
The function is in the Agents class named '__timeseries_from_specs_hp'.
All auxiliary functions should also be included in this class.
Sells the energy that it bought previously on the market as balancing energy and uses flexibility instead.
# Make all plants controllable
self.df[f"{key}/sizing/controllable_{num}"] = True
All plants are made controllable in the Creator in function def _pv_grid at the sfh file
Not All plants are made controllable in the Creator
No response
No response
Some of the parameters only add unnecessary complexity which are not required in a simulation environment. Therefore, the functionality is not needed and the parameters should be taken out.
The parameters to take out are:
The database should contain methods that allow to obtain the entire general info, the weather as well as the market dict from the respective region (see #23 for the structure). The entire general info can then also be used to obtain the weather data as it is a subdict.
def get_general_data() -> dict:
# code
return data
def get_weather_data() -> dict:
# code
return data
def get_market_data(region: str) -> dict:
# code
return data
Bug in:
setup.py: __execute_agents_parallel() and __execute_agents()
Line where bug is found:
# Get the data of the agents that are part of the tasklist
agents = self.database.get_agent_data(region=tasklist[0, 'region'])
Issue
The following line returns a dict with AgentDB objects. However, I cannot inspect the objects and see what information they contain. When calling something like agents['sfh']['B0gUAuzRUlrkye1']
I would expect to see a dict of dicts instead of <hamlet.executor.utilities.database.agent_db.AgentDB object at 0x000002A4C0241300>
I got the following error when creating the single market scenario:
I opened the debugger to figure out how to fix the error. I noticed at line 507 in setup.py (where the error occurs) self.grids
is an empty dict. Then I went to lines 488-502 in setup.py, the following code defines self.grids
:
Here, path_grids
is defined as 'HAMLET/04 - scenarios/example_single_market/grids'
. But this folder doesn't exist at all. So at line 495, when using os.walk(path_grids)
to define self.grids
, we will get an empty dict for self.grids
because nothing was looped through.
Let me know if this error while creating grids also occurs on your laptop or if I misunderstood something.
If the input format of the demand/generation in the pandapower Excel file is too long, it cannot be casted as integer. Correct the line for all agent types and devices. The line looks most often like this:
old:
self.df[f"{key}/sizing/power_{num}"] = (self.df.index.map(df['power']) * 1e6).astype('Int64')
new (add round()):
self.df[f"{key}/sizing/power_{num}"] = (self.df.index.map(round(df['demand'] * 1e6))).astype('Int64')
Currently all timeseries files have timestamps instead of datetime dtypes. This decreases the functionality of the timeseries and requires to write more complicated and less standard code. To right more standardized code, the files need to be changed.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.