calliope-project / calliope Goto Github PK

Allow resource parameters for supply techs to temporarily become negative to emulate parasitics or sub-sections of a connected grid

To be fixed by more automation in setup.py:

# Get package data files
package_data = ['config/*.yaml', 'test/common/*.yaml']
for d in glob.glob('example_models/*/'):
    package_data.append(d + '*.yaml')
for d in glob.glob('example_models/*/*/'):
    package_data.append(d + '*.csv')
    package_data.append(d + '*.yaml')
for d in glob.glob('test/common/*/'):
    package_data.append(d + '*.csv')

Need to add conda-forge install of GLPK in the Windows Calliope packages.

• ability to draw values for a specific parameter from a given distribution

Hello,

After testing the ramping constraint on conversion technology by adding:

• in model.yaml : constraints: - constraints.optional.ramping_rate
• in tech.yaml : constraints: e_ramping: 0.1

I observed the error:
ERROR: Rule failed when generating expression for constraint c_ramping_up with index ('boiler', 'L1', Timestamp('2005-01-01 01:00:00')):
OptionNotSetError: Can not get parent for conversion and no default defined (boiler.carrier).

The same error appears for conversion_plus technology but not for supply technology

Thanks!

Currently the function 'get_mean_from_clusters' assumes that a time-varying constraint will vary over (y, t, x). With the addition of time-varying costs (e.g. time-varying export) there are DataSet variables which vary over (y, k, t, x), which cannot be handled.

Looking through time_clustering.py, this is a problem for many of the time clustering functions!

It would be good to create expressions to which each constraint/cost must fit (e.g. text, float, integer, bool). That way, on loading a model, they can all be checked against the relevant expressions, for data consistency. This consistency checker could also check and warn about possible issues (e.g. if a technology is defined, but hasn't mentioned e_cap.max/equals/min so will be ignored, or a MILP tech defining units.max/min/equals hasn't defined e_cap_per_unit, etc.).

In the Abstract base technologies section in configuration_reference.rst

Currently, when mapping transmission, metadata is taken to be [lat, long] and used to populate a basemap figure. When producing fictitious problems, it can be easier to just set the problem up in cartesian coordinates, describing locations as [x, y] instead. Some switch in metadata to identify the information as lat/long or x/y, and corresponding mapping using basemap or just matplotlib is then required.

If there are multiple carriers in or out for a technology (e.g. for CHP in urban scale model, both 'power' and 'heat' are carriers out) then a tuple of those carriers is given in the model.solution.metadata DataArray. This is produced without problem when working with Python interactively, but when working with a solution file, the tuples are all replaced with None.

This is the same part of processing the data that causes issue #54, so we probably want another way of mixing single and multiple carriers in a single DataArray.

There is currently no constraint to stop a conversion_plus tech from producing energy of one of its non-primary carriers at any location that the technology is not allowed. i.e. if a CHP can produce power (primary carrier) and heat (secondary carrier) at X1, but the CHP is not allowed at X2, then there is no constraint to stop the CHP producing heat at X2, even though there is a constraint to stop power production.

Currently the solution file provides the user with the costs associated with a technology as a lump sum. Pyomo offers outputs of fixed and variable costs, but this still doesn't give access to e.g. fuel costs, storage capacity costs, etc.

To get the information, the constraint equations need to be applied in reverse to the results of the optimisation.

• Keep track of source file and line of each setting
• Persist inline comments
• Permit dynamically updating configuration files

The code has an error (ZeroDivisionError: float division by zero) while running the example model. The system information and the full error output as well as the proceeding are listed in the respective txt-files.

System-information.txt
Traceback.txt
Vorgehen.txt

• should Model.data be added to saved files (both pre- and post timeseries adjustment)?
  • possibly as an optional setting
• always return a Model object when reading in a solution?
• harmonise analysis functions to always accept a Model object
• document the key properties of a Model object and which of them are persisted when saving -- e.g. Model.data, Model.solution, Model.m, ...
  • perhaps an overview table with Model properties as rows and the different sources of Model objects as columns

Overly wordy:

    try:
        per_distance = model.get_option(y + '.per_distance')
    except: # assume one unit distance
        per_distance = 1

Something like this should be possible instead:

per_distance = model.get_option(y + '.per_distance', 1)

See #32

For clarity, where applicable, to change names of python variables to more readable values. This applies to parts of the YAML files and of Calliope innards.
e.g. r -> resource

I am Vijay from National institute of Advanced Studies, Bengaluru. I am using calliope as the base for my project. I am currently running a single iteration (one configuration) of the model and saving it as csv.

But I cannot load the entire solution using calliope.read.read_dir('path_to_output'). It raises error while reading iterations.csv

FileNotFoundError: File b'./5Node_2017_base/iterations.csv' does not exist

Is csv based reading only for multiple iterations? if so, perhaps a check should be added before reading. Or am I missing something?

I am using 0.3.7.

Once a new stable sphinx version is available which includes sphinx-doc/sphinx/pull/3675

Consolidation and generalisation of data input from file to account for any variable, rather than just a few. This includes those which are time dependant (e.g. weather data, energy purchase/selling price) or load dependant (e.g. piecewise linearisation of non-linear technology characteristic consumption curves).
Load dependant requires integration of MILP functionality (issue #2).

More a topic for discussion: in the same way as carriers are defined (power, heat, etc.), should there be a fuel.yaml file in which each fuel types is defined (coal, electricity, gas, wind etc.), including aspects such as fuel price (buy & sell) and environmental impact? Saves having to define them individually for each technology.

If you set up a problem as:

reversible_heat_pump:
    name: 'Reversible Heat Pump'
    parent: conversion_plus
    primary_carrier: power #
    carrier_in: gas
    carrier_out:
        power: 1
        heat: 1
    constraints:
        e_cap.max: 1300 # cannot produce more than 1300kWh per hour

Then e_cap.max applies to power (the primary carrier), but not to heat, allowing heat to be produced above the value set by e_cap.max. Requires editing c_prod_max_rule and c_prod_min_rule to consider the sum of all carriers in carrier_out as being limited by e_cap

Hello,

I just discovered that overwriting e_cap.equals for a location isn't working with the 0 value if e_cap.max has been defined in techs.yaml. But it is working with any value above 0.

I am defining e_cap.max: inf in techs.yaml for a conversion_plus technology named conv_tech (the goal is to use it in different locations).
In a specific location in locations.yaml, I am overwriting e_cap.equals for that technology because the capacity if fixed at that location (but could take other values in other locations).

If I overwrite conv_tech.constraints.e_cap.equals: 1, the model works fine and e_cap = 1 in the simulation at that location
But if I overwrite conv_tech.constraints.e_cap.equals: 0, the model is giving a value different from 0 to e_cap at that location.
Finally if I overwrite conv_tech.constraints.e_cap.max: 0, the model works fine and e_cap = 0 in the simulation at that location.

Arnaud

CHP facility produces heat and electricity (CCHP facility also includes cooling), so YAML file needs to allow for multiple carriers to be defined for any technology.

• Spit out .bat files?
• Provide instructions to run with bash on Windows?

Need to fix MatplotlibDeprecationWarning: The axisbg attribute was deprecated in version 2.0. Use facecolor instead.

For higher spatial resolution, the integration of building and network placement with maps. Location coordinates can be loaded from map, as well as any physical constraints to connecting any resource with demand. Output can also be loaded to map for spatial visualisation of results.

Calliope currently defines a number of capacities: s_cap, e_cap, r_cap, rb_cap, r_area. These are decision variables in planning mode. However, they shouldn't be decision variables in operational mode. Currently, there is no change in the constraint generation except that if a .max constraint is given for a capacity, it will be converted to .equals in operational mode, essentially setting it as a parameter.
This means that any capacity given as 'inf' can be changed on each iteration of the rolling horizon. When cost is associated with the capacity, this will be different in each time step (when it should be constant).

In operational mode, the user shouldn't be allowed to define capacity costs unless they have also defined a .equals or .max for that capacity. The ideal case would be that there is a completely different constraint set which removes all capacities as decision variables, having them as parameters instead. But, this still requires that the user has defined .equals or .max for every capacity they want to cost. There's no issue with having an infinite capacity limit, provided there is no cost related to it.

An alternative is to consider only operational costs in operational mode, so all capacity costs are ignored. This makes sense from a minimisation perspective, the technologies can be assumed already purchased so their depreciated and time-weighted capacity costs should not affect the objective.

Raising as issue to discuss best method for addressing. My vote is removing all capacity costs, meaning the user need not define .equals or .max for every capacity and all capacities can be removed as decision variables (set as parameters instead).

Ability for energy to be exported beyond the considered grid, including price

output of one technology acts as the input of another (e.g. heat for the production of cooling, electricity for the production of Hydrogen). Resource availability for the secondary technology will then depend on production by the primary technology (and any transmission between the two).

When generating parallel run on Windows path have backwards slash instead of forward slash

Packaging Calliope in a way to allow non-technical users to test scenarios in it, perhaps a web-based interface?

option_getter should include:

• cost_getter
• cost_per_distance_getter
• any_option_getter
• depreciation_getter

Shows up in test_model_conversion.py and in test_model_export.py, but not always. Running tests over and over again will eventually lead to an exception:

calliope/test/test_model_conversion.py:52: in create_and_run_model
    model.run()
calliope/core.py:1416: in run
    self.load_solution()
calliope/core.py:1588: in load_solution
    self.process_solution()
calliope/core.py:1550: in process_solution
    md = self.get_metadata()
calliope/core.py:1774: in get_metadata
    lambda y: self.get_carrier(y, direction='in', all_carriers=True))
../../../../Anaconda/envs/calliope/lib/python3.6/site-packages/pandas/core/indexes/base.py:2780: in map
    names=attributes.get('name'))
../../../../Anaconda/envs/calliope/lib/python3.6/site-packages/pandas/core/indexes/multi.py:1144: in from_tuples
    arrays = list(lib.tuples_to_object_array(tuples).T)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   TypeError: Expected tuple, got str

tuples contains :

(Pdb) p tuples
array([('power', 'heat', 'gas'), 'resource', 'heat', 'low_T', 'resource',
       'resource', 'power', 'resource', 'resource', 'resource', 'V_low_T',
       'power'], dtype=object)

Would be good to have node label background, so labels aren't obscured by edges, but newtorkx.draw_networkx_labels doesn't have that functionality it seems.

Hello,

I am testing conversion_plus technology after seeing some strange behavior while using it.

I created a model with a power demand, a power supply and a conversion plus technology consuming power to release another carrier.

techs:
    supply_power:
        name: 'Electricity import'
        parent: supply
        carrier: power
        constraints:
            r: inf
            e_cap.max: 100
        costs:
            monetary:
                e_cap: 1
                om_fuel: 50

    conv_tech:
        name: 'conversion tech consuming power'
        parent: conversion_plus
        carrier_in: power    
        carrier_out:   
            name_carrier_out: 0.01
        primary_carrier: name_carrier_out

    demand_power:
        name: 'Electrical demand'
        parent: demand
        carrier: power

    unmet_demand_power:
        name: 'Unmet electrical demand'
        parent: unmet_demand
        carrier: power

In fact the name of the carrier_out has an influence on the result of the model:

• In general, the power supply will provide the power for the demand.
  

• But if the name of the carrier out contains 'power', like for example carrier_out: power_out_conversion: 0.01, or carrier_out: test_power_out_conversion: 0.01 the conversion_plus technology ends up feeding the power demand
  

• If I just change the name of the carrier_out to: carrier_out: test_pow_er_out_conversion: 0.01 the supply is once again feeding the power demand
  

I also changed the primary_carrier every time to match the carrier_out name.

When running the model in a python interpreter, the solution Dataset that is produced directly after running a model has the config_model and config_run as strings, not as AttrDicts. When loading the same solution from file (netCDF), the config_model and config_run are correctly formatted.

This causes an error with plotting transmission, as it gets information from config_model for the location metadata.

This is a recent change, before ~0.5.0 the solution Dataset gave config_model and config_run as AttrDicts.

I don't know if it is a bug or if I misunderstand it but I have some troubles to understand some difference between the behavior of conversion and conversion_plus technologies, especially regarding the secondary carrier and the export option. The model is a bit fictional but here are the code I used for a conversion technology and if you remove the # and change the "conversion" by "conversion_plus", you will have a conversion_plus.

    conversion_techno:
        name: 'Test conversion and conversion_plus'
        parent: conversion #conversion_plus
        export: true
        carrier_in: gas
        carrier_out: 
            heat
        #carrier_out_2:
            #power: 1
        constraints:
            e_cap.max: 1300
        costs:
            monetary:
                om_var: -100.001

I used a om_var for the supply of gas of 1 and a constant demand of -1 for heat and power

It appears that the conversion technology will work at full capacity and produce an excess of heat (because of the negative O&M cost). But on the contrary, the conversion_plus technology will not produce more than the demand, even if it would be more economically interesting to produce at full capacity.

I guess it is because the export option isn't working for one of the 2 carriers in conversion_plus.

Conversion

Conversion_plus

I was trying to use number in scientific notation in the yaml files, but apparently it is read as a string and not a number:

boiler:
    costs:
        monetary:
            e_cap: 1e3

=> TypeError: Cannot convert object of type 'str' (value = 1e3) to a numeric value.

Currently, Calliope works on the assumption that power will be an energy carrier in the system. It is sufficiently ingrained that the construction of the solution file will fail if it isn't included. The metadata summary is also only constructed for that one energy carrier.

In situations where there are multiple energy carriers, the summary is erroneous. In situations where there is no power carrier (e.g. giving it the perhaps more apt title electricity), Calliope fails after solving the optimisation, but before finishing construction of the solution DataArray.

There needs to be a cleaner way of dynamically adapting for multiple energy carriers, particularly in core.Model.get_summary , but I haven't yet worked out how to make it as efficient as the current XArray indexing, as I can only see it being possible by looping over all the technologies.

Perhaps add a section on "Debugging failing models" to docs:

• e.g. "have you set this or that parameter"?

s_time.max is a way to set a maximum storage based on the length of discharge time you want available, rather than setting a capacity limit. The issue with it was that it used a reference efficiency for discharge.

When using a timeseries varying e_eff, it's better to check the situation with worst efficiency, then ensure that your maximum capacity allows discharge over the time periods of that worst efficiency. This update was made in 0.5.0: https://github.com/calliope-project/calliope/blob/master/calliope/constraints/base.py#L387-L427

Currently though, if you change timestep resolution, this function will continue to think that each timestep is worth one unit of time, creating an erroneous result.

Currently, creating a technology with the parent 'storage' and simply defining an s_cap.max for it will not work. The constraints c_es_prod_max_rule and c_es_con_max_rule force the storage facility to produce and consume no energy because e_cap.max isn't defined (and is thus automatically reduced to zero).

e_cap is useful to use to allow a maximum flow of energy to/from the storage facility, but it shouldn't be necessary for storage. I'd suggest saying that if e_cap.max is not defined for a storage facility that e_cap and s_cap is matched. I haven't patched and pulled this because it's probably worth discussing first.

Hello,
I try to override different parameters in the locations.yaml file and it is working fine except for the r constraint of supply technologies.

For example, overriding the r parameter of photovoltaic technology. Here are my locations.yaml and techs.yaml files:

###
# LOCATIONS
###
locations:
  Loc_South_tyrol_2014:
    available_area: 100000000.0
    override:
      demand_power.constraints.r: file
      photovoltaic.constraints.r: file 

    techs:
    - demand_power
    - unmet_demand_power
    - photovoltaic
    - supply_power

###
# TECHNOLOGY
###
techs:
  demand_power:
    carrier: power
    name: Power demand
    parent: demand

  photovoltaic:
    name: 'Solar photovoltaic power'
    parent: supply
    carrier_out: power
    constraints:
      #r: file
      e_eff: 0.85
      e_cap.max: inf
      r_area.max: 7000
    costs:
      monetary:
        e_cap: 5

  supply_power:
    carrier: power
    constraints:
      e_cap.max: .inf
    costs:
      monetary:
        om_fuel: 100.0
    name: Electrical Grid
    parent: supply

  unmet_demand_power:
    carrier: power
    name: Unmet power demand
    parent: unmet_demand

The results is :

Gurobi isn't happy and it is the same with glpk.

But if I comment photovoltaic.constraints.r: file in locations.yaml and uncomment #r: file in techs.yaml, the optimization runs fine.

If I understand it well, both option with r defined in locations.yaml and techs.yaml should give the same results.
The strange part is that overriding the demand_power.constraints.r in locations.yaml is not causing any trouble.

Thanks for your help! 😀

Hello,

I try to overwrite the plant_life and monetary.interest for a technology but it isn't changing the cost-result for that technology.

locations.yaml:

locations:
    L1:
        techs: [
         'supply_power',
         'demand_power',
         'unmet_demand_power'
         ]
        override:
            demand_power.constraints.r: -10
            supply_power.depreciation.plant_life: 4
            supply_power.depreciation.interest.monetary: 0.5

techs.yaml:

techs:
    supply_power:
        name: 'Electricity import'
        parent: supply
        carrier: power
        constraints:
            r: inf
            e_cap.max: inf
        costs:
            monetary:
                e_cap: 10
                om_fuel: 1 # €/MWh electricity price
        depreciation:
            plant_life: 20  # Lifetime of a plant (years)

    demand_power:
        name: 'Electrical demand'
        parent: demand
        carrier: power

    unmet_demand_power:
        name: 'Unmet electrical demand'
        parent: unmet_demand
        carrier: power