Data consistency checker about calliope HOT 10 CLOSED

Possibly with type comments in YAML? e.g. https://www.python.org/dev/peps/pep-0484/#type-comments

ruamel_yaml can read comments: https://yaml.readthedocs.io/en/stable/detail.html#round-trip-including-comments

from calliope.

Yup, e.g.:

ruamel_yaml.round_trip_load(open('calliope/config/defaults.yaml','r'))['techs']['defaults']['constraints'].ca.items['force_r'][2].value

returns:

'# Forces this technology to use all available ``r``, rather than making it a maximum upper boundary (for production) or minimum lower boundary (for consumption)\n'

We could prepend all comments with types, in this case the output would then be e.g.:

'# type: bool. Forces this technology to use all available ``r``, rather than making it a maximum upper boundary (for production) or minimum lower boundary (for consumption)\n'

Then on loading a model there is a type dictionary, taken from defaults that can be used in the consistency checks

from calliope.

Preference for one of these?

string = '# type: bool. Forces this technology to use all available ``r``, rather than making it a maximum upper boundary (for production) or minimum lower boundary (for consumption)\n

option 1 (requires additional package):

import re
type = re.findall(r'type: (.*)\.', string)

option 2 (will throw up errors if things aren't as expected in string):

type = string.split('type: ', 1)[1].split('.', 1)[0]

from calliope.

I think option 1.

What happens where currently there can be one of several types, e.g. float or bool in e_ramping? Set to 0 instead of False to disable?

from calliope.

Create a list? e.g. type: [False, bool]...

That would allow us to also set a list for e.g. r_unit of ["power", "energy"] rather than saying it has to be str

from calliope.

I think that's confusing as it suggests that the type is a list. Better to have a type and stick to it. And possibly separately specify allowed options. Maybe adding all this information to the YAML comments is going too far...

from calliope.

True, I guess there could be 'type: str. options: ["power", "energy"]'. Might be going too far, but that's where it should be for reference anyway! It would make the options that much clearer

from calliope.

So the comment format might be something like type: str. options: ["power", "energy"]. description: ... - this could be parsed, but is starting to look a bit messy, so perhaps a better solution would be to add metadata separately, e.g. via a separate description key for each entry in defaults.yaml, à la e_ramping: ..., e_ramping_metadata: ...?

from calliope.

Or to have a duplicate of defaults.yaml which is called something like metadata.yaml, in which we can include: type, options, valid technology sets?

from calliope.

Closing as this being worked on now with https://github.com/calliope-project/typedconfig

from calliope.