Giter VIP home page Giter VIP logo

units's Introduction

Unit definitions for integrated-assessment research

PyPI version

Build status

© 2020–2023 IAM-units authors; licensed under the GNU GPL version 3.

The file definitions.txt gives Pint-compatible definitions of energy, climate, and related units to supplement the SI and other units included in Pint's default_en.txt. These definitions are used by:

and may be used for research in integrated assessment, energy systems, transportation, or other, related fields.

Please open a GitHub issue or pull request to:

  • Add more units to definitions.txt.
  • Add your usage of iam-units to this README.
  • Request or contribute additional features.

Usage

iam_units.registry is a pint.UnitRegistry object with the definitions from definitions.txt loaded:

>>> from iam_units import registry

# Parse an energy unit defined by iam-units
>>> qty = registry('1.2 tce')
>>> qty
1.2 <Unit('tonne_of_coal_equivalent')>

>>> qty.to('GJ')
29.308 <Unit('gigajoule')>

To make the registry from this package the default:

>>> import pint
>>> pint.set_application_registry(registry)

# Now used by default for pint top-level classes and methods
>>> pint.Quantity('1.2 tce')
1.2 <Unit('tonne_of_coal_equivalent')>

Warnings

iam_units overwrites Pint's default definitions in the following cases:

pint default iam_units Note
'kt' = knot [velocity] 'kt' = 1000 metric tons 'kt' is commonly used for emissions in the IAM-context.

Technical details

Emissions and GWP

The function convert_gwp() converts from mass (or mass-related units) of one specific greenhouse gas (GHG) species to an equivalent quantity of second species, based on global warming potential (GWP) metrics. The supported species are listed in species.txt and the variable iam_units.emissions.SPECIES.

The metrics have names like <IPCC report>GWP<years>, where <years> is the time period over which heat absorption was assessed. The supported metrics are listed in the variable iam_units.emissions.METRICS.

>>> qty = registry('3.5e3 t').to('Mt')
>>> qty
3.5 <Unit('megametric_ton')>

# Convert from mass of N2O to GWP-equivalent mass of CO2
>>> convert_gwp('AR4GWP100', qty, 'N2O', 'CO2')
0.9275 <Unit('megametric_ton')>

# Using a different metric
>>> convert_gwp('AR5GWP100', qty, 'N2O', 'CO2')
1.085 <Unit('megametric_ton')>

The function also accepts input with the commonly-used combination of mass (or related) units and the identity of a particular GHG species:

# Expression combining magnitude, units, *and* GHG species
>>> qty = '3.5 Mt N2O / year'

# Input species is determined from *qty*
>>> convert_gwp('AR5GWP100', qty, 'CO2')
1.085 <Unit('megametric_ton / year')>

Strictly, the original species is not a unit but a nominal property; see the International Vocabulary of Metrology (VIM) used in the SI. To avoid ambiguity, code handling GHG quantities should also track and output these nominal properties, including:

  1. Original species.
  2. Species in which GWP-equivalents are expressed (e.g. CO₂ or C)
  3. GWP metric used to convert (1) to (2).

To aid with this, the function format_mass() is provided to re-assemble strings that include the GHG species or other information:

# Perform a conversion
>>> qty = convert_gwp('AR5GWP100', '3.5 Mt N2O / year', 'CO2e')
>>> qty
927.5 <Unit('megametric_ton / year')>

# Format a string with species and metric info after the mass units of *qty*
>>> format_mass(qty, 'CO₂-e (AR5)', spec=':~')
'Mt CO₂-e (AR5) / a'

See Pint's formatting documentation for values of the spec argument.

Data sources

The GWP unit definitions are generated from the package globalwarmingpotentials. The version of that package used to generate the definitions is stated in the variable iam_units.emissions.GWP_VERSION.

See DEVELOPING.rst for details on updating the definitions.

Currency

iam_units defines deflators for:

  • USD (United States dollar) for annual periods from 2000 to 2022 inclusive.
  • EUR (Euro) for the periods 2005, 2010, 2015, and 2020 only.

These can be used via pint-compatible unit expressions like USD_2019 that combine the ISO 4217 alphabetic code with the period.

To enable conversions between different currencies, use the function configure_currency():

>>> configure_currency(method="EXC", period="2005")

# Then, for example
>>> qty = registry("42.1 USD_2020")
>>> qty
42.1 <Unit('USD_2020')>

>>> qty.to("EUR_2005")
26.022132012144635 <Unit('EUR_2005')>

Currently iam_units only supports:

  • period-average exchange rates for annual periods (method="EXC");
  • period="2005"; and
  • the two currencies mentioned above.

Contributions that extend the supported currencies, methods, and periods are welcome.

Tests and development

Use pytest iam_units --verbose to run the test suite included in the submodule iam_units.test_all. See DEVELOPING.rst for further details.

units's People

Contributors

danielhuppmann avatar francescolovat avatar khaeru avatar phackstock avatar pitmonticone avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

units's Issues

Conversion of monetary units

As discussed in other contexts, it would be highly useful to add a module to convert between different currencies taking into account deflation and the year (or more granular?) of conversion.

I.e., converting from 2010-USD to 2020-EUR would yield different results depending on whether you first deflate from 2010 to 2020 and then convert, or vice versa.

Establish a Python package

Among other reasons: iiasa/message_data#115 showed that the use of unsynced versions of definitions.txt in two different packages can clash. For me, that's enough headache to make the work of packaging worthwhile.

#13 and #14 also seem like they may need fixes that consist of code rather than changes to definitions, so this will be a pre-requisite.

Allow gwp-conversion between C and CO2 with metric=None

This is a similar issue to #20, but refers to the conversion between C (carbon) and CO2 (carbon dioxide).

I would expect either of these two options to work:

  1. Using GWP-conversion without a metric.
    iam_units.convert_gwp(None (1, 't'), 'CO2', 'C')
    
  2. or using direct conversion in the registry.
    iam_units.registry('1 t C').to('t CO2')
    

Adjust for pint 0.20

CI workflows for message_data (private) fail with messages like the following. Adjust.

Run pytest message_data \
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.8/x64/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 190, in console_main
    code = main()
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 148, in main
    config = _prepareconfig(args, plugins)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 329, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/helpconfig.py", line 103, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1058, in pytest_cmdline_parse
    self.parse(args)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1346, in parse
    self._preparse(args, addopts=addopts)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1248, in _preparse
    self.hook.pytest_load_initial_conftests(
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
    return outcome.get_result()
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1125, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 557, in _set_initial_conftests
    self._try_load_conftest(anchor, namespace.importmode, rootpath)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 574, in _try_load_conftest
    self._getconftestmodules(anchor, importmode, rootpath)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 603, in _getconftestmodules
    mod = self._importconftest(conftestpath, importmode, rootpath)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 651, in _importconftest
    self.consider_conftest(mod)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 732, in consider_conftest
    self.register(conftestmodule, name=conftestmodule.__file__)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 496, in register
    self.consider_module(plugin)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 740, in consider_module
    self._import_plugin_specs(getattr(mod, "pytest_plugins", []))
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 747, in _import_plugin_specs
    self.import_plugin(import_spec)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/_pytest/config/__init__.py", line 774, in import_plugin
    __import__(importspec)
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/message_ix_models/__init__.py", line 4, in <module>
    from iam_units import registry
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/iam_units/__init__.py", line 18, in <module>
    registry.load_definitions(str(Path(__file__).parent / "data" / "definitions.txt"))
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pint/facets/plain/registry.py", line 521, in load_definitions
    for definition in self._def_parser.iter_parsed_project(parsed_project):
  File "/opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/pint/delegates/txt_defparser/defparser.py", line 84, in iter_parsed_project
    raise stmt
pint.delegates.txt_defparser.common.DefinitionSyntaxError: Cannot define 'vkm' (AliasDefinition): the alias v km is not a valid unit alias (must not contain spaces)
    73,0-73,23 @alias vkm = vkt = v km
    /opt/hostedtoolcache/Python/3.10.8/x64/lib/python3.10/site-packages/iam_units/data/definitions.txt

pint can't handle kilotons (kt)

pint can handle common orders of magnitude, but the term kt is defined as 'knot' rather than interpreted as 'kiloton'. As kt is quite common for emissions, I suggest to remove ' knot' as a unit when importing the ism-units definitions.

To replicate the error, try:

pint.UnitRegistry()('1000 kt').to('Mt')

Expected output: 1 Mt

Refactor defs to avoid defining non-quantities

This issue is to carry forward discussion from in #7 (comment), #9, and #10.

Details

The goal in using pint for unit handling (in the 'client' packages of this repo) is to be scientific and rigorous about measurement.

  1. One way pint helps us do this is by ensuring common definitions are used, rather than constants hardcoded in many places across other code.
  2. Another way is ensuring precision about what is being measured, and forcing implicit conversions to become explicit.

Per the second point, we should follow, rather than circumvent, the logic of SI, as encoded by pint. The International vocabulary of metrology (VIM) is the best reference here, defining e.g.:

quantity:

property of a phenomenon, body, or substance, where the property has a magnitude that can be expressed as a number and a reference.

quantity value:

number and reference together expressing magnitude of a quantity.

EXAMPLE 2 Mass of a given body: 0.152 kg or 152 g

EXAMPLE 8 Mass fraction of cadmium of a given sample of copper: 3 μg/kg or 3 ×10^-9

nominal property:

property of a phenomenon, body, or substance, where the property has no magnitude.

EXAMPLE 1 Sex of a human being.
EXAMPLE 2 Colour of a paint sample.

EXAMPLE 5 Sequence of amino acids in a polypeptide.

NOTE 1 A nominal property has a value, which can be expressed in words, by alphanumerical codes, or by other means.

In "quantity value" example 8, note that nominal properties and other information about the quantity value are supplied together with the quantity value:

  • The "phenomenon, body, or substance" is "a given sample of copper".
  • The quantity value is for the mass fraction of a specific species ("cadmium").

These are essential information. But only "µg/kg" is the unit (reference) in this quantity value.

As a convention in many disciplines, we often mix these things together, e.g.:

  • "vehicle kilometres," "passenger kilometres": in both cases the units are length. Nominal information about the measured phenomenon is "travelled by a {vehicle,person}," sometimes "in a certain period of time."
  • GWP (per links above): the unit is mass. The nominal information is that it is the mass of CO₂ that causes global warming equivalent to a given mass of another species.

tl;dr:

  • Nominal properties are distinct from units.
  • Pint is designed to deal with units.
  • Forcing pint to handle nominal properties is always going to be awkward.
  • We should try to avoid this.

Actions

  • As far as possible, implement definitions and conversions using pint features and only physical quantities.
  • Provide example code (e.g. in tests) for how to use these to do common conversions between the notional units (i.e. actual units plus nominal properties) that we commonly use.
  • For each of the categories of quantities in this repo, provide clear description that helps users understand metrology better.

"Unknown conversion specified" in test suite with pint 0.18

A new version of pint (0.18) was released last week (2021.10.26), which caused the pyam tests connected to units to fail.
An easy and quick fix would be to pin the version of pint <= 0.17.
Additionally, it might be a good idea to set up nightly tests that run at least once a week so that we catch these dependency changes.

CO2-eq units (plus design decisions)

We were getting quite deep in the weeds in a conversation over #7. My suggestion is that we move that to an issue to hash out opinions that are not directly related to the implementation in the PR.

@khaeru opined (with example implementation here):

My comments to @danielhuppmann, repeated here for the benefit of others:

In the case of GWP, thinking out loud:

  • Warming per se is measured as [temperature] e.g. of global mean surface air.

    • It can also be measured as radiative forcing, e.g. [power] / [length]² (W/m²).
  • These changes (measured in [temperature] or [power]/[length]²) are caused by (under certain assumptions) a certain amount of carbon.

  • Carbon in turn can be measured in [mass] or [substance].

  • So one way (not the common one, I'll get to that) to define a context would be like this. I am copying here the syntax used by Pint:

    [mass] -> [temperature]: value * gwp
    
  • In this case, one supplies the gwp for carbon per se, i.e. in [mass]/[temperature], e.g. kelvin per gigatonne (the exact units don't matter). (Also, per this, maybe the parameter should be a for radiative efficiency; TBD.)

  • Other GHGs are converted to [temperature] by their own gwp parameter.

  • In layman's thinking, GWP is roughly [mass] of "CO2 that would cause equivalent warming to a certain [mass] of CH4".

  • i.e. these both have the same physical quantity [mass], but they describe different things:

    • One is a certain amount of CH4.
    • The other is a hypothetical amount of CO2 that would cause (…etc.)
  • To convert [mass] of CH4 to [mass] of "CO2 that would cause (…etc.)", one could:

    1. Convert [mass] CH4 to [temperature] using the gwp for CH4.
    2. Convert [temperature] to [mass] using the gwp for CO2.
  • This conversion is clear and explicit because the [temperature] is the same measurement of the same system.

I would prefer to use some variant of this approach, rather than what's in this commit, because it avoids abusing Pint by introducing physical quantities that are not actually physical quantities. (Using the definition “A physical quantity is a property of a material or system that can be quantified by measurement,” [mass] or [substance] (number of molecules) are properties of a system, e.g. a certain lump of carbon. “[carbon]” is the system itself, not a property thereof.)

@znicholls identified and already implemented solution here

@danielhuppmann expressed some reservations regarding the heaviness implied on using that approach.

The conversation to be decided here is:

  • do we have a preferred approach?
  • if we like the approach by scmdata, are we ok with derivative effects? (moving units to a package)

Test GWP conversions with vectors

Pint supports wrapping np.ndarray and other classes that, in turn, wrap ndarray. This allows calculations—e.g. applying conversion factors—to be vectorized by fast, low-level code, while higher-level code (pint) supplies those factors and handles unit descriptions. Packages using this repo should need to do very little to perform vectorized conversions.

test.py should be expanded to check that this mode of operation works for the definitions in this repo. This can also serve as a simple example of how to efficiently convert units of vector data.

Follow-up for #10.

Add non-greenhouse gas emission species

While working on some sanity-checks for the new IAMC project templates, I realized that several emissions species are relevant for IAM reporting but are not included in the list of emissions - because they are not greenhouse gases.

I suggest to add them anyway here so that pyam and a unit-validation-schema can directly build on ism-units (instead of requiring a separate solution).

I was trying to use the following line to check whether a unit like "Mt BC/yr" is valid.

iam_units.emissions.pattern.split(unit)

This fails for the following emissions species:
BC, CO, NH3, NO2, OC, SO2, VOC, HFC43-10

Of course, trying to convert them using GWP should (continue to) fail.

Any objections @khaeru @phackstock?

Add defs based on MIT Energy Club fact sheet

I have this 2007 (!) fact sheet produced by a former member of the MIT Energy Club: UnitsAndConversions.pdf

This suggests several possible conversions that could be incorporated in iam-units to make it more broadly useful:

  • Density: conversion from [volume] to [mass] and vice versa. Like GWP, this would require a context and substance identifier.
  • Energy content, e.g. Lower Heating Value (LHV): conversion from [mass] to [energy] and vice versa.
    • With density, this also allows conversion from [volume] to [energy].
    • Note that pint already includes some definitions like tonne_of_oil_equivalent = 1e10 * international_calorie = toe; these "_equivalent" would be duplicated somewhat by the new context.
  • Emissions factors, prices: these are technology- and location- specific, so probably not appropriate for this package.
  • The other conversions on the fact sheet should be supported by pint defaults and thus by iam_units.registry, but could still be added to the test suite. They could also be added in a user-friendly format like a Jupyter notebook/Binder.

pint can't handle 'million', 'billion', ...

In integrated-assesment models, we have variables like 'Population' which are given in million or billion (people being implicitly understood).

It would be practical of being able to do the following:

pint.UnitRegistry()('1000 million').to('billion')

Expected output: 1 billion

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.