Giter VIP home page Giter VIP logo

amply's Introduction

Amply

https://travis-ci.com/willu47/amply.svg?branch=master PyPI https://coveralls.io/repos/github/willu47/amply/badge.svg?branch=master

Introduction

Amply allows you to load and manipulate AMPL data as Python data structures.

Amply only supports a specific subset of the AMPL syntax:

  • set declarations
  • set data statements
  • parameter declarations
  • parameter data statements

Declarations and data statements

Typically, problems expressed in AMPL consist of two parts, a model section and a data section. Amply is only designed to parse the parameter and set statements contained within AMPL data sections. However, in order to parse these statements correctly, information that would usually be contained within the model section may be required. For instance, it may not be possible to infer the dimension of a set purely from its data statement. Therefore, Amply also supports set and parameter declarations. These do not have to be put in a separate section, they only need to occur before the corresponding data statement.

The declaration syntax supported is extremely limited, and does not include most elements of the AMPL programming language. The intention is that this library is used as a way of loading data specified in an AMPL-like syntax.

Furthermore, Amply does not perform any validation on data statements.

About this document

This document is intended as a guide to the syntax supported by Amply, and not as a general AMPL reference manual. For more in depth coverage see the GNU MathProg manual, Chapter 5: Model data or the following links:

Quickstart Guide

>>> from amply import Amply

Import the class:

>>> from amply import Amply

A simple set. Sets behave a lot like lists.

>>> data = Amply("set CITIES := Auckland Wellington Christchurch;")
>>> print data.CITIES
<SetObject: ['Auckland', 'Wellington', 'Christchurch']>
>>> print data['CITIES']
<SetObject: ['Auckland', 'Wellington', 'Christchurch']>
>>> for c in data.CITIES: print c
...
Auckland
Wellington
Christchurch
>>> print data.CITIES[0]
Auckland
>>> print len(data.CITIES)
3

Data can be integers, reals, symbolic, or quoted strings:

>>> data = Amply("""
...   set BitsNPieces := 0 3.2 -6e4 Hello "Hello, World!";
... """)
>>> print data.BitsNPieces
<SetObject: [0.0, 3.2000000000000002, -60000.0, 'Hello', 'Hello, World!']>

Sets can contain multidimensional data, but we have to declare them to be so first.

>>> data = Amply("""
... set pairs dimen 2;
... set pairs := (1, 2) (2, 3) (3, 4);
... """)
>>> print data.pairs
<SetObject: [(1, 2), (2, 3), (3, 4)]>

Sets themselves can be multidimensional (i.e. be subscriptable):

>>> data = Amply("""
... set CITIES{COUNTRIES};
... set CITIES[Australia] := Adelaide Melbourne Sydney;
... set CITIES[Italy] := Florence Milan Rome;
... """)
>>> print data.CITIES['Australia']
['Adelaide', 'Melbourne', 'Sydney']
>>> print data.CITIES['Italy']
['Florence', 'Milan', 'Rome']

Note that in the above example, the set COUNTRIES didn't actually have to exist itself. Amply does not perform any validation on subscripts, it only uses them to figure out how many subscripts a set has. To specify more than one, separate them by commas:

>>> data = Amply("""
... set SUBURBS{COUNTRIES, CITIES};
... set SUBURBS[Australia, Melbourne] := Docklands 'South Wharf' Kensington;
... """)
>>> print data.SUBURBS['Australia', 'Melbourne']
['Docklands', 'South Wharf', 'Kensington']

Slices can be used to simplify the entry of multi-dimensional data.

>>> data=Amply("""
... set TRIPLES dimen 3;
... set TRIPLES := (1, 1, *) 2 3 4 (*, 2, *) 6 7 8 9 (*, *, *) (1, 1, 1);
... """)
>>> print data.TRIPLES
<SetObject: [(1, 1, 2), (1, 1, 3), (1, 1, 4), (6, 2, 7), (8, 2, 9), (1, 1, 1)]>
>

Set data can also be specified using a matrix notation. A '+' indicates that the pair is included in the set whereas a '-' indicates a pair not in the set.

>>> data=Amply("""
... set ROUTES dimen 2;
... set ROUTES : A B C D :=
...            E + - - +
...            F + + - -
... ;
... """)
>>> print data.ROUTES
<SetObject: [('E', 'A'), ('E', 'D'), ('F', 'A'), ('F', 'B')]>

Matrices can also be transposed:

>>> data=Amply("""
... set ROUTES dimen 2;
... set ROUTES (tr) : E F :=
...                 A + +
...                 B - +
...                 C - -
...                 D + -
... ;
... """)
>>> print data.ROUTES
<SetObject: [('E', 'A'), ('F', 'A'), ('F', 'B'), ('E', 'D')]>

Matrices only specify 2d data, however they can be combined with slices to define higher-dimensional data:

>>> data = Amply("""
... set QUADS dimen 2;
... set QUADS :=
... (1, 1, *, *) : 2 3 4 :=
...              2 + - +
...              3 - + +
... (1, 2, *, *) : 2 3 4 :=
...              2 - + -
...              3 + - -
... ;
... """)
>>> print data.QUADS
<SetObject: [(1, 1, 2, 2), (1, 1, 2, 4), (1, 1, 3, 3), (1, 1, 3, 4), (1, 2, 2, 3), (1, 2, 3, 2)]>

Parameters are also supported:

>>> data = Amply("""
... param T := 30;
... param n := 5;
... """)
>>> print data.T
30
>>> print data.n
5

Parameters are commonly indexed over sets. No validation is done by Amply, and the sets do not have to exist. Parameter objects are represented as a mapping.

>>> data = Amply("""
... param COSTS{PRODUCTS};
... param COSTS :=
...   FISH 8.5
...   CARROTS 2.4
...   POTATOES 1.6
... ;
... """)
>>> print data.COSTS
<ParamObject: {'POTATOES': 1.6000000000000001, 'FISH': 8.5, 'CARROTS': 2.3999999999999999}>
>>> print data.COSTS['FISH']
8.5

Parameter data statements can include a default clause. If a '.' is included in the data, it is replaced with the default value:

>>> data = Amply("""
... param COSTS{P};
... param COSTS default 2 :=
... F 2
... E 1
... D .
... ;
... """)
>>> print data.COSTS['D']
2.0

Parameter declarations can also have a default clause. For these parameters, any attempt to access the parameter for a key that has not been defined will return the default value:

>>> data = Amply("""
... param COSTS{P} default 42;
... param COSTS :=
... F 2
... E 1
... ;
... """)
>>> print data.COSTS['DOES NOT EXIST']
42.0

Parameters can be indexed over multiple sets. The resulting values can be accessed by treating the parameter object as a nested dictionary, or by using a tuple as an index:

>>> data = Amply("""
... param COSTS{CITIES, PRODUCTS};
... param COSTS :=
...  Auckland FISH 5
...  Auckland CHIPS 3
...  Wellington FISH 4
...  Wellington CHIPS 1
... ;
... """)
>>> print data.COSTS
<ParamObject: {'Wellington': {'FISH': 4.0, 'CHIPS': 1.0}, 'Auckland': {'FISH': 5.0, 'CHIPS': 3.0}}>
>>> print data.COSTS['Wellington']['CHIPS'] # nested dict
1.0
>>> print data.COSTS['Wellington', 'CHIPS'] # tuple as key
1.0

Parameters support a slice syntax similar to that of sets:

>>> data = Amply("""
... param COSTS{CITIES, PRODUCTS};
... param COSTS :=
...  [Auckland, * ]
...   FISH 5
...   CHIPS 3
...  [Wellington, * ]
...   FISH 4
...   CHIPS 1
... ;
... """)
>>> print data.COSTS
<ParamObject: {'Wellington': {'FISH': 4.0, 'CHIPS': 1.0}, 'Auckland': {'FISH': 5.0, 'CHIPS': 3.0}}>

Parameters indexed over two sets can also be specified in tabular format:

>>> data = Amply("""
... param COSTS{CITIES, PRODUCTS};
... param COSTS: FISH CHIPS :=
...  Auckland    5    3
...  Wellington  4    1
... ;
... """)
>>> print data.COSTS
<ParamObject: {'Wellington': {'FISH': 4.0, 'CHIPS': 1.0}, 'Auckland': {'FISH': 5.0, 'CHIPS': 3.0}}>

Tabular data can also be transposed:

>>> data = Amply("""
... param COSTS{CITIES, PRODUCTS};
... param COSTS (tr): Auckland Wellington :=
...            FISH   5        4
...            CHIPS  3        1
... ;
... """)
>>> print data.COSTS
<ParamObject: {'Wellington': {'FISH': 4.0, 'CHIPS': 1.0}, 'Auckland': {'FISH': 5.0, 'CHIPS': 3.0}}>

Slices can be combined with tabular data for parameters indexed over more than 2 sets:

>>> data = Amply("""
... param COSTS{CITIES, PRODUCTS, SIZE};
... param COSTS :=
...  [Auckland, *, *] :   SMALL LARGE :=
...                 FISH  5     9
...                 CHIPS 3     5
...  [Wellington, *, *] : SMALL LARGE :=
...                 FISH  4     7
...                 CHIPS 1     2
... ;
... """)
>>> print data.COSTS
<ParamObject: {'Wellington': {'FISH': {'SMALL': 4.0, 'LARGE': 7.0}, 'CHIPS': {'SMALL': 1.0, 'LARGE': 2.0}}, 'Auckland': {'FISH': {'SMALL': 5.0, 'LARGE': 9.0}, '

API

All functionality is contained within the Amply class.

load_string(string)

Parse string data.

load_file(file)

Parse contents of file or file-like object (has a read() method).

from_file(file)

Alternate constructor. Create Amply object from contents of file or file-like object.

The parsed data structures can then be accessed from an Amply object via attribute lookup (if the name of the symbol is a valid Python name) or item lookup.

from pulp import Amply

data = Amply("set CITIES := Auckland Hamilton Wellington")

# attribute lookup
assert data.CITIES == ['Auckland', 'Hamilton', 'Wellington']

# item lookup
assert data['CITIES'] == data.CITIES

Note that additional data may be loaded into an Amply object simply by calling one of its methods. A common idiom might be to specify the set and parameter declarations within your Python script, then load the actual data from external files.

from pulp import Amply

data = Amply("""
  set CITIES;
  set ROUTES dimen 2;
  param COSTS{ROUTES};
  param DISTANCES{ROUTES};
""")

for data_file in ('cities.dat', 'routes.dat', 'costs.dat', 'distances.dat'):
    data.load_file(open(data_file))

Development Notes

Many thanks to Johannes Ragam (@thet), former custodian of the "amply" project on PyPi. Johannes graciously transferred the project to this. Thanks!

amply's People

Contributors

chmduquesne avatar pchtsp avatar willu47 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

amply's Issues

Amply doesn't allow alphanumerical element names

param default 1 : AvailabilityFactor :=
01_AUS POW_Other_Coal_PP 2017 0.0

results in a parsing error.

Amply doesn't allow numerical characters to start a symbol, and also does not allow underscores in names.

Support symbolic parameters

Amply does not support symbolic parameters such as param month symbolic default ’May’ in {’Mar’, ’Apr’, ’May’}; or param resultpath symbolic default 'results' := 'results/path';

make amply.AmplyError available in __init__ file

Hello Will,

Could you make the AmplyError available outside of the package on top of offering the "Amply" object? I'm trying to take it out from pulp while keeping the tests to be sure everything is still possible.

Thanks!

Franco

x in <SET> in parameter definition not handled

In GNU MathProg files, legal syntax is to include enumeration over a set in a parameter definition subscript domain using x in set_name. For example:

            set REGION := Kenya;
            set TECHNOLOGY := TRLV_1_0;
            set YEAR := 2016 2017 2018 2019 2020;
            param Peakdemand {r in REGION, t in TECHNOLOGY, y in YEAR};

However, this currently raises an error in amply. Omitting the in statement works:

            set REGION := Kenya;
            set TECHNOLOGY := TRLV_1_0;
            set YEAR := 2016 2017 2018 2019 2020;
            param Peakdemand {REGION,TECHNOLOGY,YEAR};

[Bug] Mixed inputformat error

Error in sys.excepthook:
Traceback (most recent call last):
  File "c:\nobackup\anaconda3\envs\snakemake\lib\site-packages\otoole\cli.py", line 282, in exception_handler
    print("{}: {}".format(exception_type.__name__, exception.message))
AttributeError: 'AmplyError' object has no attribute 'message'

Original exception was:
Traceback (most recent call last):
  File "c:\nobackup\anaconda3\envs\snakemake\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\nobackup\anaconda3\envs\snakemake\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Nobackup\Anaconda3\envs\snakemake\Scripts\otoole.exe\__main__.py", line 7, in <module>
  File "c:\nobackup\anaconda3\envs\snakemake\lib\site-packages\otoole\cli.py", line 287, in main
    args.func(args)
  File "c:\nobackup\anaconda3\envs\snakemake\lib\site-packages\otoole\cli.py", line 112, in result_matrix
    input_data, _ = ReadDatafile().read(args.input_datafile)
  File "c:\nobackup\anaconda3\envs\snakemake\lib\site-packages\otoole\read_strategies.py", line 212, in read
    amply_datafile = self.read_in_datafile(filepath, config)
  File "c:\nobackup\anaconda3\envs\snakemake\lib\site-packages\otoole\read_strategies.py", line 229, in read_in_datafile
    datafile_parser.load_file(datafile)
  File "c:\nobackup\anaconda3\envs\snakemake\lib\site-packages\amply\amply.py", line 817, in load_file
    self.load_string(f.read())
  File "c:\nobackup\anaconda3\envs\snakemake\lib\site-packages\amply\amply.py", line 806, in load_string
    obj.eval(self)
  File "c:\nobackup\anaconda3\envs\snakemake\lib\site-packages\amply\amply.py", line 305, in eval
    self.name, self.tokens
amply.amply.AmplyError: Error in number of records of Peakdemand when reading ['param', 'Peakdemand', 'default', 1.0, [<SliceRecord: ('Kenya', '*', '*')>, <TabularRecord: {'TRLV_1_0': {2016.0: 0.0, 2017.0: 0.0, 2018.0: 0.0, 2019.0: 0.035503748, 2020.0: 0.073847796, 2021.0: 0.115202562, 2022.0: 0.159747552, 2023.0: 0.207671818, 2024.0: 0.259174428, 2025.0: 0.314464973, 2026.0: 0.373764082, 2027.0: 0.437303976, 2028.0: 0.437303976, 2029.0: 0.437303976, 2030.0: 0.437303976, 2031.0: 0.437303976, 2032.0: 0.437303976, 2033.0: 0.437303976, 2034.0: 0.437303976, 2035.0: 0.437303976, 2036.0: 0.437303976, 2037.0: 0.437303976, 2038.0: 0.437303976, 2039.0: 0.437303976, 2040.0: 0.437303976}, 'TRLV_2_0': {2016.0: 0.0, 2017.0: 0.0, 2018.0: 0.0, 2019.0: 0.031715426, 2020.0: 0.065968085, 2021.0: 0.102910213, 2022.0: 0.142702162, 2023.0: 0.18551281, 2024.0: 0.231519987, 2025.0: 0.280910918, 2026.0: 0.333882691, 2027.0: 0.390642748, 2028.0: 0.390642748, 2029.0: 0.390642748, 2030.0: 0.390642748, 2031.0: 0.390642748, 2032.0: 0.390642748, 2033.0: 0.390642748, 2034.0: 0.390642748, 2035.0: 0.390642748, 2036.0: 0.390642748, 2037.0: 0.390642748, 2038.0: 0.390642748, 2039.0: 0.390642748, 2040.0: 0.390642748}, 'TRLV_3_0': {2016.0: 0.0, 2017.0: 0.0, 2018.0: 0.0, 2019.0: 0.03760931, 2020.0: 0.078227364, 2021.0: 0.122034688, 2022.0: 0.169221434, 2023.0: 0.219987865, 2024.0: 0.274544855, 2025.0: 0.333114424, 2026.0: 0.395930287, 2027.0: 0.463238436, 2028.0: 0.463238436, 2029.0: 0.463238436, 2030.0: 0.463238436, 2031.0: 0.463238436, 2032.0: 0.463238436, 2033.0: 0.463238436, 2034.0: 0.463238436, 2035.0: 0.463238436, 2036.0: 0.463238436, 2037.0: 0.463238436, 2038.0: 0.463238436, 2039.0: 0.463238436, 2040.0: 0.463238436}, 'TRLV_4_0': {2016.0: 0.0, 2017.0: 0.0, 2018.0: 0.0, 2019.0: 0.052990896, 2020.0: 0.110221063, 2021.0: 0.171944858, 2022.0: 0.238430203, 2023.0: 0.309959264, 2024.0: 0.386829161, 2025.0: 0.469352715, 2026.0: 0.557859227, 2027.0: 0.652695296, 2028.0: 0.652695296, 2029.0: 0.652695296, 2030.0: 0.652695296, 2031.0: 0.652695296, 2032.0: 0.652695296, 2033.0: 0.652695296, 2034.0: 0.652695296, 2035.0: 0.652695296, 2036.0: 0.652695296, 2037.0: 0.652695296, 2038.0: 0.652695296, 2039.0: 0.652695296, 2040.0: 0.652695296}, 'TRLV_5_0': {2016.0: 0.0, 2017.0: 0.0, 2018.0: 0.0, 2019.0: 0.034974676, 2020.0: 0.072747325, 2021.0: 0.113485828, 2022.0: 0.157367014, 2023.0: 0.204577119, 2024.0: 0.255312244, 2025.0: 0.309778856, 2026.0: 0.368194298, 2027.0: 0.430787328, 2028.0: 0.430787328, 2029.0: 0.430787328, 2030.0: 0.430787328, 2031.0: 0.430787328, 2032.0: 0.430787328, 2033.0: 0.430787328, 2034.0: 0.430787328, 2035.0: 0.430787328, 2036.0: 0.430787328, 2037.0: 0.430787328, 2038.0: 0.430787328, 2039.0: 0.430787328, 2040.0: 0.430787328}, 'TRLV_6_0': {2016.0: 0.0, 2017.0: 0.0, 2018.0: 0.0, 2019.0: 0.037202401, 2020.0: 0.077380994, 2021.0: 0.12071435, 2022.0: 0.167390565, 2023.0: 0.217607735, 2024.0: 0.271574453, 2025.0: 0.329510336, 2026.0: 0.391646571, 2027.0: 0.458226488, 2028.0: 0.458226488, 2029.0: 0.458226488, 2030.0: 0.458226488, 2031.0: 0.458226488, 2032.0: 0.458226488, 2033.0: 0.458226488, 2034.0: 0.458226488, 2035.0: 0.458226488, 2036.0: 0.458226488, 2037.0: 0.458226488, 2038.0: 0.458226488, 2039.0: 0.458226488, 2040.0: 0.458226488}, 'TRLV_7_0': {2016.0: 0.0, 2017.0: 0.0, 2018.0: 0.0, 2019.0: 0.023055127, 2020.0: 0.047954664, 2021.0: 0.074809276, 2022.0: 0.103735529, 2023.0: 0.134856188, 2024.0: 0.168300522, 2025.0: 0.204204634, 2026.0: 0.242711793, 2027.0: 0.283972798, 2028.0: 0.283972798, 2029.0: 0.283972798, 2030.0: 0.283972798, 2031.0: 0.283972798, 2032.0: 0.283972798, 2033.0: 0.283972798, 2034.0: 0.283972798, 2035.0: 0.283972798, 2036.0: 0.283972798, 2037.0: 0.283972798, 2038.0: 0.283972798, 2039.0: 0.283972798, 2040.0: 0.283972798}, 'TRLV_8_0': {2016.0: 0.0, 2017.0: 0.0, 2018.0: 0.0, 2019.0: 0.029656132, 2020.0: 0.061684755, 2021.0: 0.096228218, 2022.0: 0.133436462, 2023.0: 0.1734674, 2024.0: 0.216487316, 2025.0: 0.262671276, 2026.0: 0.312203574, 2027.0: 0.365278182, 2028.0: 0.365278182, 2029.0: 0.365278182, 2030.0: 0.365278182, 2031.0: 0.365278182, 2032.0: 0.365278182, 2033.0: 0.365278182, 2034.0: 0.365278182, 2035.0: 0.365278182, 2036.0: 0.365278182, 2037.0: 0.365278182, 2038.0: 0.365278182, 2039.0: 0.365278182, 2040.0: 0.365278182}, 'TRLV_9_0': {2016.0: 0.0, 2017.0: 0.0, 2018.0: 0.0, 2019.0: 0.040579304, 2020.0: 0.084404952, 2021.0: 0.131671725, 2022.0: 0.182584792, 2023.0: 0.23736023, 2024.0: 0.296225567, 2025.0: 0.359420354, 2026.0: 0.427196764, 2027.0: 0.499820214, 2028.0: 0.499820214, 2029.0: 0.499820214, 2030.0: 0.499820214, 2031.0: 0.499820214, 2032.0: 0.499820214, 2033.0: 0.499820214, 2034.0: 0.499820214, 2035.0: 0.499820214, 2036.0: 0.499820214, 2037.0: 0.499820214, 2038.0: 0.499820214, 2039.0: 0.499820214, 2040.0: 0.499820214}}>]]

I have mixed input formats in my datafile for parameters. Some inputs are in matrix form and some are in "long" format.
I get the following Error message:
amply.amply.AmplyError: Error in number of records of Peakdemand
when reading the first parameter which is in matrix format.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.