Giter VIP home page Giter VIP logo

mo-book's Introduction

Companion Jupyter notebooks for Hands-On Optimization with Python book

This is the source repository for the collection of Jupyter notebooks associated with the upcoming book Hands-On Optimization with Python. The book is currently in preparation and will be published by Cambridge University Press in 2024.

If you wish to cite this work, please use

@book{PostekZocca2024,
title     = "Hands-On Optimization with Python",
author    = "Postek, Krzysztof and Zocca, Alessandro and Gromicho, Joaquim and Kantor, Jeffrey"
year      = 2024,
publisher = "Online",
url       = "https://mobook.github.io/MO-book/"
}

mo-book's People

Contributors

alessandrozocca avatar gromicho avatar jckantor avatar krzysztofpostek avatar leonlan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mo-book's Issues

Lingering issues in Shift-Scheduling.ipynb

Lingering issues in Shift-Scheduling.ipynb:

  • Typo "perferences"
  • Typos "Inicator if worker has been assigned a weekend shift."
  • Incomplete sentence "This is formulated here as an objec"
  • gamma variable not displayed correctly
  • is there a way to report the solution as a pandas dataframe? (low priority)
  • There are some rendering issues in some of the formulas in the model constraints section

Landing pages for book and for chapters

The current landing pages for the book and for each chapter are plain and bare bones. It may be useful to a guide, index, or help to the reader about what is contained in the chapter notebooks in terms of (1) math model features, (2) pyomo model features, (3) applications.

  • Main Page
  • Chapter 1
  • Chapter 2
  • Chapter 3
  • Chapter 4
  • Chapter 5
  • Chapter 6
  • Chapter 7
  • Chapter 8
  • Chapter 9
  • Chapter 10
  • Chapter 11

Add more examples in the style guide

Some sections in the style guide have a lot of examples:

#### Use decorators to improve readability
Indexed Pyomo constraints are constructed by a rule. When using `pyo.Constraint()` rules are normally named by adding `_rule` as a suffix to the name of the associated constraint. For example, assuming model `m` and the associated sets, parameters, and variables have been previously defined,
```python
def new_constraint_rule(m, s):
return m.x[s] <= m.ub[s]
m.new_constraint = pyo.Constraint(m.S, rule=new_constraint_rule)
```
A recent innovation in Pyomo is the use of Python decorators to create Constraint, Objective, and Disjunction objects. Using decorators, the above example is written as
```python
@m.Constraint(m.S)
def new_constraint_rule(m, s):
return m.x[s] <= m.ub[s]
```
The use of decorators improves readability by eliminating the need for the `rule` keyword and the need for writing multiple versions of the same constraint name.
The decorator syntax is straightforward for objectives and simple constraints. For Python users unfamiliar with decorators, decorators can be described as a way to 'tag' functions that are to be incorporated into the Pyomo model. Indices and keywords are used modify the extend to the bahavior of the decorator.
````python
@model.Constraint()
def demand_constraint(model):
return model.x + model.y <= 40
@model.Objective(sense=pyo.maximize)
def profit(model):
return 3*model.x + 4*model.y
````
Indices are also included in the decorator for indexed objects.
```python
@model.Constraint(model.SOURCES)
def capacity_constraint(model, src):
return sum(model.ship[src, dst] for dst in model.DESTINATIONS) <= model.capacity[src]
@model.Constraint(model.DESTINATIONS)
def demand_constraint(model, dst):
return sum(model.ship[src, dst] for dst in model.SOURCES) <= model.demand[dst]
```

It would be nice to add more examples in some of the sections that currently have none, e.g.:

### Parameters
Consistent with good programming practice, non-constant global parameters should be avoided in Pyomo models. Non-constant global parameters lead to inconsistent state in the code, and complicate understanding of models when parameters can be defined, and redefined, in remote places in the code.
Pyomo modelers may prefer to use native Python data structures rather declare and use instances of parameters created with the `pyo.Param()` class. In these cases, best practice is to limit the scope of parameters by constructing the model within a Python function, using the arguments of the function to provide a clear interface to the global scope.
Pyomo parameter created with `pyo.Param()` localize parameter values to a specific model or block. By default, Pyomo parameters are immutable which assures their values will be consistent throughout the model construction and transformations. Parameters determining the size of index sets, or fixed upper and lower bounds on decision variables, are examples where using an immutable Pyomo is good practice.
Pyomo parameters created with `mutable=True` are used to build models that can be re-solved for parametric or sensitivity analysis.
### Variables
#### Use `domain` rather than `within`
The `pyo.Var()` class accepts either `within` or `domain` as a keyword to specify decision variables. Offering options with no functional difference places an unnecessary cognitive burden on new users. Consistent use of `domain` is preferred because of its common use in mathematics to represent the set of all values for which a variable is defined.
#### Use `bounds` when known and fixed
A Pyomo model can place bounds on decision variables with either the `bounds` keyword in the argument to `pyo.Var`, or as explicit constraints in the model.
When upper or lower bounds for a variable are known and fixed, use of `bounds` when creating the variable is a best practice in mathematical optimization. This practice can reduce the number of explicit constraints in the model and simplify coding and model display.
If, however, variable bounds may be subject to change during the course of problem solving, then explicit constraints should be used.
### Constraints and Objective

Wine quality notebook has errors

The wine quality notebook has a broken cell that needs repair. I've assigned this to myself and will close the issue when fixed.

Minor issues in Pop-Up-Shop.ipynb

  • Text says demand is 400 for good weather, code says 350.
  • Text says unit cost is 5, code says 12.
  • The probabilities of the three scenarios are also inconsistent between text and code.
  • TypeError: unsupported format string passed to NoneType.__format__
  • ERROR: evaluating object as numeric value: f[sunny skies] (object: <class 'pyomo.core.base.var._GeneralVarData'>) No value for uninitialized NumericValue object f[sunny skies]
  • Do we want the glpsol script?

Notebooks Chapter 2

Regression example

  • code is not here on GitHub, but on colab
  • is not consistent with Pyomo style guide
  • misses description and intuition

MAD Portfolio example

  • the name of the file is also inconsistent with the rest?
  • still incomplete/work in progress

Minmax microchip example

  • code is not here on GitHub, but on colab
  • is not consistent with Pyomo style guide
  • misses description and intuition
  • the concrete model is defined inside a function, which is a standard we want to move away from, right?

Fractional microchip example

  • code is not here on GitHub, but on colab
  • is not consistent with Pyomo style guide
  • misses description and intuition
  • the concrete model is defined inside a function, which is a standard we want to move away from, right?

Dual microchip example

  • code is not here on GitHub, but on colab
  • preamble is largely unnecessary?
  • is not consistent with Pyomo style guide
  • misses description and intuition
  • the various concrete models are all defined inside functions, which is a standard we want to move away from, right?
  • it still contains tons of the old trophy formulation (search for words like football, golf, wood, plastic, etc.)

Final complete example -- multistage microchip production

  • this notebook is far from the standard that we want in terms of style guide (it's colab-oriented, lots of import commands, etc.)
  • the name of the file is also inconsistent with the rest?
  • misses description and intuition

Add Github Action to deploy MO-book

Currently the book is built with JupyterBook and pushed to the main branch. The ghp-import moves the HTML folder to gh-pages. This works, but could potentially result in a difficult merge conflict if multiple folks are building the book.

A github action would provide a single and reproducible path for deploying the book following a push to the main branch. See https://jupyterbook.org/publish/gh-pages.html.

  • Create and test github action to deploy MO-book

Missing notebook "Dual of BIM production problem"

This is a notebook created by Joaquim that hasn't been ported yet. The comments refer to that version.

  • preamble is largely unnecessary?
  • is not consistent with Pyomo style guide
  • misses description and intuition
  • the various concrete models are all defined inside functions, which is a standard we want to move away from, right?
  • it still contains tons of the old trophy formulation (search for words like football, golf, wood, plastic, etc.)

Lingering issues in large-example-chapter-2.ipynb

Lingering issues in large-example-chapter-2.ipynb:

  • new name for the file? Maybe "Multistage microchip production"?
  • check preamble
  • code (and Pyomo model) are scattered in too many cells, probably should be consolidated in fewer?same for the markdown cells?
  • code not aligned with Pyomo style guide, please refactor/rewrite it
  • check data import (can it be streamlined even more?)
  • missing description and intuition throughout the file? Plenty of accompanying text (it's in the book, section 2.5)
  • missing full mathematical formulation of the optimization problem (it's in the book, section 2.5)
  • Python/Pyomo variables should reflect those in the mathematical (e.g., x,y instead of buy, stock)
  • Remove/update section title "NOTE: The functions below follow closely the naming in Overleaf"

economic-order-quantity.ipynb

  • Postpone bibliographic notes rather than having them at the top?
  • Typo: "incorrectly cited ince 1931"
  • Add the URL for Ziegler paper, that is https://www.sciencedirect.com/science/article/abs/pii/016763778290030X
  • Unclear sentence: "The problem is to find the size of a that minimizes the cost of maintaining that item in an inventory". In the book I rephrased it as "The problem consists of finding the order quantity that minimizes the cost of maintaining a specific item in inventory"
  • Typo: "given given"
  • Typo: "where $x$ is the...?"
  • Typo: "is define"
  • Unclear sentence "As the following diagrams the solution to this optimization problem."
  • Remove label/description on top of the 2nd figure (but maybe only when @gromicho gets here with figure regeneration)
  • Typo: "...the hyperbola described by the constraint $x y \leq 1$" should be $x y \geq 1$?
  • Too much white space around the Lorentz cone picture, needs to be trimmed (but maybe only when @gromicho gets here with figure regeneration)
  • "Note the improvement in accuracy of this calculation compared to the analytical solution." --> compared to the previous implementations?

Add CITATION.cff

Need to add a citation file. I'll need any ORCID id's, institutions, etc. to complete the file.

  • Create tag/vers number
  • get DOI
  • [ ]

Job Shop NEOS

The Job Shop notebook needs to be validated with NEOS. Currently this isn't working.

dina-tables-and-variations.ipynb

  • Move all the import commands in an initial cell and make it coherent with the preambles of the other notebooks
  • Related to the previous point, the current output display an import error for the module pyperclip
  • Need to add more story and accompanying text and sections split and names need to be homogenized
  • Maybe a thorough check for consistency with the Pyomo style guide could be helpful @jckantor
  • Several cells in the file that currently appears in github and in the jupyter book have not been run, i.e., all the outputs are not visible
  • There are a lot of RuntimeError in the second part of the notebook

Electric vehicle recharging notebook

Some things I noticed

  • rechange => recharge
  • The charging stations are located at positions d_i with capacities C_i => ... with charging rate per time unit C_i
  • if a decision is make => if a decision is made
  • it would be good to state explicitly that in the taken solution approach it is assumed that stopping = recharge = rest (in general case one could stop to rest without recharging as well, or only recharge but not rest)

Add support for HiGHS solver

HiGHS is a new open-source, high-performance solver for MILPs. It would be nice to add support for the HiGHS solver installation.

HiGHS has been recently added to PyPi, and there's now also an interface with Pyomo. I haven't tried it yet, but I think it could be as simple as pip install highspy and then selecting 'highs' in the solver interface.

Pooling and Blending notebook

Hi,
in the textbook I have changed the solver in this code snippet to IPOPT. I also added in the textbook a code file named
ch5_milkpooling_nonlinear_badstart,py to overleaf with the following ending to the script:

solver = pyo.SolverFactory('ipopt')
solver.options["bound_frac"] = 0.4
solver.solve(m)

This one makes IPOPT converge to another point p = 0.45 and the objective function value equal to 101392.15, which is about 1.4% worse than the best solution. Would be great if it could be also updated over here! Many thanks.

Krzysztof

Lingering issues in fractional-bim.ipynb

Lingering issues in fractional-bim.ipynb:

  • there are actually two models in the notebook, only one of which is fractional BIM. There should be two sections separating them and a new overarching title for the notebook, namely "Different profit formulations for BIM microchips production"
  • related to the new title, the name of the file could change accordingly?
  • preamble cells are outdated, delete and replace with the new one
  • no story nor accompanying text for both models (fraction and subtraction, please copy it from the book (see Example 6)
  • delete the functions and rearrange the code as to define the two Pyomo model directly
  • the name of the concrete models should be different (maybe just use the current names of the functions?)

shortest-path-road-networks.ipynb

  • Move all the import commands in an initial cell and make it coherent with the preambles of the other notebooks
  • The whole notebook needs to be reorganized according to our style
  • There are several cells with not-so-useful output that probably belong to the development phase of the notebook?
  • Need to add more story and accompanying text, it's not clear what mathematical problem we're trying to solve and under which constraints.

svm-linear.ipynb

  • Typos: an device, reccall, [...] considerations result i the [...]
  • Weird/incomplete sentence "false positive a counterfeit banknote, clearly an undesirable outcome for the seller. "
  • Incomplete sentence "The classifier is then....?"

Lingering issues in fit-absolute.ipynb

Lingering issues in fit-absolute.ipynb:

  • add to the current explanatory/introductory text the various paragraps and equations that I wrote in the book (see Example 2)?
  • change title to "Least Absolute Deviation Regression" to make it consistent with the book?
  • change m = pyo.ConcreteModel('min abs') into m = pyo.ConcreteModel('LAD regression')
  • consolidate the whole Pyomo model in a single cell?
  • add one or two text cells to comment data generation and data visualization?
  • The domains of the variables X and y are displayed as \mathcal{R}, but they should be \mathbb{R}
  • The sentence "we can use the trick illustrated above to transform it into a linear problem" is appropriate for the book, but not for the companion notebook.

large-example-chapter-3.ipynb

  • Move all the import commands in an initial cell and make it coherent with the preambles of the other notebooks
  • Split the notebook in meaningful sections?
  • Maybe a thorough check for consistency with the Pyomo style guide could be helpful @jckantor
  • All the cells in the file that currently appears in github and in the jupyter book have not been run, i.e., all the outputs are not visible
  • Need to add more story and accompanying text, it's not clear at all what are the differences between the various model versions
  • Related to the previous point, can we give more informative names to the functions that create the pyomo models instead of VersionOne, VersionTwo and VersionThree?

Lingering issues in Pop-Up-Shop.ipynb

Lingering issues in Pop-Up-Shop.ipynb

  • "A naive solution to this proc is to place" -> "A naive solution to this problem is to place"
  • Pet Shop should be Pop-up Shop
  • All the variables are defined as NonNegativeReals, but the description clearly talks about items? What's the best way to resolve this?
  • Do we need the very big image at the very beginning of the notebook?

Copy-pasting here some of the old issues:

  • Do we want to address the reader as "you" and phrase the problem as a dialogue? As in, you need to solve this, you need to help X dealing with this issue, etc.?
  • I changed a bit the wording in the book, shall I change it accordingly in the notebook? Or is it fine if they're not identical?
  • Euro symbol before or after the number? With the space or without?
  • Do we need the capitals for Sunny Skies, Good Weather and Poor Weather? The corresponding dictionary keys have all small caps. Shall we go for the more boring "good, average, poor weather" classification? Or shall we stick to the more fun and creative sunny skies, but then add cloudy day and heavy downpour?
  • Should the mathematical style of the equations (and of the full optimization problem) be consistent between book and online companion?

Lingering issues with making-the-best-of-the-worst.ipynb

making-the-best-of-the-worst.ipynb:

  • instead of being a standalone file, this code should just become a 3rd variant inside the fractional-bim.ipynb, see also #10
  • preamble cells are outdated, delete and replace with the new one
  • no story nor accompanying text for both models (fraction and substraction, please copy it from the book (see Example 6)
  • delete the function and rearrange the code as to define the Pyomo model directly
  • the name of the concrete model should be different (maybe just use the current names of the function?)

Strip Packing: Placing Boxes on a Shelf

  • The subsequent models constructions and solutions are still written as functions - we need to switch them to line code.
  • I propose to state in advance in the text that (i) the dox dimensions shall be sampled, (ii) the depth of a shelf is going to be twice the depth of the deepest box, right now it appears suddenly in the code
  • "problem statment" => problem statement

Change `pyomo.xxx` to `pyo.xxx` in style guide

Hi Jeffrey,

Great style guide on pyomo, I love it!

One additional and small suggestion is to change references of pyomo.Var, pyomo.Constraint, etc. to pyo.Var and pyo.Constraint to stick to the namespace convention.

Best,
Leon

main book drawings.ipynb

  • the cell with the command draw.SetOutputPath( '/work in progress/MO book/results' ) currently displays an error (but maybe the full notebook just need to be re-run?)
  • create different notebooks for different chapters

Chapter 2 notebooks

The file /notebooks/02/EF_RO.ipynb should be move to a later chapter (being about robust optimization)

In the file /notebooks/02/02.02-MAD-Portfolio-Optimization.ipynb

  • there is a wrong reference to pulp. fixed
  • there is a weird cite {{...}} environment which doesn't display correctly
  • there is cross-reference [[w: ... ]] which doesn't work

Complete example ch4 (power network optimization)

  • too much text, shorter and more polished version is the one currently in the book
  • code needs to be polished and adherent to the style guide
  • there are also some external dependency to import the graph details which we should resolve by having the same files in the repo. I’ll search them and upload them asap.
  • Add variant with line switching actions (with edge addition/removal)

Needs a stable URL

The current URL includes the name of the style guide file. Need to establish a stable URL before distributing to others.

  • New Repository for Chapters 1-3
  • Set landing page to match book
  • Find stable URL

forex-arbitrage

  • unclear why we're referencing https://github.com/hzjken/crypto-arbitrage-framework
  • typos: opporunities, caculation
  • set $N$ is never introduced
  • role of phases $t$ is a bit unclear as well as that of the time (better name could be horizon?) $T$?
  • full mathematical formulation could be added?
  • more comments should be added about the solution?
  • more explicit connection to the theory presented in chapter 4: mention that is a weighted shortest path problem on a directed graph?
  • we also need to explain how is the length of the "cycle" encoded in the context of the shortest path problem
  • Graphviz isn't reliable on MacOS. Switch to networkx for visualizing the networks.
  • Add bibilographic notes

Lingering issues in mad-portfolio-optimization.ipynb

Lingering issues in mad-portfolio-optimization.ipynb:

  • missing extensive mathematical formulation of the optimization problem (at least early on in the notebook)
  • the statistical analysis of the historical data should be signalled better with a dedicated section
  • lots of incomplete/missing textual cells, especially to comment statistics
  • some of the code cells still contains bugs/errors and/or don't run properly
  • a lot of code/cells might be obsolete (like the pulp model) or not relevant or simply out of place? Please check
  • possibly related to the previous point, the notebook seem very long and too complex?
  • add narrative on risk vs return
  • add explanation for $T$ and $J$. In the book I added the following sentence: "Assume that we can make investment decisions at every trading day $t$ over a fixed time horizon ranging from $t=1,\dots,T$ and that there is a set of $J$ assets in which we can choose to invest."
  • Due to the absolute values, as it is written now in the notebook, $MAD$ is not the sum over $j$ of the $MAD_j$. This does not matter for the book, but maybe it'd be good to make the exposition crystal clear in the notebook.

Provide style advice about index sets

In the style guide, there's currently no advice on how to create different index sets. For example, given a set of nodes $V$, a common occurrence is that we want to construct a specific set of constraints for a subset $U \subseteq V$. I've seen a lot of people do this:

model.U = pyo.Set([v for v in model.V if v not in U]) # Create a new set

@model.Constraint(model.U)
def constraint(model, v):
  return expression(v) == 1

My personal preference is to keep the number of sets as small as possible. Instead, add conditionals in the constraint rule together with pyo.Constraint.Skip:

@model.Constraint(model.V)
def constraint(model, v):
  if v in U:
     return expression(v) == 1
  else:
     return pyo.Constraint.Skip

What are your thoughts on this?

Lingering issues in alice-rose.ipynb

Lingering issues in Alice's rose notebook:

  • preamble (pyomo loading) and packages import is not consistent (but maybe it's fine like this for didactical purposes?)
  • to the small extent where Pyomo is used, it is not consistent with the Pyomo's style guide
  • notebook is too long/wordy? can we consolidate the same material in fewer cells? can we remove some material, e.g. the prettify function so that it doesn't get in the way of Alice's story?
  • do we need to import NEOS?
  • shall we remove the %time commands?
  • all the external links/references to colab files that are not part of the companion notebook should be removed
  • the name of the file is also inconsistent with the rest?
  • Remove sentences "We now meet pyomo which will keep us company for the rest of the course. Note that in particular you may find this slide-deck quite useful."?
  • Replace links on installation with links to notebooks in this repository on installation once they are created and available.

Lingering issues in Transportation.ipynb

Lingering issues in Transportation.ipynb:

  • dollar signs are not rendered correctly in the text (cf. $)
  • we can stick to dollars and gallons, but maybe state somewhere that the franchise is based in the US?
  • is there another (more common?) word instead of incumbant?maybe my confusion arises because this is also never defined
  • mathematical model is never given explicitly and some textual cells could be added here and there to guide the reader
  • we are currently displaying the solution in 3 different ways, which may be unnecessary (unless there is something didactical about it, but then more text is needed)
  • story is too long? can we streamline it even more?

Production Model with Disjuncts

  • title: "Production Model with Disjuncts" = > Production Model with Disjunctions
  • we need to write something like "the starting production model is..." in the beginning before the first code snippet
  • pls add a word of comment that the latest code snippet is equivalent to the penultimate one except for the way the disjuctions are defined
  • in general a narrative is missing a bit in this example

Colab code inside every notebook?

Every page of the book can be run in google colab with a simple click, so the code below (taken from the Pop Up Shop example) is extremely useful to make the notebook "runnable". However, is there a way to hide this or avoid it "interrupts" the story flow?

import sys
at_colab = "google.colab" in sys.modules

if at_colab:
    _=!pip install -q pyomo
    _=!wget -N -q "https://ampl.com/dl/open/ipopt/ipopt-linux64.zip"
    _=!unzip -o -q ipopt-linux64
    _=!apt-get install -y -q coinor-cbc
    _=!pip install -q cplex
    _=!pip install -q gurobipy
    _=!pip install -q xpress

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.