Giter VIP home page Giter VIP logo

vrplib's Introduction

VRPLIB

PyPI version vrplib codecov

vrplib is a Python package for working with Vehicle Routing Problem (VRP) instances. The main features are:

  • reading VRPLIB and Solomon instances and solutions, and
  • writing VRPLIB-style instances and solutions.

Outline

Installation

vrplib works with Python 3.8+ and only depends on numpy. It may be installed in the usual way as

pip install vrplib

Example usage

Reading files

import vrplib

# Read VRPLIB formatted instances (default)
instance = vrplib.read_instance("/path/to/X-n101-k25.vrp")
solution = vrplib.read_solution("/path/to/X-n101-k25.sol")

# Read Solomon formatted instances
instance = vrplib.read_instance("/path/to/C101.txt", instance_format="solomon")
solution = vrplib.read_solution("/path/to/C101.sol") # only 1 solution format

instance and solution are dictionaries that contain all parsed data.

>>> instance.keys()
dict_keys(['name', ..., 'edge_weight'])

>>> solution.keys()
dict_keys(['routes', 'cost'])

Writing files

The functions write_instance and write_solution provide a simple interface to writing instances and solutions in VRPLIB-style:

  • write_instance adds indices to data sections when necessary (EDGE_WEIGHT_SECTION and DEPOT_SECTION are excluded).
  • write_solution adds the Route #{idx} prefix to routes.

Note that these functions do not validate instances: it is up to the user to write correct VRPLIB-style files.

Instances

import vrplib

instance_loc = "instance.vrp"
instance_data = {
    "NAME": "instance",
    "TYPE": "CVRP",
    "VEHICLES": 2,
    "DIMENSION": 1,
    "CAPACITY": 1,
    "EDGE_WEIGHT_TYPE": "EUC_2D",
    "NODE_COORD_SECTION": [[250, 250], [500, 500]],
    "DEMAND_SECTION": [1, 1],
    "DEPOT_SECTION": [1],
}

vrplib.write_instance(instance_loc, instance_data)
NAME: instance
TYPE: CVRP
VEHICLES: 2
DIMENSION: 1
CAPACITY: 1
EDGE_WEIGHT_TYPE: EUC_2D
NODE_COORD_SECTION
1	250	250
2	500	500
DEMAND_SECTION
1	1
2	1
DEPOT_SECTION
1
EOF

Solutions

import vrplib

solution_loc = "solution.sol"
routes = [[1], [2, 3], [4, 5, 6]]
solution_data = {"Cost": 42, "Vehicle types": [1, 2, 3]}

vrplib.write_solution(solution_loc, routes, solution_data)
Route #1: 1
Route #2: 2 3
Route #3: 4 5 6
Cost: 42
Vehicle types: [1, 2, 3]

Downloading from CVRPLIB

Warning

This functionality is deprecated and will be removed in the next major version.

import vrplib

# Download an instance and a solution file 
vrplib.download_instance("X-n101-k25", "/path/to/instances/")
vrplib.download_solution("X-n101-k25", "/path/to/solutions/")

# List all instance names that can be downloaded 
vrplib.list_names()                      # All instance names
vrplib.list_names(low=100, high=200)     # Instances with between [100, 200] customers
vrplib.list_names(vrp_type="cvrp")       # Only CVRP instances
vrplib.list_names(vrp_type="vrptw")      # Only VRPTW instances

Documentation

VRPLIB instance format

The VRPLIB format is the standard format for the Capacitated Vehicle Routing Problem (CVRP). An example VRPLIB instance looks as follows:

NAME: Example 
EDGE_WEIGHT_TYPE: EUC_2D
NODE_COORD_SECTION
1 0 0
2 5 5
SERVICE_TIME_SECTION
1 0
2 3
DEPOT_SECTION
1
EOF

A VRPLIB instance contains problem specifications and problem data.

  • Specifications are key-value pairs separated by a colon. In the example above, NAME and EDGE_WEIGHT_TYPE are the two data specifications.
  • Data are explicit array-like values such as customer coordinates or service times. Each data section should start with a header name that ends with _SECTION, e.g., NODE_COORD_SECTION and SERVICE_TIME_SECTION. It is then followed by rows of values and each row must start with an index representing the depot or customer. There are two exceptions: values in EDGE_WEIGHT_SECTION and DEPOT_SECTION should not start with an index.

Besides the rules outlined above, vrplib is not strict about the naming of specifications or sections. This means that you can use vrplib to read VRPLIB instances with custom specifications like MY_SPECIFICATION: SOME_VALUE and custom section names like MY_SECTION.

Reading the above example instance returns the following dictionary:

{'name': 'Example',
 'edge_weight_type': 'EUC_2D',
 'node_coord': array([[0, 0], [5, 5]]),
 'service_time': array([1, 3]),
 'depot': array([0]),
 'edge_weight': array([[0.  , 7.07106781], [7.07106781, 0.  ]])}

The depot section specifies which location index corresponds to the depot data. The convention is to let index 1 represent the depot. vrplib subtracts one from the depot value to make it easier to index.

Computing edge weights

Note that the example instance did not include any explicit information about the edge weights, yet the output includes edge weights data. This is because vrplib automatically computes the edge weights based on the instance specifications, if applicable. In the example, the edge weight type specification and node coordinates data are used to compute the Euclidean distance. You can set the compute_distances argument in read_instance to disable this feature.

Following the VRPLIB conventions, the edge weights are computed based on the EDGE_WEIGHT_TYPE specification, and in some cases the EDGE_WEIGHT_FORMAT specification. vrplib currently supports two categories of edge weight types:

  • *_2D: Euclidean distances based on the node coordinates data.
    • EUC_2D: Double precision distances without rounding.
    • FLOOR_2D: Round down all distances to down to an integer.
    • EXACT_2D: Multiply the distances by 1000, round to the nearest integer.
  • EXPLICIT: the distance data is explicitly provided, in partial or full form. The EDGE_WEIGHT_FORMAT specification must be present. We support the following two edge weight formats:
    • LOWER_ROW: Lower row triangular matrix without diagonal entries.
    • FULL_MATRIX: Explicit full matrix representation.

Line comments

Lines starting with # are interpreted as comments and not parsed when reading the instance.

More information about VRPLIB

The VRPLIB format is an extension of the TSPLIB95 format. Additional information about the VRPLIB format can be found here.

Solomon instance format

The Solomon format was used to introduce the Solomon instances for the Vehicle Routing Problem with Time Window (VRPTW) and also the extended instance set by Homberger and Gehring. A Solomon instance looks like this:

Example

VEHICLE
NUMBER     CAPACITY
  50          200

CUSTOMER
CUST NO.  XCOORD.    YCOORD.    DEMAND   READY TIME  DUE DATE   SERVICE TIME
    0      70         70          0          0       1351          0
    1      33         78         20        750        809         90

Reading this Solomon instance returns the following dictionary:

{'name': 'Example',
 'vehicles': 50,
 'capacity': 200,
 'node_coord': array([[70, 70], [33, 78]]),
 'demand': array([ 0, 20]),
 'time_window': array([[ 0, 1351], [ 750,  809]]),
 'service_time': array([ 0, 90]),
 'edge_weight': array([[ 0.  , 37.85498646], [37.85498646,  0.  ]])}

The edge weights are computed by default using the Euclidean distances.

Solution format

Here's an example of a solution format:

Route #1: 1 2 3
Route #2: 4 5
Cost: 100

A solution is represented by a set of routes, where each route specifies the sequence of customers to visit. Each route should start with "Route", followed by the route number, and followed by a colon. The customers to be served on the route are then listed. The solution file can also include other keywords like Cost, which will be separated on the first colon or whitespace.

The convention is that customer indices start counting from 1, but vrplib simply parses the file without strict requirements about those indices.

Reading the above example solution returns the following dictionary:

{'routes': [[1, 2, 3], [4, 5]], 'cost': 100}

Other remarks

  • The XML100 benchmark set is not listed in list_names and cannot be downloaded through this package. You can download these instances directly from CVRPLIB.
  • In the literature, some instances use rounding conventions different from what is specified in the instance. For example, X instance set proposed by Uchoa et al. (2017) assumes that the distances are rounded to the nearest integer. When you use the vrplib package to read this instance, it will return non-rounded Euclidean distances because the instance specifies the EUC_2D edge weight type which implies no rounding. To adhere to the convention used in the literature, you can manually round the distances matrix.
  • For large instances (>5000 customers) it's recommended to set the compute_edge_weights argument to False in read_instance.

vrplib's People

Contributors

leonlan avatar n-wouda avatar pre-commit-ci[bot] avatar wouterkool avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

vrplib's Issues

Speed-up parsing

It currently takes about 2 seconds to read a 1000-customer instance without explicit matrix:

In [73]: %timeit vrplib.read_instance(f"tmp/X-n1001-k43.txt")
2.14 s ± 44 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

This is slow. I think it's mainly due to computing the Euclidean distances. For example, another 1000 customer instance with explicit matrix reads much faster:

In [76]: %timeit vrplib.read_instance(f"tmp/Loggi-n1001-k31.txt")
246 ms ± 8.88 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Use numpy testing library

I have been using standard assert statements and numpy.testing assertions. I think it's good to make this more consistent, and to use numpy.testing everywhere.

Standardize output

To support more instances, it's best to standardize the output to a dictionary. Currently I have added extra attributes etc. which I might want to remove as well later.

TODOs

  • Reorganizes code base
  • Standardizes output to dictionary / removes Instance class.
  • Removes DEPOT constant

Ignore -1 in depot section

The -1 in the depot section does not seem to serve any purpose. I think it's OK if we do not raise in that case.

Distance matrix computation performance

It seems reading a large instance is rather slow, most likely due to computing the distance matrix using a Python for loop. E.g. for GH1000 instance, running on Colab:

!pip install vrplib
import vrplib
vrplib.download_instance("RC2_10_8", "RC2_10_8.vrp")
%%timeit
instance = vrplib.read_instance("RC2_10_8.vrp", instance_format="solomon")

3.35 s ± 178 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Roadmap V1.0

The library currently supports reading and downloading instances from the CVRPLIB library. I'm going to work towards a V1 version of this library to make it support more general VRPLIB instances, particularly those from the LKH-3 library. I think that the LKH-3 and CVRPLIB library are currently the best maintained VRP libraries with a consistent format. Moreover, LKH-3 has way more instances than CVRPLIB, which only has CVRP and VRPTW instances.

New features that I want to introduce:

  • Read LKH-3 VRPLIB formatted instances
  • Write VRPLIB formatted instances #14
  • Extensively test VRPLIB reading

Some edge cases (mostly about how to support CVRPLIB):

  • Maybe drop support for downloading CVRPLIB instances? I think it's nice feature to have, but such a feature adds complexity to the library.
    • I'll keep the download feature for now because I think it's quite useful for running quick instances on Colab.
  • Maybe drop support for Solomon/GH instances that are not formatted by VRPLIB standards? I think we need this still because CVRPLIB relies on it.
    • We will not drop support for Solomon-formatted instances. Instead, read_instance will take the style argument which allows to parse different instance formatting styles (i.e., vrplib and solomon).

Issues

Deprecate pkg_resource.read_text

========================================================================================================== warnings summary ==========================================================================================================
tests/download/test_list_names.py::test_list_names[case0]
  /Users/leonlan/Dropbox/cvrplib/vrplib/download/list_names.py:52: DeprecationWarning: read_text is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
    fi = pkg_resource.read_text(__package__, "instance_data.csv")

tests/download/test_list_names.py::test_list_names[case0]
  /opt/homebrew/Cellar/[email protected]/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/resources/_legacy.py:80: DeprecationWarning: open_text is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
    with open_text(package, resource, encoding, errors) as fp:

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform darwin, python 3.11.3-final-0 ----------
Coverage XML written to file coverage.xml

=================================================================================================== 86 passed, 2 warnings in 3.39s ===================================================================================================
leonlan@Leons-Air cvrplib % 

Bug in distance calculation from flattened matrix

Just found a bug in the distance calculation from flattened distance matrices.
The function 'from_flattened' (line 141 in cvrp.py) assumes that
" The numbers in a flattened list correspond the matrix element indices
(1, 0), (2, 0), (2, 1), (3, 0), (3, 1), (3, 2), (4, 0), ..."
However, for the instance E-n13-k4 the flattened list correponds to indices
(0, 1), (0, 2), (0, 3), (0, 4), ... (11,12).
Changing line 150
indices = sorted([(i, j) for (j, i) in combinations(range(n), r=2)])
to
indices = sorted([(i, j) for (i, j) in combinations(range(n), r=2)])
resolves this issue.
However, I do not know whether the original sorting might be correct for some other instances.

Separate read/download instance and solution

To make the code more explicit, I want to make separate functions for reading instances and solutions, and also downloading instances and solutions.

Example:

import cvrplib

# Read
instance = cvrplib.read_instance('/path/to/A-n32-k5.vrp')
solution = cvrplib.read_solution('/path/to/A-n32-k5.sol')

# Download
instance = cvrplib.download_instance('/path/to/A-n32-k5.vrp')
solution = cvrplib.download_solution('/path/to/A-n32-k5.sol')

After finishing #14, I also want to have separate functions for writing instances and solutions:

# Write
cvrplib.write_instance('/path/to/A-n32-k5.vrp', specifications, sections)
cvrplib.write_solution('/path/to/A-n32-k5.sol', routes)

Make download faster

The download and list_instances functions take several seconds to complete. The parsing of the file itself goes very fast, but receiving the response from the server takes a while. I believe it has to do with the server that hosts the CVRPLIB library. If you know how to improve this, please let me know.

Verification functions

It would be nice to provide functions that can verify any VRP-type solution. Of course, we need to make a verification function for each type of VRP-type problem.

I have no idea yet how the interface will look like. But here's an idea:

from cvrplib import read_instance, read_solution, verify

instance = read_instance(instance_path)
solution = read_solution(solution_path)

# Pass the solution and all relevant instance attributes
verify.cvrp(solution, instance['capacity']) 
verify.vrptw(solution, instance['capacity'], instance['duration_matrix'], instance['time_windows'])

Hard-code instance names

It should not be necessary to make a request to the library to obtain all instance names. The instance names don't change over time, only the best known solutions do.

We can simply add a .txt file to this library, which gets updated at every new instances set.

  • Change list_instances to list_names

Read LKH-3 VRPLIB formatted instance

cvrplib.read currently supports instances from CVPRLIB. I want to extend this to be able to read instances for LKH-3 as well. The formatting is not much different I think, so it's mostly about how the codebase is organized.

The output does need to change. I currently use CVRP and VRPTW classes, but I think I want only to output a dictionary.

Download instances and solutions locally

Downloading an instance currently does not save the instance locally. We should rewrite the code to actually download the files to save them locally.

The user would then have to write this:

import vrplib

# Download the instance and solution locally
vrplib.download_instance("X-n101-k25", "/path/to/X-n101-k25.vrp")
vrplib.download_solution("X-n101-k25", "/path/to/X-n101-k25.sol")

# Read the downloaded instance and solution
instance = vrplib.read_instance("/path/to/X-n101-k25.vrp")
solution = vrplib.read_solution("/path/to/X-n101-k25.sol")

Check validity of instances

Except for some edge weight format and types, the current code does not check if the instance data are correct. For example, time windows can be checked for early <= late. Or that all data sections should have enough entries.

As of now, the library is just really about reading existing instances, so there should be no need to checking validity of the instances. But at some point it might be useful to include this as well, combined with #34 and #14.

Write VRP instances

It would be nice to have a function that could also write VRP instances. This was provided in the EURO-NeurIPS competition.

Change package name to cvrplib

I thought it would be helpful to name the package to pycvrplib to make it apparant that the library is written for Python, but I think that's being overly pedantic. I should rename the package to cvrplib, just like tsplib95

Optional distance computation for large instances

CVRPLIB has an XXL instance set (e.g., http://vrp.atd-lab.inf.puc-rio.br/index.php/en/plotted-instances?data=Flanders2). vrplib takes very long to parse these large instances due to the distance matrix (size $O(n^2)$). For Brussels1 (11K nodes), it takes about 1 minute on my laptop and 99.9% time is spent on computing the distance matrix.

I think it's unreasonable to compute the distances matrix for such large instances, so we should probably add an argument to disable the computation of distance matrices.

Plotting functions

We should provide some simple plotting functionalities at some point.

Add support for VRPTW instances

Our package currently does not support reading and downloading VRPTW instances, i.e., instances that belong to the Solomon and Homberger and Gehring set. Support for this will be in a later version. A new parser function needs to be written for these instances.

Host instances and best known solutions

In the long term, we could make a separate repository to maintain the instances and best known solutions.

  • Using Github Actions we can make this process mostly automated.
  • Templates can streamline the submission process
  • We need to write a validator for each problem type. See #27

Redesign test suite

I have been sloppy in writing my tests. Especially with the module rehaul, most of the tests are not in the right place, or not even implemented at all. I think that coverage can be very helpful to detect the missing spots.

TODOs

  • Add coverage
  • Implement tests for parse module
  • Gather all the VRPLIB instances that you can find
    • ORTEC Euro-NeurIPS VRPTW instances

Optional parameters for distance rounding

The current implementation always rounds the distances to the nearest integer. It might be helpful for the non-integral instances to allow for different rounding functions.

Related issue: #7

Verify that instance is VRPLIB format

We should verify that an instance is in VRPLIB format. I don't know yet what needs to be checked (is there a minimum requirement?), but that's something we can figure out along the way.

Timeout when download takes too long

The CVRPLIB website has been offline the entire day. The code currently just hangs waiting for a response. I think we should throw an error if downloading hasn't finished after 20 seconds.

Download full instance and solution sets

Would be nice to download the full instance and solution sets in one function call. I'm not sure yet how to implement this. Some ideas:

  • Provide a new function named download_set, which can be used like
download_set("X", output_dir="instances/")
- Two directions here: 
	- This function downloads the corresponding "X" set zip and unzips it.
	- It downloads all "X" instances and solutions.
  • Allow for globs in download_instance and download_solution.

Instance/solution fields as numpy arrays instead of lists

The array-like fields in the Instance and Solution classes are current of type List or List[List]. I prefer to change this to numpy arrays, as they provide more useful methods when solving VRP problems. The only hurdle is that I'm not sure how to annotate numpy arrays in a meaningful way. Maybe refer to this?

Default argument for path in download & allow dirs

Users must pass a path argument to download_instance.

  • This shouldn't be necessary. If it's unspecified, then it'll just be downloaded and saved to where the function is called from?
  • The file name does not need to be specified. If a dir is provided instead, then save it to that dir with the original file name.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.