Giter VIP home page Giter VIP logo

hgs-cvrp's Introduction

CI_Build

HGS-CVRP: A modern implementation of the Hybrid Genetic Search for the CVRP

This is a modern implementation of the Hybrid Genetic Search (HGS) with Advanced Diversity Control of [1], specialized to the Capacitated Vehicle Routing Problem (CVRP).

This algorithm has been designed to be transparent, specialized, and highly concise, retaining only the core elements that make this method successful. Beyond a simple reimplementation of the original algorithm, this code also includes speed-up strategies and methodological improvements learned over the past decade of research and dedicated to the CVRP. In particular, it relies on an additional neighborhood called SWAP*, which consists in exchanging two customers between different routes without an insertion in place.

References

When using this algorithm (or part of it) in derived academic studies, please refer to the following works:

[1] Vidal, T., Crainic, T. G., Gendreau, M., Lahrichi, N., Rei, W. (2012). A hybrid genetic algorithm for multidepot and periodic vehicle routing problems. Operations Research, 60(3), 611-624. https://doi.org/10.1287/opre.1120.1048 (Available HERE in technical report form).

[2] Vidal, T. (2022). Hybrid genetic search for the CVRP: Open-source implementation and SWAP* neighborhood. Computers & Operations Research, 140, 105643. https://doi.org/10.1016/j.cor.2021.105643 (Available HERE in technical report form).

We also recommend referring to the Github version of the code used, as future versions may achieve better performance as the code evolves. The version associated with the results presented in [2] is v1.0.0.

Other programming languages

There exist wrappers for this code in the following languages:

  • C: The C_Interface file contains a simple C API
  • Python: The PyHygese package is maintained to interact with the latest release of this algorithm
  • Julia: The Hygese.jl package is maintained to interact with the latest release of this algorithm

We encourage you to consider using these wrappers in your different projects. Please contact me if you wish to list other wrappers and interfaces in this section.

Scope

This code has been designed to solve the "canonical" Capacitated Vehicle Routing Problem (CVRP). It can also directly handle asymmetric distances as well as duration constraints.

This code version has been designed and calibrated for medium-scale instances with up to 1,000 customers. It is not designed in its current form to run very-large scale instances (e.g., with over 5,000 customers), as this requires additional solution strategies (e.g., decompositions and additional neighborhood limitations). If you need to solve problems outside of this algorithm's scope, do not hesitate to contact me at [email protected].

Compiling the executable

You need CMake to compile.

Build with:

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -G "Unix Makefiles"
make bin

This will generate the executable file hgs in the build directory.

Test with:

ctest -R bin --verbose

Running the algorithm

After building the executable, try an example:

./hgs ../Instances/CVRP/X-n157-k13.vrp mySolution.sol -seed 1 -t 30

The following options are supported:

Call with: ./hgs instancePath solPath [-it nbIter] [-t myCPUtime] [-bks bksPath] [-seed mySeed] [-veh nbVehicles] [-log verbose]
[-it <int>] sets a maximum number of iterations without improvement. Defaults to 20,000                                     
[-t <double>] sets a time limit in seconds. If this parameter is set, the code will be run iteratively until the time limit           
[-seed <int>] sets a fixed seed. Defaults to 0                                                                                    
[-veh <int>] sets a prescribed fleet size. Otherwise a reasonable UB on the fleet size is calculated                      
[-round <bool>] rounding the distance to the nearest integer or not. It can be 0 (not rounding) or 1 (rounding). Defaults to 1. 
[-log <bool>] sets the verbose level of the algorithm log. It can be 0 or 1. Defaults to 1.                                       

Additional Arguments:
[-nbIterTraces <int>] Number of iterations between traces display during HGS execution. Defaults to 500
[-nbGranular <int>] Granular search parameter, limits the number of moves in the RI local search. Defaults to 20               
[-mu <int>] Minimum population size. Defaults to 25                                                                            
[-lambda <int>] Number of solutions created before reaching the maximum population size (i.e., generation size). Defaults to 40
[-nbElite <int>] Number of elite individuals. Defaults to 5                                                                    
[-nbClose <int>] Number of closest solutions/individuals considered when calculating diversity contribution. Defaults to 4     
[-nbIterPenaltyManagement <int>] Number of iterations between penalty updates. Defaults to 100
[-targetFeasible <double>] target ratio of feasible individuals between penalty updates. Defaults to 0.2
[-penaltyIncrease <double>] penalty increase if insufficient feasible individuals between penalty updates. Defaults to 1.2
[-penaltyDecrease <double>] penalty decrease if sufficient feasible individuals between penalty updates. Defaults to 0.85

There exist different conventions regarding distance calculations in the academic literature. The default code behavior is to apply integer rounding, as it should be done on the X instances of Uchoa et al. (2017). To change this behavior (e.g., when testing on the CMT or Golden instances), give a flag -round 0, when you run the executable.

The progress of the algorithm in the standard output will be displayed as:

It [N1] [N2] | T(s) [T] | Feas [NF] [BestF] [AvgF] | Inf [NI] [BestI] [AvgI] | Div [DivF] [DivI] | Feas [FeasC] [FeasD] | Pen [PenC] [PenD]

[N1] and [N2]: Total number of iterations and iterations without improvement
[T]: CPU time spent until now
[NF] and [NI]: Number of feasible and infeasible solutions in the subpopulations 
[BestF] and [BestI]: Value of the best feasible and infeasible solution in the subpopulations 
[AvgF] and [AvgI]: Average value of the solutions in the feasible and infeasible subpopulations 
[DivF] and [DivI]: Diversity of the feasible and infeasible subpopulations
[FC] and [FD]: Percentage of naturally feasible solutions in relation to the capacity and duration constraints
[PC] and [PD]: Current penalty level per unit of excess capacity and duration

Code structure

The main classes containing the logic of the algorithm are the following:

  • Params: Stores the main data structures for the method
  • Individual: Represents an individual solution in the genetic algorithm, also provides I/O functions to read and write individual solutions in CVRPLib format.
  • Population: Stores the solutions of the genetic algorithm into two different groups according to their feasibility. Also includes the functions in charge of diversity management.
  • Genetic: Contains the main procedures of the genetic algorithm as well as the crossover
  • LocalSearch: Includes the local search functions, including the SWAP* neighborhood
  • Split: Algorithms designed to decode solutions represented as giant tours into complete CVRP solutions
  • CircleSector: Small code used to represent and manage arc sectors (to efficiently restrict the SWAP* neighborhood)

In addition, additional classes have been created to facilitate interfacing:

  • AlgorithmParameters: Stores the parameters of the algorithm
  • CVRPLIB Contains the instance data and functions designed to read input data as text files according to the CVRPLIB conventions
  • commandline: Reads the line of command
  • main: Main code to start the algorithm
  • C_Interface: Provides a C interface for the method

Compiling the shared library

You can also build a shared library to call the HGS-CVRP algorithm from your code.

mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -G "Unix Makefiles"
make lib

This will generate the library file, libhgscvrp.so (Linux), libhgscvrp.dylib (macOS), or hgscvrp.dll (Windows), in the build directory.

To test calling the shared library from a C code:

make lib_test_c
ctest -R lib --verbose

Contributing

Thank you very much for your interest in this code. This code is still actively maintained and evolving. Pull requests and contributions seeking to improve the code in terms of readability, usability, and performance are welcome. Development is conducted in the dev branch. I recommend to contact me beforehand at [email protected] before any major rework.

As a general guideline, the goal of this code is to stay simple, stand-alone, and specialized to the CVRP. Contributions that aim to extend this approach to different variants of the vehicle routing problem should usually remain in a separate repository. Similarly, additional libraries or significant increases of conceptual complexity will be avoided. Indeed, when developing (meta-)heuristics, it seems always possible to do a bit better at the cost of extra conceptual complexity. The overarching goal of this code is to find a good trade-off between algorithm simplicity and performance.

There are two main types of contributions:

  • Changes that do not impact the sequence of solutions found by the HGS algorithm when running ctest and testing other instances with a fixed seed. This is visible by comparing the average solution value in the population and diversity through a test run. Such contributions include refactoring, simplification, and code optimization. Pull requests of this type are likely to be integrated more quickly.
  • Changes that impact the sequence of solutions found by the algorithm. In this case, I recommend to contact me beforehand with (i) a detailed description of the changes, (ii) detailed results on 10 runs of the algorithm for each of the 100 instances of Uchoa et al. (2017) before and after the changes, using the same termination criterion as used in 2.

License

License

hgs-cvrp's People

Contributors

igormcoelho avatar n-wouda avatar th-yoon avatar vidalt avatar wouterkool avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hgs-cvrp's Issues

[bug] segfault if number of vehicles is too low

Thank you for the implementation, it gives very promising results in our use case. However, it seems the program segfaults if the number of vehicles is too low for the particular problem:

Program$ ./genvrp ../Instances/CVRP/CMT1.vrp  ../test.sol -seed 1 -veh 2
----- READING DATA SET: ../Instances/CVRP/CMT1.vrp
----- INSTANCE LOADED WITH 50 CLIENTS AND 2 VEHICLES
----- BUILDING INITIAL POPULATION
Segmentation fault (core dumped)

The code above runs the algorithm and returns the solution if -veh is increased to 5.

duration_limit problem

Is it possible that setting duration_limit when we have a large quantity of data brings the algorithm into a soft lock state where it fails to build the initial population ?

Git action recently failing on Windows

For some reason, the git actions designed to test the code started failing recently after a commit that only involved the README.md (https://github.com/vidalt/HGS-CVRP/actions/runs/3420070648). As the considered update had no impact on the code, it seems that some configuration or upgrade in the way git actions run makes the experiment fail. So far, we know that only the Windows action fails (the others get automatically canceled as a consequence). @chkwon tested the code locally on Windows, and everything worked fine. I'm not personally an expert on Git actions and could not pinpoint so far where the issue is; anyone has a clue?

Temporal Distance or Spatial DIstance

Is the distance matrix that we pass as input to the algorithm made of spatial or temporal distances? Because looking at the Individual.cpp section, it seems that the distances are indeed temporal! In case the distance are spatial, is it possible to use temporal distances?
Thank you in advance for your attention.

Unexpected Calculation of BiasedFitnesses

I am confused about the calculation of BiasedFitnesses.

The calculation of BiasedFitnesses seems not consistent with that given in the paper (Hybrid genetic search for the CVRP: Open-source implementation and SWAP* neighborhood)

In this HGS-CVRP code it looks like this:

double divRank = (double)i / (double)(pop.size() - 1); // Ranking from 0 to 1
double fitRank = (double)ranking[i].second / (double)(pop.size() - 1);
if ((int)pop.size() <= params.ap.nbElite) // Elite individuals cannot be smaller than population size
pop[ranking[i].second]->biasedFitness = fitRank;
else
pop[ranking[i].second]->biasedFitness = fitRank + (1.0 - (double)params.ap.nbElite / (double)pop.size()) * divRank;

In the paper it is formulated as:
drawing

I suppose the difference is that the divRank in the source code is actually the ranking of fitness while fitRank is the ranking of diversity.

Is it possible to let the vehicle return to the depot and start a new route?

It seems the current implementation enforces one route per vehicle. Assume we have one vehicle with CAPACITY 1 and two nodes each with DEMAND 1, and the DURATION is set to 5. Let the travel time between Depot -> 1 -> Depot equal 2 and Depot -> 2 -> Depot equal 2. I would like to extend the current implementation in such a way that it creates the following route: Route #1: 0 1 0 2 0. Can you maybe give me a hint how to start to implement this feature?

Of course in the actual use case we have multiple vehicles and many shops to serve.

Broken pairs distance is not symmetric

I am bringing our VRPTW solver under test, by writing unit tests for most code points. It's an immense amount of work, but it's going well. This testing effort forces me to look at every bit of code in considerable detail, to determine how it works and how to encode that in meaningful test cases.

I am currently testing our implementation of the broken pairs distance (BPD) measure. HGS-CVRP also uses this, and ours is a direct descendant of your efforts. There are two issues that I noticed with our relatively unmodified broken pairs distance:

  1. The code (in various places) assumes it is symmetric, that is, BPD(X, Y) == BPD(Y, X). For example, here, where it is computed only once but used in both ways:

    for (Individual * myIndividual2 : subpop)
    {
    double myDistance = brokenPairsDistance(*myIndividual,*myIndividual2);
    myIndividual2->indivsPerProximity.insert({ myDistance, myIndividual });
    myIndividual->indivsPerProximity.insert({ myDistance, myIndividual2 });
    }

  2. The BPD distance as currently implemented is not symmetric. A fairly minimal example are individuals with the following two route sets: X = {{1, 2, 3, 4}, {}, {}} and Y = {{1, 2}, {3}, {4}}. Now, BPD(X, Y) = 2, but BPD(Y, X) = 3. The issue is a double increment for a single client when both conditions here are true:

    double Population::brokenPairsDistance(const Individual & indiv1, const Individual & indiv2)
    {
    int differences = 0;
    for (int j = 1; j <= params.nbClients; j++)
    {
    if (indiv1.successors[j] != indiv2.successors[j] && indiv1.successors[j] != indiv2.predecessors[j]) differences++;
    if (indiv1.predecessors[j] == 0 && indiv2.predecessors[j] != 0 && indiv2.successors[j] != 0) differences++;
    }
    return (double)differences / (double)params.nbClients;
    }

    My fix for this has been to merge the two conditions into a single update, as follows:

    if ((indiv1.successors[j] != indiv2.successors[j] && indiv1.successors[j] != indiv2.predecessors[j])
            || (indiv1.predecessors[j] == 0 && indiv2.predecessors[j] != 0 && indiv2.successors[j] != 0))
        differences++;

    This passes all the symmetry tests I have so far thrown at it.

I am not sure if this is a "bug", but is definitely something I struggled understanding for a while. Am I misunderstanding BPD as it is implemented now?

Last offspring kept after resetting the algorithm

For tests involving successive runs until a time limit is reached, the algorithm/population is reset each time maxIterNonProd is reached. From my analysis of the code, the population is completely reset, but the next run starts with the last offspring of the previous run. Is that intended behavior? I'm asking this because it is very important for our current research to know if each run is completely independent from other runs during the same execution of the program.

In this piece of code the population and the iteration counter are reset, but not offspring:

if (params.ap.timeLimit != 0 && nbIterNonProd == params.ap.nbIter)
{
population.restart();
nbIterNonProd = 1;
}

And offspring is used in the beginning of the main loop:

crossoverOX(offspring, population.getBinaryTournament(),population.getBinaryTournament());
/* LOCAL SEARCH */
localSearch.run(offspring, params.penaltyCapacity, params.penaltyDuration);

crossoverOX method does not seem to completely overwrite offspring: some of the not-overwritten values are used in localSearch.run (namely chromR).

Thanks!

Question about divRank and fitRank in updateBiasedFitnesses()

As mentioned in the paper "Vidal, T. (2022). Hybrid genetic search for the CVRP: Open-source implementation and SWAP* neighborhood.", the fitness in HGS is based on objective value and diversity considerations with formula (1).

The diversity is measured as its average broken-pairs distance; We found the calculation of diversity rank in the function updateBiasedFitnesses() is as follows:
double divRank = (double)i / (double)(pop.size() - 1);
while the solution quality is
double fitRank = (double)ranking[i].second / (double)(pop.size() - 1);

Are these two calculations reversed? Maybe it should be:
double divRank = (double)ranking[i].second / (double)(pop.size() - 1);
double fitRank = (double)i / (double)(pop.size() - 1);

since the ranking store the averageBrokenPairsDistance
ranking.push_back({-pop[i]->averageBrokenPairsDistanceClosest(params->nbClose),i});

compiling a new shared library

I apologize for all these issues, but these are doubts I need to clarify: I am using your algorithm along with a Python wrapper created by Changhyun Kwon. By installing the hygese library with pip, I am able to run the algorithm successfully in Python (except for the issue related to the previous duration_limit issue). I have modified the C code and compiled the shared library (following the instructions on your algorithm's GitHub page) to allow Changhyun Kwon's Python wrapper to use my modified version of C (the library in question is libhgscvrp.so). The tests conducted via the terminal are good and end with success. However, when I try to use the algorithm with the Python wrapper, I consistently get 0 iterations. Clearly, there is an issue! The folder where I copy libhgscvrp.so is the same folder where the "functional" library is installed using "pip install hygese". Additionally, there are differences in size between the library compiled following your instructions and the one downloaded via pip. Could you please provide some solutions to this problem?

I apologize for the inconvenience and thank you for your attention while awaiting your response.

The plan for improving the solver interface

Dev Environment

  • CMake, .gitignore #8 #9
  • Use CMake and ctest in CI actions to check the solution correctness #14

Interface

  • time measure improvement, floating timeLimit47848a5 623d821
  • refactoring Params, file read by CVRPLIB #13
  • Algorithm parameters control #16
  • extern "C" interface for Python/Julia wrappers #18

New features

  • verbose control for solver logs #15
  • turn off SWAP* local search for dist_mtx only instances #20
  • better handling of duration constraints
  • improve CVRPLIB to handle more general .vrp files, e.g. instances without coordinates

Testing

  • test for instances with duration constraints #22

What if `start` is greater than `end` ?

When I use a instance with small customers for example 15, then start is greater than end. so the while loop can't be stopped.
Don't we need to check whether start is greater than end?

// Picking the beginning and end of the crossover zone
std::uniform_int_distribution<> distr(0, params.nbClients-1);
int start = distr(params.ran);
int end = distr(params.ran);
// Avoid that start and end coincide by accident
while (end == start) end = distr(params.ran);
// Copy from start to end
int j = start;
while (j % params.nbClients != (end + 1) % params.nbClients)
{
result.chromT[j % params.nbClients] = parent1.chromT[j % params.nbClients];
freqClient[result.chromT[j % params.nbClients]] = true;
j++;
}

package naming

This is to brainstorm wrapper package names of Julia and Python.

Python

There are three names required, which can be the same, or different.

  • Project Name (say, the GitHub repository name)
  • pip package name (to be used when we install the package: pip install packagename)
  • module name (to be used when we import the module: import modulename)

Examples:

  • PyTorch / torch / torch
  • PyConcorde / pyconcorde / concorde
  • elkai / elkai / elkai (a python wrapper for LKH)
  • NumPy / numpy / numpy
  • PyVroom / pyvroom / vroom

Python style guide says:

Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. Python packages should also have short, all-lowercase names, although the use of underscores is discouraged.

I think the Project Name can be anything, but it will be better to make the package name and the module name the same.

Some ideas for the package/module name:

  1. hgscvrp
  2. hygese
  3. hygesecvrp
  4. hgs_cvrp (the use of underscores is discouraged)
    ...

Julia

There is only one name to determine: PackageName.jl.

The GitHub repo will be called PackageName.jl.
When we install, we will type add PackageName.
When we use it in the code, we will type using PackageName.

The package naming guidelines are given here. It needs to be CamelCase without hyphen or underscore.

Some examples:

  • DifferentialEquations.jl
  • Pluto.jl
  • DiffEqFlux.jl
  • NeuralPDE.jl
  • Convex.jl
  • PATHSolver.jl

Julia generally doesn't like all capital letters, but we could probably make a case, since the original C++ code is called HGS-CVRP. In that case, it will be called HGSCVRP.jl, which could a little difficult to read and type.

Some ideas:

  1. HGSCVRP.jl
  2. Hygese.jl
  3. HygeseCVRP.jl
  4. HgsCvrp.jl
  5. HybridGeneticSearchCVRP.jl
    ...

In general, I think the benefit of having both HGS and CVRP in the name is clear. On the other hand, using only HGS or its variant can leave the door open to the possibility that the package extends to HGS like algorithms applied in other types of routing/transportation problems. But this may not be important at this moment.

Please let me know if you have any suggestions. To satisfy the two different style guidelines with a single name for both Python and Julia, I ended up with hygese or Hygese.jl, while keeping usability, readability, and consistency, but sacrificing some clarity. But maybe we don't need to use the same name for Python and Julia. Let me know what you think.

Branch Management

The latest updates to the repository had been done over the dev branch but stashed+merged on the main branch. This had led to a different commit history on both branches, which is problematic for a long-lived branch such as dev. To correct this situation, I have made a copy of dev into dev-old to preserve access to un-stashed older commits, and resetted dev to the main branch head.

Therefore, before editing the code, make sure you have checked out the current state of the dev branch. Contact me if you encounter any issues. In the future, I will only register clean (aggregated commits) on the dev branch and directly merge them (without stash) into the main, so we will not encounter again this issue.

Kind Regards,
--Thibaut

EDGE_WEIGHT_FORMAT keyword in vrp file is not supported

Hello, I tried simple vrp file that works in lkh-3 and in your solver it is not supported. Can the solver find a solution for ACVRP instances ?

NAME : SIMPLE_TEST
TYPE : ACVRP
EDGE_WEIGHT_TYPE : EXPLICIT
EDGE_WEIGHT_FORMAT : FULL_MATRIX
DIMENSION : 3
VEHICLES : 2
CAPACITY : 2
EDGE_WEIGHT_SECTION
0 2 0 100 0 1000 0 0 0
DEMAND_SECTION
1 1
2 1
3 0
DEPOT_SECTION
3
EOF

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.