Giter VIP home page Giter VIP logo

pgrserver's Introduction

pgrServer

Dijkstra Shortest Path Search Alt text

Driving Distance Isochrone Alt text

Traveling Salesperson Problem Alt text

All Directed Path Alt text

Vehicle Routing Problem (VRP) Alt text

Introduction

pgrServer is a routing service that is able to use pgRouting topologies to load data into a JGraphT graph for very fast searches even with dense networks such as the OpenStreetMap (OSM) data set.

The graph is created at startup when the topology is read from a PostgreSQL database. This graph though can be re-created at regular intervals by making a service request, for networks that have dynamic costs.

And similar to pgRouting, this application is not road navigation centric. This application can be used for a wide variety of networks: i.e. utilities (fiber optic lines), water systems, etc.

As of this version, the following search algorithms are included as a service:

  • Dijkstra ( for dense networks )
  • A-Star ( for dense networks )
  • ContractionHierarchyBidirectionalDijkstra ( for dense networks * )
  • ClosestFirstIterator ( for Driving Distance Isochrone creation )
  • NearestNeighborHeuristicTSP ( for Traveling Salesperson Problem )
  • All Directed Paths ( for sparse networks or short distance search)
  • Bellman-Ford ( for sparse networks )
  • BFS ( for sparse networks )
  • Johnson ( for sparse networks )
  • Floyd-Warshall ( for sparse networks )

(*Note: Initial call to a ContractionHierarchyBidirectionalDijkstra request will take time since a contraction graph will be created first. Subsequent calls will result in a much faster response.)

pgrServer is also able to solve Vehicle Routing Problems (VRP) using the JSprit VRP engine in order to find the optimal set of routes for a fleet of vehicles to traverse orders from a set of customers.

When to use pgrServer

  • When a web service is required to serve route data. pgrServer can be used to easily serve data to a variety of web or mobile application clients.

  • When the network is very dense and pgRouting struggles with long distance searches. pgrServer stores the entire graph into memory at start and can do route searches within the entire graph.

  • When performance is paramount. pgrServer can return routes within ~50 kilometer searches in milliseconds even in very dense networks.

  • When the cost (weight) of the graph is not dynamic. pgrServer can be used when the cost does not have to be computed at each request, since pgrServer only reads the cost whenever the graph is loaded. pgrServer can be forced to re-read the graph for routes that have semi-dynamic costs.

Requirements

  • PostgreSQL > 9.4
  • PostGIS
  • PgRouting or Osm2Po (for topology creation)
  • Maven
  • Tomcat Application Server for deployment

Docker

For convenience, a Docker image can be built for this project. There are a few environment variables that can be optionally set in order for the Docker image to work properly:

  • POSTGRES_HOST: the host IP/fully qualified domain. Default localhost.
  • POSTGRES_PORT: the port designated for the Postgres instance. Default 5432.
  • POSTGRES_DB : the database you. Default pgr.
  • POSTGRES_USER: the Postgres user name. Default postgres.
  • POSTGRES_PASS: the Postgres password. Default postgres.

Note, it is still necessary to prepare the topology with pgRouting or osm2po. Also, the docker usage is advisable for testing or development purposes only. It is highly advisable not to run a production server with the Docker image.

# build the image and spin up the container(s)
docker build -t mbasa/pgrserver:latest .
docker-compose up -d

From here, the application can now be tested by displaying the List of APIs.

Preparing the Topology

  • Create a topology table. Refer to pgRouting's Documentations on the pgr_createTopology function.

  • It is also possible to use OSM2PO to create a topology table using OSM pbf data files. Refer to this pgrServer WIKI for more information on how to use OSM2PO to create topologies for pgrServer.

  • Ensure that there is an index on an unique id field, an index on the source field, an index on the target field, and a spatial index on the geometry field of the topology table.

  • Create a View Table pgrserver based on the topology table that will contain the following fields: id, source, target, cost, reverse_cost, length, geom.

CREATE VIEW pgrserver AS SELECT id,node_from AS source,node_to AS target,cost, reverse_cost, length, wkb_geometry AS geom FROM kanto ;

Note:

  1. the length column has to be in meters(m) units.

  2. the reverse_cost value has to be greater than the cost in order for the edge to be considered as a one-way street.

Getting the Application

The latest WAR file of the application can be downloaded from each stable release of this repository, and can either be placed in a Tomcat Application Server or run stand alone via the command:

java -jar pgrServer.war

Alt text

Building the Application

  • Edit src/main/resources/application.properties and modify the PostgreSQL Database URL and login parameters.

  • Create a WAR file that can be deployed to a Tomcat Server.

    mvn clean install -DskipTests
  • Or run and test the application with the built-in Tomcat container.
    mvn spring-boot:run

Display the List of APIs

The list of APIs can be viewed by displaying the Swagger page:

http://localhost:8080/pgrServer/

Alt text

Reload the Graph

To reload the graph if the cost has changed, send a POST request with the authcode parameter value. The authcode value can be set by updating the installed pgrs_auth table in the PostgreSQL database.

curl -X POST -F "authcode=abc12345" "http://localhost:8080/pgrServer/api/graphreload"

Viewing the Data

A demo application, pgrServerDemo , has been created to easily display selected features of pgrServer.

Also, pgrServer returns a GeoJSON object for the created route or driving distance polygon, hence any application that supports GeoJSON can be used to view the results.

To quickly view the results, GeoJSONLint web service can be used:

http://www.geojsonlint.com

It is also possible to use the result as a Vector Layer in QGIS by doing:

Layer -> Add Layer -> Add Vector Layer

and set the protocol to HTTP and add the URL request of pgrServer.

Tomcat Deployment

Since this is a memory intensive application, configuring Tomcat to use more memory will be necessary, especially with very large data sets ( greater than or equal to 9 million edges ) to avoid OutOfMemoryError problems. The argument settings below can be used to let Tomcat allocate up to 8GB of memory for its usage:

-Xms2048m -Xmx8192m -server

Refer to Tomcat documentation for more information on memory allocation.

When using the spring-boot application's Tomcat, the following can be performed to pass the required memory parameters:

mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Xms2048m -Xmx8192m"

pgrserver's People

Contributors

mbasa avatar nilsnolde avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

pgrserver's Issues

docker & docker-compose

I think it'd make a lot of sense for this repo to have a docker setup. A lot of things could be automated this way, like spinning up a PostGIS/Tomcat instance, creating graphs, easily running on major OSs etc. Would really lower the barrier for getting started.

I can help out a little early next year if you want. Some suggestions:

Looking forward to give this a try!

First feedback

I tried it now for a graph spanning all of Austria:

  1. generate the topology with osm2po: java -Xmx2g -jar osm2po-core-5.3.2-signed.jar cmd=c prefix=austria austria-latest.osm.pbf
  2. load the resulting sql: psql -U postgres -W -h localhost -d gis_test -f austria/austria_2po_4pgr.sql
  3. create the view following the README: CREATE VIEW pgrserver AS SELECT id,source,target,cost,geom_way AS geom FROM austria_2po_4pgr ;

That leads to:

  • 1.7 Mio (bidirectional) edges
  • ~ 2 GB RAM base graph
  • ~ 40 seconds to load the graph from PG

That I'm actually quite happy with, good performance, bit too much RAM maybe.

When I try to query even short distances I get an empty response (tried Dijkstra, AStar, CH). If I actually try from on end of Austria to the other (Vienna -> Liechtenstein), CH is running for 10 minutes at > 7 GB RAM on 8 threads before I canceled.

However, small extracts (I tried Sydney real quick) with the same workflow works well!

Did you ever try a bigger graph?

Turn restricted shortest path (TRSP)

Hi Mario,
this project looks amazing. It solves almost all of our pgr hurdles. We were just considering writing something similar to pgrServer, but now we don't have to ;). But there is one major feature missing for us and that is support for some kind of TRSP algorithm (like pgr_trsp). Do you have any plans to support TRSP algorithm?

Error while running the docker image

which building using the:
docker-compose build pgrserver
we get: ERROR: No such service: pgrserver

Then, I checked the docker-compose.yml and tried:
docker-compose build pgrServer
we get some exception: pgrServer uses an image, skipping

The issue with
docker-compose up

Pulling pgrServer (mbasa/pgrserver:latest)...
ERROR: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.

Continue with the new image? [yN]y
Pulling pgrServer (mbasa/pgrserver:latest)...
ERROR: pull access denied for mbasa/pgrserver, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Add network changes to the network

Hi,

thanks a lot for this interesting project. I was wondering if it would be very difficult to implement a user-based network change capability into the project.

e.g. imagine one user would like to add a new network link to the graph and wants to perform routing operations with this changed graph. However, at the same time other users which are using the same graph should not be affected by this network change. Instead, their routing operation should use no changes or their own changes.

I was wondering if there is any direction you could point me in. As we are currently using a pgRouting for this but we would favor a solution loading the network in memory due to performance.

Cheers,

Elias

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.