Giter VIP home page Giter VIP logo

node-osrm's Issues

Handle failed loading of data referenced by server.ini

Currently OSRM just prints to std::cerr and calls exit if data, like the .hsgr, is not found as referenced in the server.ini (in typedefs.h -> ERR). This makes it impossible for the node module to recover from an error like this.

Ideally OSRM could throw an exception in QueryObjectsStorage.cpp:36.

Migrate binaries from "node-osrm" bucket to "mapbox-node-binary" bucket

The current batch of binaries are hosted under my personal aws account, but these should be migrated to a Mapbox controlled account so other committers on this project can have admin access to the bucket.

To do this I will:

  • - Upgrade node-osrm master to node-pre-gyp 0.5.x - 6b5b6d3
  • - Copy all existing binaries from https://node-osrm.s3.amazonaws.com to https://mapbox-node-binary.s3.amazonaws.com/osrm/ (note the osrm prefix)
  • - Point package.json binary/host value at mapbox-node-binary - 495eff5
  • - Re-encode the node-pre-gyp publishing keys in .travis.yml with new keys that work to publish to this new location - f48cfa4
  • - In a few months once a new node-osrm tag is out and most people have upgraded I'll delete the node-osrm bucket.

boost program options symbol error on ubuntu precise

> [email protected] test /home/ubuntu/node-OSRM
> mocha -R spec


module.js:356
  Module._extensions[extension](this, filename);
                               ^
Error: /home/ubuntu/node-OSRM/lib/_osrm.node: undefined symbol: _ZNK5boost15program_options29value_semantic_codecvt_helperIcE5parseERNS_3anyERKSt6vectorISsSaISsEEb
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)

Adapting to differently named boost libraries

Boost libs sometimes have -mt at the end to signify they are the multithreaded versions. After Project-OSRM/osrm-backend@4a3db7e node-osrm now needs to link all dependent libraries so we need to know their exact name.

So, builds against homebrew based boost will start failing at 1c51388 until we come up with a solution.

Possibilities are:

  • depend on OSRM to provide a pkg-config that provides all library names
  • come up with some custom node-osrm script to find and detect boost library names

I think pkg-config is ideal, just need to start testing this post Project-OSRM/osrm-backend@8383222

bootstrap test data

Create network data on the fly before tests are run using a small osm extract

Small, but representative sample data for node-osrm tests

Ideally we could store a very small set of data for the tests to make it dead easy to test node-osrm on different platforms quickly (without having to download and wait to process data with osrm-extract and osrm-prepare). Is the data portable across architectures or will we need to generate copies for both 32 bit and 64 bit systems? Is the data fairly portable across node-osrm versions?

API suggestion

More JSy API:

  • No separate Options or Query objects. JS idiom is just to pass options hashes directly.
  • new osrm.Engine({ini: <path>, sharedMemory: <boolean>}) (better would be to eliminate the assumption of having a file on disk with options and pass them directly -- don't know enough about OSRM to gauge if that's feasible)
  • Engine#route(options, callback)
  • Engine#locate(latLon, callback)
  • Engine#nearest(latLon, callback)
  • Do we even need sync versions of these?
  • Engine functions return JS objects to callback rather than a JSON string.

Shared Memory Question

If I have the shared memory flag on, and have the .ini file pointing to .osrm files, does it still try to load those files? I tried to have the location of the files pointed to in the .ini files point to invalid locations, but it just throws an error that the files don't exist.

Test shared memory usage from node

node-osrm latest now builds and runs against the latest develop code from OSRM with shared memory support. But current defaults are in place that do not turn on shared memory usage. This ticket stands to track actually testing it from node.js

compile error with http reply

Looks like we need to catch up with Project-OSRM/osrm-backend@cabaad4#diff-ce251cb4263f3c33dfb996c10931b6f2

In file included from ../src/node_osrm.cpp:15:
../src/engine.hpp:106:55: error: no member named 'c_str' in 'std::vector<std::basic_string<char>, std::allocator<std::basic_string<char> > >'
    return scope.Close(String::New(osrm_reply.content.c_str()));
                                   ~~~~~~~~~~~~~~~~~~ ^
../src/engine.hpp:165:25: error: no viable overloaded '='
        closure->result = osrm_reply.content;
        ~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~
/usr/include/c++/4.2.1/bits/basic_string.h:499:7: note: candidate function not viable: no known conversion from 'std::vector<std::string>' to
      'const std::basic_string<char>' for 1st argument
      operator=(const basic_string& __str) 

Installation

Installation is currently non-standard and currently depends on a clone of this repo being next to a built Project-OSRM directory. What would need to happen in order to it possible to install by simply running npm install node-osrm?

More tests for engine initialization

Possible tests:

  • - invalid Port, IP, and Threads values in server.ini.
  • - a single missing data reference: e.g. all data references are valid except edgesData points to a corrupt file.

test shared memory mode

The existing shared memory test never actually tested shared memory, and is now disabled. It will requrie some pre-test setup to get working.

Disabling INFO and WARN std::cerr

Ideally, for node-osrm, there could be a way to disable this OSRM output so that meaningful logging could be controlled by the application calling into node-osrm

Missing support for 'nearest' and 'locate' queries

It should be possible to run 'nearest' and 'locate' queries in addition to 'viaroute'. Something like this for nearest:

var query = new osrm.Query({
    service: "nearest",
    coordinates: [[52.4224,13.333086]]
});
var sync_result = engine.run(query);

or this for locate:

query = new osrm.Query({
    service: "locate",
    coordinates: [[52.4224,13.333086]]
});
sync_result = engine.run(query);

Error when launching node-osrm

When I load node-osrm with data files that correspond to the entire planet, I get this error:

var engine = new osrm.Engine(opts);
^
Error: std::bad_alloc
at Object. (/home/ubuntu/routing_osrm_nodejs/server.js:10:14)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.runMain (module.js:492:10)
at process.startup.processNextTick.process._tickCallback (node.js:245:9)

simplify test data needs

Berlin extract was helpful for early testing, but since OSRM core has extensive tests, I think we can scale back the testing extract size in node-osrm. A smaller size will help devs get set up faster and will ensure we don't hit travis timeouts (saw these today).

load testing

Goal:

  • Put heavy load on node-osrm.Engine.run() to ensure that the async approach to running the route query holds up (does not hang or crash) and then produce initial profiling results to ensure no bottlenecks exist in the bindings themselves.

Tasks:

  • Write a small http server on top of the node-osrm
  • Devise a set of random route requests against the berlin test dataset
  • Consider adapting the https://github.com/mapbox/bench tool for making the requests and timing responses.
  • Run these tests on various hardware

Shared Memory With OSRM-Routed

Is it possible to have node-osrm use the same shared memory with an osrm-routed instance? This would also be extremely helpful!

return value of libOSRM

@springmeyer @jfirebaugh

So far, libOSRM returns a JSON formatted return string. Would it make more sense in terms of efficiency, and the way NodeJS would like to have things, to return an object that has a certain interface that NodeJS can use.

The idea is to expose some object with a well-known interface/structure that can be used in any subsequent transformation of the result. I'd like to save some time on (costly) string generation/duplication.

IP and Port in INI File

Does the IP address and Port in the .ini file point to the address and port of the OSRM server that node-osrm will be connecting to? Does this indicate then that node-osrm is sending and receiving to the engine through a socket?

Multiple Connections

If I use node-osrm as my bindings to OSRM, does this allow me to have multiple simultaneous connections to OSRM?

Linker problems

Linking fails on OS X with home-brew installed boost. It comes with the -mt suffix. Leads to:

ld: library not found for -lboost_thread
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Release/osrm.node] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/Cellar/node/0.10.22/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:267:23)
gyp ERR! stack     at ChildProcess.EventEmitter.emit (events.js:98:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:789:12)
gyp ERR! System Darwin 13.0.0
gyp ERR! command "node" "/usr/local/Cellar/node/0.10.22/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/dennisluxen/Coding/node-osrm
gyp ERR! node -v v0.10.22
gyp ERR! node-gyp -v v0.11.0
gyp ERR! not ok 

Nearest Road Point with node-osrm

Is it possible to get the nearest road point through node-osrm? I've only seen routing requests calculated. I was hoping to have the APIs that are available through the OSRM http API available through node-osrm.

Provide ini options as a JS object

We could get a slightly simpler configuration if we were able to pass a hash into the Options constructor in addition to an ini file path.

new osrm.Options({
    threads: 8,
    hsgrData: 'path/to/data'
    ...
}, true);

Symbol not found: _g_GIT_DESCRIPTION

It appears that g_GIT_DESCRIPTION is not available in libOSRM.dylib and therefore ./Util/GitDescription.cpp may need to be added to libOSRM. The result is that when running node-osrm on OS X (at least) we hit:

Error: dlopen(/Users/dane/projects/Project-OSRM/node-osrm/lib/_osrm.node, 1): Symbol not found: _g_GIT_DESCRIPTION
  Referenced from: /Users/dane/projects/Project-OSRM/node-osrm/lib/_osrm.node
  Expected in: dynamic lookup

Failure to build on RedHat

When I try to build or install node-osrm on RedHat, I get the following error:

module.js:485
  process.dlopen(filename, module.exports);
          ^
Error: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/mobius/node-osrm-master-3/lib/binding/osrm.node)
    at Object.Module._extensions..node (module.js:485:11)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:362:17)
    at require (module.js:378:17)
    at Object.<anonymous> (/home/mobius/node-osrm-master-3/lib/osrm.js:6:15)
    at Module._compile (module.js:449:26)
    at Object.Module._extensions..js (module.js:467:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)

What can I do to resolve this issue?

Shared memory isn't working with node-osrm

I'm using OSRM-develop (ce784e0491) and doing this to load the shared memory:

$ osrm-datastore /absolute/path/to/data.osrm

I can see the shared memory with ipcs -m. osrm-routed is able to access this shared memory.

Then I'm using node-osrm like this, where data.ini contains same absolute paths to data:

var opts = new osrm.Options("./data.ini", true /*used shared memory*/);
var engine = new osrm.Engine(opts);

node-osrm works, but it's not using the shared memory. I can manually free the shared memory segment with ipcrm -m {shmid} and my node app still does routing, for example.

Any ideas? Is it because I'm not using the latest version of OSRM, perhaps?

explicit Engine shutdown

On the C++ side an OSRM engine has non-memory resources that are deallocated when it's destructed right? I think we need to expose that explicitly on the JS side so it doesn't leak if JS garbage isn't immediately collected.

What I'm seeing running tests with shared memory that create an Engine per test case is that after a few tests, Engine creation starts failing with "Error: Too many open files".

Enable skipped tests

Need to enable the two skipped/pending (blue tests) once upstream issues are fixed:

@DennisOSRM - can you let me know if those issues are ones you'd like to jump on or would prefer me to take a look at?

Once solved these tests can be enabled by doing:

diff --git a/test/osrm.test.js b/test/osrm.test.js
index e03ba19..1664415 100644
--- a/test/osrm.test.js
+++ b/test/osrm.test.js
@@ -16,7 +16,7 @@ describe('osrm', function() {
     });

     // @TODO not safe yet to test: https://github.com/DennisOSRM/Project-OSRM/issues/692
-    it.skip('should throw if ini file is blank', function(done) {
+    it('should throw if ini file is blank', function(done) {
         assert.throws(function() { new osrm.Engine("./test/data/bogus.ini"); },
         /@TODO/);
         done();
@@ -38,7 +38,7 @@ describe('osrm', function() {


     // @TODO not safe yet to test: https://github.com/DennisOSRM/Project-OSRM/issues/693
-    it.skip('should throw if ini references corrupt files', function(done) {
+    it('should throw if ini references corrupt files', function(done) {
         assert.throws(function() { new osrm.Engine("./test/data/references-corrupt-files.ini"); },
         /hsgr not found/);
         done();

npm install doesn't copy lib/binding/ folder

Is there some way to install my locally-built copy of node-osrm as a dependency to my own project, and also get the lib/binding/ folder included?

I use the following to build node-osrm and install it into my local, global npm repo:

$ cd node-osrm-master
$ npm install -g --build-from-source

Later when I try to install node-osrm as a dependency for my own project, it fails to copy the binding folder:

$ cd my_project
$ npm install osrm
...
$ ls ./node_modules/osrm/lib/
index.js  osrm.js

This means that I have to manually copy the binding folder from my global repo:

$ cp -r `npm root -g`/lib/node_modules/osrm/lib/binding ./node_modules/osrm/lib/

Build Issue

When I build now, I get this error:

node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute 'node-gyp rebuild --fallback-to-build' (1)
node-pre-gyp ERR! stack at ChildProcess.module.exports.run_gyp (/home/mobius/node-osrm-master-3/node_modules/node-pre-gyp/lib/util/compile.js:34:29)
node-pre-gyp ERR! stack at ChildProcess.EventEmitter.emit (events.js:99:17)
node-pre-gyp ERR! stack at maybeClose (child_process.js:640:16)
node-pre-gyp ERR! stack at Process._handle.onexit (child_process.js:682:5)
node-pre-gyp ERR! System Linux 2.6.32-358.el6.x86_64
node-pre-gyp ERR! command "node" "/home/mobius/node-osrm-master-3/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /home/mobius/node-osrm-master-3
node-pre-gyp ERR! node -v v0.8.23
node-pre-gyp ERR! node-pre-gyp -v v0.4.2
node-pre-gyp ERR! not ok
npm ERR! [email protected] install: node-pre-gyp install --fallback-to-build
npm ERR! sh "-c" "node-pre-gyp install --fallback-to-build" failed with 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the osrm package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-pre-gyp install --fallback-to-build
npm ERR! You can get their info via:
npm ERR! npm owner ls osrm
npm ERR! There is likely additional logging output above.

npm ERR! System Linux 2.6.32-358.el6.x86_64
npm ERR! command "/opt/node-js/bin/node" "/opt/node-js/bin/npm" "install" "--build-from-source"
npm ERR! cwd /home/mobius/node-osrm-master-3
npm ERR! node -v v0.8.23
npm ERR! npm -v 1.2.18
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /home/mobius/node-osrm-master-3/npm-debug.log
npm ERR! not ok code 0

Streamline method of building against statically linked libOSRM

Currently the Travis.ci build has a static_build target that pulls down OSRM, builds and links it statically to the node-osrm binary for both node v0.8.x and v0.10.x.

There are two ways we can improve this:

  • Avoid actually compiling OSRM and instead just pull down a pre-built SDK with all headers and pre-built libs ready to build against. This will speed up builds immensely.
  • Open up a switch to make it easy to decide what git hash of OSRM to build against (likely best to have the SDK built by the OSRM core .travis and posted publicly to make it easy to switch between versions if needed)

Until the above is done: the OSRM version used in the node-osrm binaries is determined by the hash used in the source build which is done by mapnik-packaging scripts here.

Plug into Shared Memory

Node-osrm should use the shared memory facility of OSRM. From my point of view it is fair to also deprecate the ini files in the node bindings.

@springmeyer how much work do you think this is?

Make fails

$ make 
rm -f lib/_osrm.node
rm -f *.osrm*
`npm explore npm -g -- pwd`/bin/node-gyp-bin/node-gyp configure
npm ERR! Error: It doesn't look like npm is installed.
npm ERR!     at /usr/local/Cellar/node/0.10.22/lib/node_modules/npm/lib/explore.js:23:43
npm ERR!     at Object.oncomplete (fs.js:107:15)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <[email protected]>

npm ERR! System Darwin 13.0.0
npm ERR! command "/usr/local/Cellar/node/0.10.22/bin/node" "/usr/local/bin/npm" "explore" "npm" "-g" "--" "pwd"
npm ERR! cwd /Users/dennisluxen/Coding/node-osrm
npm ERR! node -v v0.10.22
npm ERR! npm -v 1.3.14
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /Users/dennisluxen/Coding/node-osrm/npm-debug.log
npm ERR! not ok code 0
/bin/sh: /bin/node-gyp-bin/node-gyp: No such file or directory
make: *** [build] Error 127
$ which npm
/usr/local/bin/npm
$ npm -v
1.3.14

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.