project-osrm / node-osrm Goto Github PK
View Code? Open in Web Editor NEWDEPRECATED Part of osrm-backend since 5.7. NodeJS bindings for OSRM
License: BSD 2-Clause "Simplified" License
DEPRECATED Part of osrm-backend since 5.7. NodeJS bindings for OSRM
License: BSD 2-Clause "Simplified" License
Currently OSRM just prints to std::cerr and calls exit if data, like the .hsgr
, is not found as referenced in the server.ini (in typedefs.h -> ERR
). This makes it impossible for the node module to recover from an error like this.
Ideally OSRM could throw an exception in QueryObjectsStorage.cpp:36
.
The current batch of binaries are hosted under my personal aws account, but these should be migrated to a Mapbox controlled account so other committers on this project can have admin access to the bucket.
To do this I will:
https://node-osrm.s3.amazonaws.com
to https://mapbox-node-binary.s3.amazonaws.com/osrm/
(note the osrm
prefix)package.json
binary/host
value at mapbox-node-binary
- 495eff5node-osrm
bucket.> [email protected] test /home/ubuntu/node-OSRM
> mocha -R spec
module.js:356
Module._extensions[extension](this, filename);
^
Error: /home/ubuntu/node-OSRM/lib/_osrm.node: undefined symbol: _ZNK5boost15program_options29value_semantic_codecvt_helperIcE5parseERNS_3anyERKSt6vectorISsSaISsEEb
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
Boost libs sometimes have -mt
at the end to signify they are the multithreaded versions. After Project-OSRM/osrm-backend@4a3db7e node-osrm now needs to link all dependent libraries so we need to know their exact name.
So, builds against homebrew based boost will start failing at 1c51388 until we come up with a solution.
Possibilities are:
I think pkg-config
is ideal, just need to start testing this post Project-OSRM/osrm-backend@8383222
Create network data on the fly before tests are run using a small osm extract
@springmeyer and I discussed this -- they are not used and inflate the Travis build times.
Work started in b8457fc, just need to actually post binaries for both OS X and Linux. This ticket is to track when this work is done.
Ideally we could store a very small set of data for the tests to make it dead easy to test node-osrm on different platforms quickly (without having to download and wait to process data with osrm-extract
and osrm-prepare
). Is the data portable across architectures or will we need to generate copies for both 32 bit and 64 bit systems? Is the data fairly portable across node-osrm versions?
More JSy API:
Options
or Query
objects. JS idiom is just to pass options hashes directly.new osrm.Engine({ini: <path>, sharedMemory: <boolean>})
(better would be to eliminate the assumption of having a file on disk with options and pass them directly -- don't know enough about OSRM to gauge if that's feasible)Engine#route(options, callback)
Engine#locate(latLon, callback)
Engine#nearest(latLon, callback)
If I have the shared memory flag on, and have the .ini file pointing to .osrm files, does it still try to load those files? I tried to have the location of the files pointed to in the .ini files point to invalid locations, but it just throws an error that the files don't exist.
node-osrm
latest now builds and runs against the latest develop
code from OSRM with shared memory support. But current defaults are in place that do not turn on shared memory usage. This ticket stands to track actually testing it from node.js
Looks like we need to catch up with Project-OSRM/osrm-backend@cabaad4#diff-ce251cb4263f3c33dfb996c10931b6f2
In file included from ../src/node_osrm.cpp:15:
../src/engine.hpp:106:55: error: no member named 'c_str' in 'std::vector<std::basic_string<char>, std::allocator<std::basic_string<char> > >'
return scope.Close(String::New(osrm_reply.content.c_str()));
~~~~~~~~~~~~~~~~~~ ^
../src/engine.hpp:165:25: error: no viable overloaded '='
closure->result = osrm_reply.content;
~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~
/usr/include/c++/4.2.1/bits/basic_string.h:499:7: note: candidate function not viable: no known conversion from 'std::vector<std::string>' to
'const std::basic_string<char>' for 1st argument
operator=(const basic_string& __str)
Installation is currently non-standard and currently depends on a clone of this repo being next to a built Project-OSRM directory. What would need to happen in order to it possible to install by simply running npm install node-osrm
?
As a sanity check, if Travis is building a tag for release, it should error out early on if the tag does not match the version in package.json.
This will catch an error I made on my first try, when I forgot to include the package.json commit in the v0.2.5 tag.
node-osrm should be able to report the git hash of OSRM it is built against.
Possible tests:
Port
, IP
, and Threads
values in server.ini
.edgesData
points to a corrupt file.The existing shared memory test never actually tested shared memory, and is now disabled. It will requrie some pre-test setup to get working.
Hitting this when using shared memory: https://gist.github.com/springmeyer/daa7705b5e81b8af95bb
Hoping that the problem is some mixing of boost headers/libs across versions: Project-OSRM/osrm-backend@9e75595#commitcomment-4529676
Ideally, for node-osrm, there could be a way to disable this OSRM output so that meaningful logging could be controlled by the application calling into node-osrm
It should be possible to run 'nearest' and 'locate' queries in addition to 'viaroute'. Something like this for nearest
:
var query = new osrm.Query({
service: "nearest",
coordinates: [[52.4224,13.333086]]
});
var sync_result = engine.run(query);
or this for locate
:
query = new osrm.Query({
service: "locate",
coordinates: [[52.4224,13.333086]]
});
sync_result = engine.run(query);
When I load node-osrm with data files that correspond to the entire planet, I get this error:
var engine = new osrm.Engine(opts);
^
Error: std::bad_alloc
at Object. (/home/ubuntu/routing_osrm_nodejs/server.js:10:14)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.runMain (module.js:492:10)
at process.startup.processNextTick.process._tickCallback (node.js:245:9)
Currently the paths appear to be interpreted as relative to the cwd of the calling process, which feels not ideal. The paths should be relative to the server.ini location since that is the file that defines the paths.
OSRM builds with clang and libc++ on Mavericks. This needs to be reflected in bindings.gyp
. Otherwise, we get undefined references.
Berlin extract was helpful for early testing, but since OSRM core has extensive tests, I think we can scale back the testing extract size in node-osrm. A smaller size will help devs get set up faster and will ensure we don't hit travis timeouts (saw these today).
Goal:
node-osrm.Engine.run()
to ensure that the async approach to running the route query holds up (does not hang or crash) and then produce initial profiling results to ensure no bottlenecks exist in the bindings themselves.Tasks:
Is it possible to have node-osrm use the same shared memory with an osrm-routed instance? This would also be extremely helpful!
So far, libOSRM returns a JSON formatted return string. Would it make more sense in terms of efficiency, and the way NodeJS would like to have things, to return an object that has a certain interface that NodeJS can use.
The idea is to expose some object with a well-known interface/structure that can be used in any subsequent transformation of the result. I'd like to save some time on (costly) string generation/duplication.
With the move to tbb of upstream OSRM we should discuss dropping static linking for this one node plugin.
Does the IP address and Port in the .ini file point to the address and port of the OSRM server that node-osrm will be connecting to? Does this indicate then that node-osrm is sending and receiving to the engine through a socket?
Travis v0.8.x tests are all passing but seeing and Illegal instruction
failure with v0.10.x: https://travis-ci.org/DennisOSRM/node-OSRM/jobs/15018896#L740.
If I use node-osrm as my bindings to OSRM, does this allow me to have multiple simultaneous connections to OSRM?
Linking fails on OS X with home-brew installed boost. It comes with the -mt
suffix. Leads to:
ld: library not found for -lboost_thread
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Release/osrm.node] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/local/Cellar/node/0.10.22/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:267:23)
gyp ERR! stack at ChildProcess.EventEmitter.emit (events.js:98:17)
gyp ERR! stack at Process.ChildProcess._handle.onexit (child_process.js:789:12)
gyp ERR! System Darwin 13.0.0
gyp ERR! command "node" "/usr/local/Cellar/node/0.10.22/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/dennisluxen/Coding/node-osrm
gyp ERR! node -v v0.10.22
gyp ERR! node-gyp -v v0.11.0
gyp ERR! not ok
Is it possible to get the nearest road point through node-osrm? I've only seen routing requests calculated. I was hoping to have the APIs that are available through the OSRM http API available through node-osrm.
We could get a slightly simpler configuration if we were able to pass a hash into the Options constructor in addition to an ini file path.
new osrm.Options({
threads: 8,
hsgrData: 'path/to/data'
...
}, true);
parsing of server.ini and command line options will change with the shared_memory branch of OSRM. This needs to be implemented/pulled in here as well.
@DennisOSRM - I added travis config in 454fe6e.
Could you enable when you have a chance?
It appears that g_GIT_DESCRIPTION
is not available in libOSRM.dylib
and therefore ./Util/GitDescription.cpp
may need to be added to libOSRM
. The result is that when running node-osrm
on OS X (at least) we hit:
Error: dlopen(/Users/dane/projects/Project-OSRM/node-osrm/lib/_osrm.node, 1): Symbol not found: _g_GIT_DESCRIPTION
Referenced from: /Users/dane/projects/Project-OSRM/node-osrm/lib/_osrm.node
Expected in: dynamic lookup
When I try to build or install node-osrm on RedHat, I get the following error:
module.js:485
process.dlopen(filename, module.exports);
^
Error: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/mobius/node-osrm-master-3/lib/binding/osrm.node)
at Object.Module._extensions..node (module.js:485:11)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:362:17)
at require (module.js:378:17)
at Object.<anonymous> (/home/mobius/node-osrm-master-3/lib/osrm.js:6:15)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
What can I do to resolve this issue?
I'm using OSRM-develop (ce784e0491) and doing this to load the shared memory:
$ osrm-datastore /absolute/path/to/data.osrm
I can see the shared memory with ipcs -m
. osrm-routed
is able to access this shared memory.
Then I'm using node-osrm like this, where data.ini
contains same absolute paths to data:
var opts = new osrm.Options("./data.ini", true /*used shared memory*/);
var engine = new osrm.Engine(opts);
node-osrm works, but it's not using the shared memory. I can manually free the shared memory segment with ipcrm -m {shmid}
and my node app still does routing, for example.
Any ideas? Is it because I'm not using the latest version of OSRM, perhaps?
Plan to look into this such that npm publish
automatically happens on git push --tags
but after new binaries are published.
On the C++ side an OSRM engine has non-memory resources that are deallocated when it's destructed right? I think we need to expose that explicitly on the JS side so it doesn't leak if JS garbage isn't immediately collected.
What I'm seeing running tests with shared memory that create an Engine per test case is that after a few tests, Engine creation starts failing with "Error: Too many open files".
Need to enable the two skipped/pending (blue tests) once upstream issues are fixed:
@DennisOSRM - can you let me know if those issues are ones you'd like to jump on or would prefer me to take a look at?
Once solved these tests can be enabled by doing:
diff --git a/test/osrm.test.js b/test/osrm.test.js
index e03ba19..1664415 100644
--- a/test/osrm.test.js
+++ b/test/osrm.test.js
@@ -16,7 +16,7 @@ describe('osrm', function() {
});
// @TODO not safe yet to test: https://github.com/DennisOSRM/Project-OSRM/issues/692
- it.skip('should throw if ini file is blank', function(done) {
+ it('should throw if ini file is blank', function(done) {
assert.throws(function() { new osrm.Engine("./test/data/bogus.ini"); },
/@TODO/);
done();
@@ -38,7 +38,7 @@ describe('osrm', function() {
// @TODO not safe yet to test: https://github.com/DennisOSRM/Project-OSRM/issues/693
- it.skip('should throw if ini references corrupt files', function(done) {
+ it('should throw if ini references corrupt files', function(done) {
assert.throws(function() { new osrm.Engine("./test/data/references-corrupt-files.ini"); },
/hsgr not found/);
done();
https://github.com/DennisOSRM/node-OSRM/blob/master/src/query.hpp#L94-L95 seems to only add origin and destination but not any intermediate stops.
Is there some way to install my locally-built copy of node-osrm
as a dependency to my own project, and also get the lib/binding/
folder included?
I use the following to build node-osrm and install it into my local, global npm repo:
$ cd node-osrm-master
$ npm install -g --build-from-source
Later when I try to install node-osrm
as a dependency for my own project, it fails to copy the binding
folder:
$ cd my_project
$ npm install osrm
...
$ ls ./node_modules/osrm/lib/
index.js osrm.js
This means that I have to manually copy the binding
folder from my global repo:
$ cp -r `npm root -g`/lib/node_modules/osrm/lib/binding ./node_modules/osrm/lib/
When I build now, I get this error:
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute 'node-gyp rebuild --fallback-to-build' (1)
node-pre-gyp ERR! stack at ChildProcess.module.exports.run_gyp (/home/mobius/node-osrm-master-3/node_modules/node-pre-gyp/lib/util/compile.js:34:29)
node-pre-gyp ERR! stack at ChildProcess.EventEmitter.emit (events.js:99:17)
node-pre-gyp ERR! stack at maybeClose (child_process.js:640:16)
node-pre-gyp ERR! stack at Process._handle.onexit (child_process.js:682:5)
node-pre-gyp ERR! System Linux 2.6.32-358.el6.x86_64
node-pre-gyp ERR! command "node" "/home/mobius/node-osrm-master-3/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /home/mobius/node-osrm-master-3
node-pre-gyp ERR! node -v v0.8.23
node-pre-gyp ERR! node-pre-gyp -v v0.4.2
node-pre-gyp ERR! not ok
npm ERR! [email protected] install: node-pre-gyp install --fallback-to-build
npm ERR! sh "-c" "node-pre-gyp install --fallback-to-build"
failed with 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the osrm package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-pre-gyp install --fallback-to-build
npm ERR! You can get their info via:
npm ERR! npm owner ls osrm
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 2.6.32-358.el6.x86_64
npm ERR! command "/opt/node-js/bin/node" "/opt/node-js/bin/npm" "install" "--build-from-source"
npm ERR! cwd /home/mobius/node-osrm-master-3
npm ERR! node -v v0.8.23
npm ERR! npm -v 1.2.18
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /home/mobius/node-osrm-master-3/npm-debug.log
npm ERR! not ok code 0
Not sure why, but I'm seeing fatal: depth 0 is not a positive number
in the logs which looks quite odd: https://travis-ci.org/DennisOSRM/node-osrm/jobs/19606691
Currently the Travis.ci build has a static_build
target that pulls down OSRM, builds and links it statically to the node-osrm binary for both node v0.8.x and v0.10.x.
There are two ways we can improve this:
Until the above is done: the OSRM version used in the node-osrm binaries is determined by the hash used in the source build which is done by mapnik-packaging scripts here.
Node-osrm should use the shared memory facility of OSRM. From my point of view it is fair to also deprecate the ini files in the node bindings.
@springmeyer how much work do you think this is?
Naming convention in npm seems to be all lowercase.
$ make
rm -f lib/_osrm.node
rm -f *.osrm*
`npm explore npm -g -- pwd`/bin/node-gyp-bin/node-gyp configure
npm ERR! Error: It doesn't look like npm is installed.
npm ERR! at /usr/local/Cellar/node/0.10.22/lib/node_modules/npm/lib/explore.js:23:43
npm ERR! at Object.oncomplete (fs.js:107:15)
npm ERR! If you need help, you may report this log at:
npm ERR! <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR! <[email protected]>
npm ERR! System Darwin 13.0.0
npm ERR! command "/usr/local/Cellar/node/0.10.22/bin/node" "/usr/local/bin/npm" "explore" "npm" "-g" "--" "pwd"
npm ERR! cwd /Users/dennisluxen/Coding/node-osrm
npm ERR! node -v v0.10.22
npm ERR! npm -v 1.3.14
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /Users/dennisluxen/Coding/node-osrm/npm-debug.log
npm ERR! not ok code 0
/bin/sh: /bin/node-gyp-bin/node-gyp: No such file or directory
make: *** [build] Error 127
$ which npm
/usr/local/bin/npm
$ npm -v
1.3.14
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.