Comments (7)
Hey @Binternet,
That's unrelated to the openaddresses download, fortunately. It will be fixed very soon with pelias/whosonfirst#324
from dockerfiles.
Not sure I understand since the error occurs in openaddresses
and not whosonfirst
from dockerfiles.
The downloads for all data sources are performed simultaneously, so the error/info messages are interleaved. It's definitely confusing, but faster :)
This issue is well known because it's due to the Mapzen Shutdown and was also reported in #50
If you have a minute, try the workaround mentioned in #50 (comment). A report of how well it works would be much appreciated.
from dockerfiles.
Just tried it, i'm still getting this error:
> [email protected] download /code/pelias/openstreetmap
> node util/download_data.js
[download] https://dist.whosonfirst.org/sqlite/whosonfirst-data-latest.db.bz2
info: [download] Attempting to download selected data files: http://s3.amazonaws.com/data.openaddresses.io/openaddr-collected-global.zip
info: [download] Downloading http://s3.amazonaws.com/data.openaddresses.io/openaddr-collected-global.zip
info: [download] Downloading sources: http://download.bbbike.org/osm/planet/planet-latest.osm.pbf
error: [download] Failed to download data Error: Command failed: unzip -o -qq -d /data/openaddresses /code/pelias/openaddresses/tmp-172zUU7skhrllF.zip
[/code/pelias/openaddresses/tmp-172zUU7skhrllF.zip]
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of /code/pelias/openaddresses/tmp-172zUU7skhrllF.zip or
/code/pelias/openaddresses/tmp-172zUU7skhrllF.zip.zip, and cannot find /code/pelias/openaddresses/tmp-172zUU7skhrllF.zip.ZIP, period.
at ChildProcess.exithandler (child_process.js:199:12)
at emitTwo (events.js:106:13)
at ChildProcess.emit (events.js:191:7)
at maybeClose (internal/child_process.js:920:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:230:5)
npm ERR! Linux 4.4.0-1049-aws
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "download"
npm ERR! node v6.13.0
npm ERR! npm v3.10.10
npm ERR! code ELIFECYCLE
npm ERR! [email protected] download: `node utils/download_data.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] download script 'node utils/download_data.js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the pelias-openaddresses package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node utils/download_data.js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs pelias-openaddresses
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls pelias-openaddresses
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR! /code/pelias/openaddresses/npm-debug.log
from dockerfiles.
This looks like an out disk space error. The OpenAddresses full planet zip is 10GB, Who's on First requires 15GB, and it looks like you're trying to download a full OSM planet file (39GB).
Even assuming you have the disk space for all that, this import is not likely to succeed with the Dockerfiles setup.
If you haven't tried running Pelias with a smaller import yet, I'd suggest you start there. After that, take a look at our full install guide if you haven't yet. It's a little out of date, but pay special attention to the "Considerations for full-planet builds" section if you really do want to import the whole planet. Let me know if you have questions with the full planet build :)
from dockerfiles.
Well I am trying to run a full-planet on an AWS i3.4xlarge
instance, i've managed to run it with the Dockerfiles setup for the default Portland data, but i'll try again with the full install guide. will keep you posted!
from dockerfiles.
Well then, an i3.4xlarge
is a little different than say a personal laptop :)
Keep in mind it will take several days and you'll probably want a dedicated Elasticsearch cluster outside of the one provided by the Dockerfiles. The importers should work fine for a full planet import, but Elasticsearch will need to be treated with more care than this repo can likely provide. It's meant just for getting up and running quickly.
from dockerfiles.
Related Issues (20)
- Data Load / WOF issues HOT 8
- Plans to allow for CI builds, local development HOT 4
- Pin version of libpostal in libpostal_baseimage? HOT 1
- Interpolation and Polylines issues HOT 8
- libpostal error on build HOT 4
- The process hangs after downloads with message "All Done!" HOT 1
- Early Errors ... build hangs on ES startup HOT 2
- Docker npm error HOT 4
- Missing WOF data? HOT 5
- Interpolation maxes out one core & the other processes & cores are idle HOT 7
- Interpolation not working as of November 7, 2017 HOT 7
- Looking for paid installation service HOT 1
- mem_limit setting in docker-compose.yml was causing imports to fail HOT 2
- whosonfirst does not start with default pelias.json configuration HOT 6
- Wall of errors during successful setup, not sure what is wrong HOT 5
- How to configure pelias.json if multiple regions/cities need to be imported? HOT 2
- error when executing "docker-compose run --rm whosonfirst npm run download" error:14077410:SSL routines:SSL23_GET_SERVER_HELLO HOT 11
- The link to sign up and generate a Mapzen API key doesn't work HOT 1
- Docker volume configuration HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dockerfiles.