It's nice that we can build graphs with the mapsforge-creator now as well. I just gave this a try and saw that I can't specify a path to the GraphHopper config, it is assumed that it is in the same directory as the gh.jar. Wouldn't it make sense to be able to separate the two?

As mentioned by Ludwig here:
There also needs to be something that refreshes the coastline files. They do change a bit and should really be in sync as much as possible. Before I run a batch on our server, I delete the input files, forcing our current script to get the coastlines again.

Allow using multiple threads to increase performance via configuration variable.

It would be useful if a machine-readable index of all the files on download.mapsforge.org (and therefore any proxy cache that third-parties run locally to avoid traffic) could be created, so that apps wanting to show a list of available areas could easily ingest such an index.

Currently, downloads are produced and end up at:
http://download.mapsforge.org/maps/v5/
However, this gives an Apache directory listing over multiple URL locations, which is non-trivial to parse.

I realise this might be more a question of deployment for the administrators of the server, as mapsforge-creator seems to run on a per-file basis, but perhaps there is scope for achieving the above by converting the repetitive scripts at
https://github.com/mapsforge/mapsforge-creator/tree/master/v4
towards an architecture which uses a wrapper script to read from a JSON file of the same locations and run each invocation that way. That would then mean an index file essentially already exists (assuming no conversion failures).

For the multilingual builds we should be adding the local language(s) for each region, for Italy Italian etc pp.

As mentioned by Ludwig here:
I also think that this should be more flexible in the sense that other sources should be allowed more easily. The main rationale for this is to allow others to hook in their extracts (or custom generated data sources). This would not only be for their benefits, but also for ours: the more the scripts get used by others, the more input we will get in terms of bug fixes, improvements (maybe wishful thinking).

Add support to create GraphHopper offline routing graphs.

We should add to the documentation that you have to install zip (e.g.apt-get install zip). This is not installed by default on every distro.

It would be great if we could specify a tag transform file before generating the map file.

Not sure if we have a best practice for mapsforge at this moment? But it would be great to add this as user option.

Something like:

if(path-to-tag-transform-file-given)
  run tag-transform

Recently the wget arguments changed and -nv was removed from the wget command. Would it be possible to add it again or to make it configurable. This spams a lot of unnecessary lines to logfiles.

shape2osm.py converter has a (user selectable) maximum number of objects in a single .osm file.
But instead of producing multiple .osm files after exceeding that limit, it overwrites the old one.

Note: after fixing this, we need to update the map-creator script to properly merge all converted .osm files.

Update or download automatically the data.

I think it is better to write the output file to the $DATA and not to the $MAPS area.

The $MAPS area is visible to external users and might be picked up be mirrors, where a half-finished file could stay for until the next sync.

The poly2bb Perl script can produce invalid coordinates because of the used buffer.

As mentioned by Ludwig here:
The size check before overwriting was also useful.

The idea is to check the size of the newly generated map compared to the old map. If the new file is smaller by e.g. 10% we would thrown an error.

The $DATA area should also be segmented, like the $MAPS area, by version (v3 etc).

This ensures that, if v3 and v4 scripts are run in parallel, they do not overwrite each other.

Add support for Mapsforges maps v5.

If you look at openstreetmapdata.com you can see that the recommend to use: osmdata.openstreetmap.de

As mentioned by Ludwig here:
Something else we put in place was to check the date of files when to regenerate something. This has been useful to rerun the whole script if errors occurred to speed the regeneration of files up that were corrupted.

So it would be nice if we could make the time range configurable so that the script can skip the map if the source map is younger than some time-range.

Following scenario happens: script downloads pbf for are from geofabrik, this download somehow fails (truncated for whatever reason). When the script is run again, it determines that the pbf file is not old and uses it to generate a map.

I think it would be safe to always download the pbf file from geofabrik, even if it is pretty new. That ensures that the md5 check is carried out. Obviously, this will lead to more downloads if the map creation process fails, but if the map itself is newer than 30 days, the download will not happen.

This also results in the newest data being used.

Currently the script does not fail, if any of the commands fail. If anything fails, we should exit the script with an error code (e.g. 1), so we should check every execution and call exit 1 if it fails. This makes it easier to check if the overall script worked or not

Geofabrik offers https for downloading files. Probably download speed will be lower with https? Not sure if we should do this or not.