Conversion HERE Shapefile > ogr2osm > bz2 > Navit .bin using Maptool about ogr2osm HOT 15 CLOSED

Here is a screenshot of all the Maptool options:

from ogr2osm.

Note that maptool can only process OSM files without wrapped or empty lines.

This sounds like a badly broken XML parser. You could try passing the ogr2osm XML through xmllint to reformat it.

from ogr2osm.

Hi Paul,

I appreciate your input. The process of running the .osm through xmllint completed without issue, however when I zipped it and ran it through maptool the result was exactly as described above :( I know Maptool isnt your department, but it is in the Ubuntu repository (sudo apt-get install maptool). Have i come to the end of the road (pun intended :)) or can you suggest any other switches in xmllint or maptool that I could try?

Or something else entirely!

Thanks again.

Regards,

Peter

from ogr2osm.

try adding --positive-id --add-version --add-timestamp. That should produce an XML file the same as the API would, and if that doesn't fix it, try reducing it to a smaller testcase and reporting it to maptool

from ogr2osm.

Apologies, add these in which step? ogr2osm or xmllint?

from ogr2osm.

Ignore that question, it's ogr2osm :) Trying it now. It'll take a while, the source shapefile is 303MB.

Peter

from ogr2osm.

Incidentally, prior to running ogr2osm I included 2 related .csv files which had previously been outside the main shapefile directory. I placed them in among the .prj, .dbf etc. files. Was that a mistake?

Peter

from ogr2osm.

Shouldn't matter.

I recommend cutting out a small part of the shapefile with ogr2ogr and working with that to test.

from ogr2osm.

Hi Paul,

You may wish to read the last paragraph below first!

No joy I'm afraid, adding those options only doubled the resulting .osm file size but the maptool result was the same, and when I tried running the new .osm file through xmllint the pc froze after 10 minutes. Tried again after a reboot but exactly the same error and freezing.

Going back to basics I did a vanilla install of Lubuntu (16.04.1 64bit 8GB+8GB swap) and did the following testing using freshly downloaded Isle of Man files from Geofabrik:

1: isle-of-man-latest.osm.bz2 (3.2MB) converted to .bin file using Maptool: SUCCESS, loads into Navit

2: isle-of-man-latest.osm.pbf (2.2MB) converted to .bin file using Maptool: SUCCESS, loads into Navit

3: isle-of-man-latest-free.shp.zip (8.5MB after unzipping) converted to .osm using ogr2osm then resulting .osm file converted to .bin file using Maptool: FAIL (error as above, " FATAL: wrong line in input data (does not start with '<'): > "

I tried step 3 using all 3 possible methods of converting .osm files listed on
http://wiki.navit-project.org/index.php/OpenStreetMap
namely:
a: cat my_OSM_map.osm | maptool my_Navit_map.bin
b: maptool -i my_OSM_map.osm my_Navit_map.bin
c, after zipping to bz2: bzcat my_OSM_map.osm.bz2 | maptool my_Navit_map.bin
all with the identical error result above.

In an effort to isolate the problem, I exported a 2.5MB .osm of my home town (Bangor, Northern Ireland) directly from https://www.openstreetmap.org/export. Amazingly I was able to convert this to a Navit friendly .bin file using both maptool methods a and b above, and they both worked in Navit!! Therefore I have proved that maptool works with OSM XML in bz2 form and OSM Protobuf data in .pbf form, both from Geofabrik, and now also OSM XML in native .osm form directly from OpenStreetMap.

However for some reason maptool doesn't like your ogr2osm converted shapefiles, either my HERE shapefile or an unzipped shapefile from Geofabrik. The conversion runs without errors, and yet maptool complains with the error as listed above, namely:

peter@peter-Lubuntu-64bit:~$ maptool -i /home/peter/ogr2osm/isle-of-man-latest-free.osm isle-of-man-latest.bin
PROGRESS: Phase 1: reading input data 0:00 0 MB
PROGRESS1: Processed 0 nodes (0 out) 0 ways 0 relations 0 tiles 0:00 0 MB
INFO: Nodes out of sequence (new 4294967293 vs old 4294967294), adding hash
INFO: Ways out of sequence (new -35 vs old -1), adding hash
FATAL: wrong line in input data (does not start with '<'): >

This does not look like a valid OSM file.
Note that maptool can only process OSM files without wrapped or empty lines.
peter@peter-Lubuntu-64bit:~$

If you think you know where the problem may lie I am happy to consider paying you for your time to fix it if you have a Paypal account. I am extemely grateful for your willingness to engage thus far. I feel ogr2osm is my best if not my only hope for what I'm trying to do.

Best regards,

Peter

from ogr2osm.

By the way for completeness, I also created a new Isle of Man .osm from my unzipped Geofabrik IoM shapefile using ogr2osm this time including --positive-id --add-version --add-timestamp. No change in maptool. Ran this new .osm through xmllint. No change in maptool. There's literally no other permutation I can try! :) Hope you can help. Peter

from ogr2osm.

Can you try opening the ogr2osm XML in a text editor and trimming it down to a test of a few dozen lines?

from ogr2osm.

trimming it down to a test of a few dozen lines?

You'll need to make sure it's valid XML, which means keeping the closing stuff.

from ogr2osm.

For taking on paid work, I am available. I suspect the problem is ultimately with maptool, but replicating exactly what maptool wants might act as a work-around. I haven't worked with the ogr2osm code for a couple years now.

from ogr2osm.

Hi Paul,

Using a sample Isle of Man shapefile from Geofabrik I found that if I removed the "way id" components of your resulting .osm and left only node and relation id's the resulting file would process successfully in maptool and load into Navit. Obviously this leaves a fairly useless map, but what does it tell us about the error?

As I said I was able to process (in maptool) a .osm of my hometown of Bangor from https://www.openstreetmap.org/export and view it in Navit.

The latest sourcecode for maptool can be found here:

https://github.com/navit-gps/navit/releases/tag/v0.5.0

You'll find it in /navit-0.5.0/navit/maptool

If you identify work that needs to be done on either maptool or ogr2osm we can discuss it if you are able to give me a quote.

Best regards,

Peter

from ogr2osm.

Hi Paul,

The .osm file created by ogr2osm included way id lines that were over 3000 columns wide. For some reason maptool was unable to process these. Your advice to run the .osm through xmllint was correct, I just wasn't using xmllint correctly. I wasn't giving xmllint a proper output and also I needed the --pretty 1 argument to actually fix the problem.

Thanks for pointing me in the right direction, sorry if you've spent time thinking about this since your last message.

Best regards,

Peter

from ogr2osm.