Giter VIP home page Giter VIP logo

Comments (8)

simon04 avatar simon04 commented on July 23, 2024

The following patch fixes the issue for the above shapefile as well as for my personal project:

a/ogr2osm.py b/ogr2osm.py
index 41541f3..d3d081d 100755
--- a/ogr2osm.py
+++ b/ogr2osm.py
@@ -387,7 +387,7 @@ def getLayerFields(layer):
 def getFeatureTags(ogrfeature, fieldNames):
     tags = {}
     for i in range(len(fieldNames)):
-        tags[fieldNames[i]] = ogrfeature.GetFieldAsString(i)
+        tags[fieldNames[i].decode('utf-8')] = ogrfeature.GetFieldAsString(i).decode('utf-8')
     return translations.filterTags(tags)

 def parseLayer(layer):

from ogr2osm.

simon04 avatar simon04 commented on July 23, 2024

Your commit 8eb57cf has the disadvantage that writing translation files is more tricky. Consider the example

# -*- coding: utf-8 -*-
def translateAttributes(attrs):
    return {'name': u'ß' + attrs['NAME']}
  • In 8eb57cf, this gives rise to UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)
  • … whereas the example works in my solution

from ogr2osm.

pnorman avatar pnorman commented on July 23, 2024

What line is it raising that error on?

from ogr2osm.

simon04 avatar simon04 commented on July 23, 2024
Traceback (most recent call last):
  File "/home/simon/src/ogr2osm/ogr2osm.py", line 639, in <module>
    parseData(data)
  File "/home/simon/src/ogr2osm/ogr2osm.py", line 345, in parseData
    parseLayer(translations.filterLayer(layer))
  File "/home/simon/src/ogr2osm/ogr2osm.py", line 402, in parseLayer
    parseFeature(translations.filterFeature(ogrfeature, fieldNames, reproject), fieldNames, reproject)
  File "/home/simon/src/ogr2osm/ogr2osm.py", line 419, in parseFeature
    feature.tags = getFeatureTags(ogrfeature, fieldNames)
  File "/home/simon/src/ogr2osm/ogr2osm.py", line 392, in getFeatureTags
    return translations.filterTags(tags)
  … Translation file, line where the concatenation takes place
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2: ordinal not in range(128)

from ogr2osm.

pnorman avatar pnorman commented on July 23, 2024

I'm going to have to check that change against shift_jis shapefiles and existing translations (e.g. https://github.com/nyampire/ogr2osm/blob/master/translations/N03-admin_border.py)

from ogr2osm.

naoliv avatar naoliv commented on July 23, 2024

With this shapefile http://people.debian.org/~naoliv/misc/monster-shape.tar.bz2 and latest code I see another related problem:

$ ./ogr2osm.py -v ~/m/35DSE250GC_SIR.shp 
running with lxml.etree
Preparing to convert file '/home/naoliv/m/35DSE250GC_SIR.shp' to '/tmp/d/ogr2osm/35DSE250GC_SIR.osm'.
Will try to detect projection from source metadata, or fall back to EPSG:4326
Using default translations
Using default filterLayer
Using default filterFeature
Using default filterTags
Using default filterFeaturePost
Using default preOutputTransform
Parsing data
Detected projection metadata:
GEOGCS["SIRGAS 2000",
    DATUM["SIRGAS2000",
        SPHEROID["Geodetic_Reference_System_of_1980",6378137,298.2572221009113]],
    PRIMEM["Greenwich",0],
    UNIT["Degree",0.017453292519943295]]
Merging points
Making list
Checking list
Outputting XML
Traceback (most recent call last):
  File "./ogr2osm.py", line 642, in <module>
    output()
  File "./ogr2osm.py", line 610, in output
    tag = etree.Element('tag', {'k':unicode(key,'utf-8'), 'v':unicode(value, 'utf-8')})
UnicodeDecodeError: 'utf8' codec can't decode byte 0xcd in position 4: unexpected end of data

from ogr2osm.

simon04 avatar simon04 commented on July 23, 2024

Note that also QGIS has encoding problems, i.e., the attribute table shows some strange symbols.

On the GDAL side, general Unicode support isn't available: http://www.mail-archive.com/[email protected]/msg01993.html

To come around this issue, the desired encoding could be given as commandline argument (default utf-8) …

from ogr2osm.

pnorman avatar pnorman commented on July 23, 2024

naoliv, for the second shapefile where there are still errors it should work if you use --encoding ISO8859-1

from ogr2osm.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.