UnicodeDecodeError in graph_from_xml on Windows Installations about osmnx HOT 4 CLOSED

What happens when you run:

import sys; print(sys.getfilesystemencoding())
import locale; print(locale.getpreferredencoding())

Is your OS set to use utf-8 for both of those encodings?

I thought Python on all platforms was defaulting to utf-8 since like 3.6 (see PEP 528 and 529)... but maybe it's somehow still opt-in on Windows (see PEP 540)?

from osmnx.

Here's another test for you:

from pathlib import Path
with Path("kaliningrad.xml").open() as f:
    print(f)

The file is encoded as utf-8 and the XML declaration states that it is utf-8... does your system open it with utf-8 encoding or does it print some other encoding instead?

from osmnx.

Thanks Geoff,

Interesting, those tests are quite revealing:

import sys; print(f"File System Encoding: {sys.getfilesystemencoding()}")
import locale; print(f"Preferred encoding: {locale.getpreferredencoding()}")
from pathlib import Path
with Path("./data/kaliningrad/kaliningrad.xml").open() as f:
    print(f)

...gives...

File System Encoding: utf-8
Preferred encoding: cp1252
<_io.TextIOWrapper name='data\\kaliningrad\\kaliningrad.xml' mode='r' encoding='cp1252'>

which is clerly the source of our problem!

This error came up as part of a practical for a course at the University of Manchester - we have been running it for years and this is the first time that this has happened, I am not sure what has changed to cause it (we are seeing it across all Windows machines that we have tried, not just a particular build etc.). I guess that this sort of thing might be part of the motivation behind PEP686, which sems to suggest that Python will override the locale settings to UTF-8 by default from version 3.15.

Nevertheless, adding an encoding argument to the function as you have seems like an ideal solution - thanks for dealing with it so quickly.

from osmnx.

Great, I'll merge that PR and dream of the day when Python 3.15 arrives 🥳

from osmnx.