Example data: (I edited out the lat long data for my privacy, in mine it has actual values)

{
  "locations" : [ {
    "timestampMs" : "1509317655000",
    "latitudeE7" : 000000000,
    "longitudeE7" : 00000000,
    "accuracy" : 47,
    "velocity" : 0,
    "heading" : 244,
    "altitude" : 139,
    "activity" : [ {
      "timestampMs" : "1509317648536",
      "activity" : [ {
        "type" : "STILL",
        "confidence" : 85
      }, {
        "type" : "IN_VEHICLE",
        "confidence" : 5
      }, {
        "type" : "ON_FOOT",
        "confidence" : 5
      }, {
        "type" : "WALKING",
        "confidence" : 5
      }, {
        "type" : "ON_BICYCLE",
        "confidence" : 3
      }, {
        "type" : "UNKNOWN",
        "confidence" : 3
      } ]
    },

A couple thousand more, then

{
      "timestampMs" : "1500967359397",
      "activity" : [ {
        "type" : "IN_VEHICLE",
        "confidence" : 41
      }, {
        "type" : "UNKNOWN",
        "confidence" : 35
      }, {
        "type" : "STILL",
        "confidence" : 22
      }, {
        "type" : "ON_BICYCLE",
        "confidence" : 2
      } ]
    } ]
  } ]
}

So, now, files have entries that don't have lat/long data. Quite strange, huh?
So I made a really simple fix:

        if args.format == "kml":
            f_out.write("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n")
            f_out.write("<kml xmlns=\"http://www.opengis.net/kml/2.2\">\n")
            f_out.write("  <Document>\n")
            f_out.write("    <name>Location History</name>\n")
            for item in items:

                if 'longitudeE7' in item: # here is the line i added

                    f_out.write("    <Placemark>\n")
                    # Order of these tags is important to make valid KML: TimeStamp, ExtendedData, then Point
                    f_out.write("      <TimeStamp><when>")
                    f_out.write(datetime.utcfromtimestamp(int(item["timestampMs"]) / 1000).strftime("%Y-%m-%dT%H:%M:%SZ"))
                    f_out.write("</when></TimeStamp>\n")
                    if "accuracy" in item or "speed" in item or "altitude" in item:
                        f_out.write("      <ExtendedData>\n")
                        if "accuracy" in item:
                            f_out.write("        <Data name=\"accuracy\">\n")
                            f_out.write("          <value>%d</value>\n" % item["accuracy"])
                            f_out.write("        </Data>\n")
                        if "speed" in item:
                            f_out.write("        <Data name=\"speed\">\n")
                            f_out.write("          <value>%d</value>\n" % item["speed"])
                            f_out.write("        </Data>\n")
                        if "altitude" in item:
                            f_out.write("        <Data name=\"altitude\">\n")
                            f_out.write("          <value>%d</value>\n" % item["altitude"])
                            f_out.write("        </Data>\n")
                        f_out.write("      </ExtendedData>\n")
                    f_out.write("      <Point><coordinates>%s,%s</coordinates></Point>\n" % (item["longitudeE7"] / 10000000, item["latitudeE7"] / 10000000))
                    f_out.write("    </Placemark>\n")
            f_out.write("  </Document>\n</kml>\n")

Here is my Google Collab Workflow:
https://colab.research.google.com/drive/1mJEyNImHHkmjdo3n_xC7OkpnHZK3AjQV?usp=sharing
vv
-- The Problem is my Final line of code:
--> Python location_history_json_converter.py "Location History.json" locations.csv -f csv -a 100 -i

For some reason, I keep getting an error message when I try to follow through with this Python code from your workflow and blog. Will you be able to alert to me what the issue is?

Tried both kml and gpxtracks formats, both output files are not imported by Google-maps->my maps -> import.
I see error "The file contains invalid or unsupported data, or the file is too large to parse." the output file is less than 2MB so it is not the large file failure. The output file is read fine by gpsvisualizer.com but not sure why google-maps is bailing.

If the input json file is huge it ends up loaded fully in RAM (due to json.loads method), for many cases some sort of streaming, iterative parsing would be enough.

I was going to inform you about the projects
https://pypi.org/project/ijson/
https://pypi.org/project/jsonslicer/

But see that you are already aware of it.

I got some heavy memory issues when using this script, non iterativly, and trying to host it on different infrastructre. So I did a quick rewrite, just to my use case using these libraries, which removed the memory issue totally, and increased the speed it took for me to parse a json file into the smaller segment this script is outputting. I wanted to make this reachable via APIs and not only terminal scripts. Also hosted on lightweight infrastructure that google offers for free.

I did not do an addition to the script so it cannot be a PR, I rather cleaned out everything I did not need and used this script as inspiration to make sure i have the right datapoints.

So just for information, this script is memory intensive, and can be solved with iterativly parsing. And I also struck the same issue as you, that it might be difficult to do chronological when iterative, but that is where I have arichitected my setup a bit differently to be able to do this. But it is then layered and using state.

But in the end, thanks for this good inspiration 🥳 and sorry for not beeing able to contribute back with the code I wrote, because it doesnt really fit into this script

Recently, when exporting location history, the Longitude values being exported seem to be invalid. I believe it may be a problem with the JSON being exported from Google, but was wondering if you had seen this before. Here is an example record from the exported JSON:

  {
    "timestampMs" : "1555378518327",
    "latitudeE7" : 42xxxxxxx,
    "longitudeE7" : 343xxxxxxx,
    "accuracy" : 17
  }

Exporting to KML, GPX, whatever, simply gives me longitude values of 343.xxxxxxx when valid values are supposed to be -180 to 180. The actual longitude I'm expecting is somewhere around -85.xxxxxxx. Simple math of subtracting 360 doesn't even work as that only gives me about -16.xxxxxxx.

Have you seen this before? Is this something your tool was designed to handle?

Hello,
Using the latest script and Python 3.11 on a mac and keep getting errors or empty CSV files.

python3 location_history_json_converter.py 2020_JANUARY.json testit2.csv -f 'csvfullest' -c

Traceback (most recent call last):
File "/Users/animo/Documents/_Code/location_history_json_converter.py", line 601, in
sys.exit(main())
^^^^^^
File "/Users/animo/Documents/_Code/location_history_json_converter.py", line 569, in main
items = data["locations"]
~~~~^^^^^^^^^^^^^
KeyError: 'locations'

I've been messing with the script myself to have it filter the data a bit better, and after figuring out that this was not possible with the commands directly went ahead and edited the code. I think a simple change adding these as params levels this up from a super powerful conversion tool into an even more powerful filtering tool

Google Takeout now includes an additional set of JSON files "Semantic Location History", which includes their estimate of what you were doing or what point of interest you were in, split into separate files by year and month.

It would be nice to take advantage of this data!

I get message: "ModuleNotFoundError: No module named 'dateutil'"
Here is how my json file look like at the begining at least "{
{
"locations": [{
"latitudeE7": 474664063,
"longitudeE7": 87602315,
"accuracy": 13,
"source": "WIFI",
"deviceTag": 1358307317,
"timestamp": "2013-08-24T09:33:50.739Z"
}, {
"latitudeE7": 474664061,
"longitudeE7": 87602352,
"accuracy": 12,
"activity": [{
"activity": [{
"type": "STILL",
"confidence": 82
}, {
"type": "IN_VEHICLE",
"confidence": 10
}, {
"type": "UNKNOWN",
"confidence": 7
}],
"timestamp": "2013-08-24T09:42:37.507Z"
}],
"source": "WIFI",
"deviceTag": 1358307317,
"timestamp": "2013-08-24T09:42:23.650Z"
}, {
"latitudeE7": 474664072,
"longitudeE7": 87602365,
"accuracy": 7,
"source": "WIFI",
"deviceTag": 1358307317,
"timestamp": "2013-08-24T09:43:12.941Z"
}, {
"latitudeE7": 474664074,
"longitudeE7": 87602396,
"accuracy": 6,
"source": "WIFI",
"deviceTag": 1358307317,
"timestamp": "2013-08-24T09:43:58.068Z"
}, {"
Completely noob here. Don t know what to do...

location-history-json-converter/location_history_json_converter.py", line 569, in main
items = data["locations"]
~~~~^^^^^^^^^^^^^
TypeError: list indices must be integers or slices, not str

This is the issue im getting. I know almost nothing about python so im not even sure how to begin fixing this. It is the standard location-history.json straight from google data for maps. Any help would be greatly appreciated.

Thanks

Hi, I'm getting a error (see attached). All the files are in the same folder and I'm running the following command:
'python location_history_json_converter.py 2020_JANUARY.json myshis.csv -f csvfullest'

Thanks in advance

Howdy!

This is my first time trying to use location_history_json_converter. I successfully installed python-pip & ijson on Debian 10.

The following execution:
python location_history_json_converter.py Records.json output.csv -f csv -s 2021-12-01 (0h00) -e 2021-12-31 (24h00)
or
python location_history_json_converter.py Records.json output.csv -f csv -s 2021-12-01 (0h00) -e 2021-12-31 (24h00) -i
results in the following error:

-bash: syntax error near unexpected token `('

This execution:
python location_history_json_converter.py Records.json output.csv -f csv -s 2021-12-01 -e 2021-12-31
or
python location_history_json_converter.py Records.json output.csv -f csv -s 2021-12-01 -e 2021-12-31 -i
results in the following error:

  File "location_history_json_converter.py", line 406
    print("\r%s / Locations written: %s" % (time.strftime("%Y-%m-%d %H:%M"), added), end="")
                                                                                        ^
SyntaxError: invalid syntax

I see the ^ is pointing to the equal sign in end=""

I also saw you recently changed line 405 from
time = datetime.utcfromtimestamp(int(item["timestampMs"]) / 1000)
to
time = datetime.utcfromtimestamp(int(_get_timestampms(item)) / 1000)

I point out these observations in hopes that they mean something to whoever responds and can guide you in how to fix the code or correct my execution syntax, whichever one is the problem.

Thanks & God bless!

I absolutely love this tool that Scarygami created, but I wanted something faster and more stripped down. I have created a version written in Haskell here, and if anyone cares, I can create binaries for whatever platform.

https://github.com/rickprice/google-location-history-converter

Scarygami, if you think this is rude, feel free to delete this.

Thanks
Frederick Price

Can I have an example of usage? The read me and -h do not match.

At first I had the issue with syntax cause I have a small file size and wasnt worried about start stop dates and times. Then I saw it was required when I was reading the issues post. I've tried it every way and then back again through the side window and kept getting unknown file attribute 1 or 2. I tried the example in the issues post, I tried the example on the description in the webpage. I even tried making it look exactly the way it is in the code. 23,59,59,999999. lol anybody out there have some knowledge about this, its driving me crazy.

python3 location_history_json_converter.py "2021_MAY.json" "NEW.csv" -f csvfullest -s 2021-04-30 --starttime (23h59) -e 2021-05-02 --endtime (23h59) -i -a 500 -p
zsh: unknown file attribute: 2

python3 location_history_json_converter.py "2021_MAY.json" "NEW.csv" -f csvfullest --starttime 2021-04-30 (1000) --endtime 2021-05-02 (1000) -i -a 500 -p -v
zsh: unknown file attribute: 1

python3 location_history_json_converter.py "2021_MAY.json" "NEW.csv" -f csvfullest -s 2021-04-30 (1000) -e 2021-05-02 (1000)
zsh: unknown file attribute: 1

python3 location_history_json_converter.py "2021_MAY.json" "NEW.csv" -f csvfullest -s 2021-04-30 (10h00) -e 2021-05-02 (2h00)
zsh: unknown file attribute: 1

python3 location_history_json_converter.py "2021_MAY.json" "NEW.csv" -f csvfullest -s 2021-04-30 (0h00) -e 2021-05-02 (0h00)
zsh: unknown file attribute: 0

python3 location_history_json_converter.py "2021_MAY.json" "NEW.csv" -f csv -s 2021-04-30 (23,59,59,999999) -e 2021-05-02 (23,59,59,999999) -i
zsh: unknown file attribute: 2

python3 location_history_json_converter.py [2021_MAY.json] [NEW.csv] -f csvfullest -s 2021-04-30 (23,59,59,999999) -e 2021-05-02 (23,59,59,999999) -i
zsh: unknown file attribute: 2

python3 location_history_json_converter.py [2021_MAY.json] [NEW.csv] -f csvfullest -s 2021-04-30 (23,59,59,999999) -e 2021-05-02 (23,59,59,999999) -i
zsh: unknown file attribute: 2

I'm getting this error:

File "location_history_json_converter.py", line 390
print("\r%s / Locations written: %s" % (time.strftime("%Y-%m-%d %H:%M"), added), end="")
^
SyntaxError: invalid syntax

When I try to specify start and end date with -s and -e it doesn't work.

python3 location_history_json_converter.py "Location History.json" vacation.gpx -f gpxtracks -s 2017–01–12 -e 2017–01–29 -a 500

location_history_json_converter.py: error: argument -s/--startdate: Not a valid date: '2017–01–12'.

I'm using Python 3.8, if it matters? Am I perhaps missing something?

Thank you very much for your great work. I have been trying to write code to do this winnowing of the huge, unwieldy Takeout file. So yesterday I let out a 'woo hoo' when I found this.

I've found that a chunk of my location data is me at (or near) home. Is there a way to include items NOT in a Polygon? Thanks.

Hi,

I get an error trying to run the script on Python 3.6.4:

/ Locations written: 0Traceback (most recent call last):
File "./location_history_json_converter.py", line 570, in
sys.exit(main())
File "./location_history_json_converter.py", line 563, in main
chronological=args.chronological
File "./location_history_json_converter.py", line 385, in convert
if "accuracy" in item and item["accuracy"] > accuracy:
TypeError: '>' not supported between instances of 'int' and 'NoneType'

I've tried with csv and gpx export.

Thanks.

The file format of the Takeout data has changed recently. Script needs to be updated to work again.

Hi,

I noticed you use datetime.fromtimestamp to convert epoch timestamps everywhere, but I believe the correct conversion would require datetime.utcfromtimestamp? My timestamps all seem to be UTC-based (and I think generally epoch timestamps are).

Thanks!

I am getting "error: unrecognized arguments" for input as described here:

http://stackoverflow.com/questions/17118999/python-argparse-unrecognized-arguments

The selected answer works: "If you omit sys.argv then parse_args parses sys.argv as expected."

Hi everybody,

with the current version I get:

  File "/home/phil/bin/location_history_json_converter.py", line 383
    print("\r%s / Locations written: %s" % (time.strftime("%Y-%m-%d %H:%M"), added), end="")
                                                                                        ^
SyntaxError: invalid syntax

Went back to the version of 20190714 which worked.

• Philip

Hello Scarygami ...
Firstly excuse my English and my null knowledge about python, but I want to use your json converter, which gives me an error so I am watching around the internet and didn't find an issue.

Could you help me???

Thanks in advance

Downloaded Location history JSON today (2019-04-17). Installed latest python and ran script. No matter what parameters I use I'm getting this error:

https://i.imgur.com/bmLDVrQ.png

I initially tried it with the following switches:

-o locations.csv -f csv -c

but one by one I've eliminated those to see whether my switches were the problem. Is this related to a change in the JSON format or am I doing something wrong? Thanks!

Hi,

based on the readme I cannot figure out how to provide the input file.
My file name is

LocationHistory.json

The command I execute:
python location_history_json_converter.py LocationHistory.json
The output I get:
usage: location_history_json_converter.py [-h] [-o OUTPUT] [-f {kml,json,csv,js,gpx,gpxtracks}] [-v VARIABLE] [-s STARTDATE] [-e ENDDATE] [-c] input location_history_json_converter.py: error: unrecognized arguments: LocationHisto ry.json

I have tried several ways to open it. I typed input in front of the file, input and filename without space. Nothing seems to work for me.
Can you please give a hint how this should be executed to parse the input file?

I love your script, really well written, but I don't see or understand how the distance between two locations is calculated, also the location names aren't saved.
I'm hoping to achieve something like this:

There seems to be a new field in the Records.json which includes the battery charging data. The script cannot properly parse and errors.

    "serverTimestamp": "2022-09-02T19:26:33.867Z",
    "deviceTimestamp": "2022-09-02T19:26:33.407Z",
    "batteryCharging": false,
    "formFactor": "PHONE",
    "timestamp": "2022-09-02T18:26:33.386Z"

Possible to add an option to convert the timestamp back to milliseconds and label from "timestamp" to "timestampMs"?

I use "location history visualizer pro" to read the json or kml location history files and it does not parse the new timestamp fields. There's no update from the developer nor does he respond to support requests recently since Google made the change in formatting.

Code is looking for "timestampMs" which no longer seems to be included in the Google Takeout data, however, "timestamp" is included and seems to be in an ISO date format.

I will see if I can create a PR to fix this.

Location of problem in code:

print("Progress:")
for item in locations:
    if "longitudeE7" not in item or "latitudeE7" not in item or "timestampMs" not in item:
        continue

Made a mistake deleting! Sorry

Hello there. Thank you for sharing the script. Unfortunately I am having two issues as described below. As I am a beginner in Python, I am grateful if anyone can tell me what I am doing wrong. Regards.

Traceback (most recent call last):
File "/Users/user/hello/location_history_json_converter.py", line 574, in
sys.exit(main())
File "/Users/user/hello/location_history_json_converter.py", line 542, in main
items = data["locations"]
KeyError: 'locations'

I'm using this project to split a very large (i.e. multiple year) file into individual files, one for each calendar day. I wanted to be able to split the days at a time other than 0000 UTC so I made the following changes.

At the top of the python file:

from datetime import timedelta

At the bottom of the python file (before the convert call):

    if args.startdate:
        args.startdate = args.startdate + timedelta(hours=7)

    if args.enddate:
        args.enddate = args.enddate + timedelta(hours=7)

It seems to work well. Of course it would be better to pass the hour offset on the command line but I haven't coded that part. If you're interested and want me to make a pull request I'd be happy to do so.

I installed python3 in my WSL2 environment to parse my location data. After getting errors I copied your commands from the Medium post and still can't get past 'invalid syntax'.

print("\r%s / Locations written: %s" % (time.strftime("%Y-%m-%d %H:%M"), added), end="") ^ SyntaxError: invalid syntax

Is this a WSL2 problem? If so then I'll not bug you anymore.

Python 3.6.9