This node script is intended to be used in order to translate Google takeout locations history to geojson.

• What is GeoJson? --> read here
• What is Google location history? --> read here
• Where can I get my Google account location history? --> look here

What follows is the content structure of a Google location history archive:

• .zip archive
  • .html info file
  • locations history folder
    • .json locations history file

A basic example of an input file

{
  "locations": [
    {
      "timestampMs": "1507330772000",
      "latitudeE7": 419058658,
      "longitudeE7": 125218684,
      "accuracy": 16,
      "velocity": 0,
      "altitude": 66,
      "activity": [
        {
          "timestampMs": "1507049587082",
          "activity": [
            {
              "type": "TILTING",
              "confidence": 100
            }
          ]
        },
        {
          "timestampMs": "1507049736368",
          "activity": [
            {
              "type": "IN_VEHICLE",
              "confidence": 33
            },
            {
              "type": "STILL",
              "confidence": 33
            },
            {
              "type": "UNKNOWN",
              "confidence": 17
            },
            {
              "type": "ON_FOOT",
              "confidence": 12
            },
            {
              "type": "WALKING",
              "confidence": 12
            },
            {
              "type": "ON_BICYCLE",
              "confidence": 6
            }
          ]
        }
      ]
    }
  ]
}

A basic example of an output file

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [
          12.5218684,
          41.9058658
        ]
      },
      "properties": {
        "timestamp": "2017-10-06T22:59:32.000Z",
        "accuracy": 16,
        "velocity": null,
        "altitude": 66
      }
    }
  ]
}

Even if the transformation is straightforward it is not possible to achieve it using a simple array map because of the huge amount of data to manage.

• File: 2.json
• Size: 1.16 GB
• Error message: Cannot create a string longer than 0x3fffffe7 characters
• Error code: ERR_STRING_TOO_LONG

Node cannot buffer the file for us because:

• the size of the file is bigger than the maximum one Node.js itself is capable of creating a string for.
• the size of the file is bigger than the maximum one Node.js is able to store in memory at once.

Defined in the streamProcessing file.

1. [Read] --- fileToStream --> Input file to stream.
2. [Transform] --- streamParser --> Consumes text, and produces a stream of data items corresponding to high-level tokens.
3. [Transform] --- streamPicker --> Is a token item filter, it selects objects from a stream ignoring the rest and produces a stream of objects (locations in our case).
4. [Transform] --- streamArrayer --> It assumes that an input token stream represents an array of objects and streams out assembled JavaScript objects (locations in our case).
5. [Transform] --- streamGeoJsoner --> It transforms google takeout locations into GeoJson locations.
6. [Transform] --- streamStringer --> It stringifies GeoJson locations.
7. [Write] --- streamToFile --> Stream to Output file.

• node v13.7.0

• npm i

• npm run test

• Copy one or more Google location history json files inside the ./input folder.
• npm run start [-- fromTimestampMs toTimestampMs]
• Access the GeoJson files results of the translation inside the ./output folder.

You can optionally set a time range window in order to filter locations out of the output.
Timestamp parameters must be defined as milliseconds epoch timestamps.

• fromTimestampMs: lower bound timestamp.
• toTimestampMs: upper bound timestamp.

Note that you can as well set only one of the two bounds either the lower or the upper one.

1. Translate files without applying time filtering:
   • npm run start
2. Maintain only locations with timestamp between Tuesday, 1 January 2019 00:00:00 and Tuesday, 31 December 2019 23:59:59:
   • npm run start -- 1546300800000 1577836799000
3. Maintain only locations with timestamp subsequent to Tuesday, 1 January 2019 00:00:00:
   • npm run start -- 1546300800000
4. Maintain only locations with timestamp prior to Tuesday, 31 December 2019 23:59:59:
   • npm run start -- x 1577836799000

• 1_locations.json
• 4_locations.json
• x_empty.json --> this is going to fail!

Please note the journey of the 2.json file:

• 1167.11297 Mb --> 1.16 GB
• 3688763 --> almost 4 milion locations processed
• 551.443 Secs --> almost 10 minutes processing

Please note how the number of GeoJson locations in the output decreases.

• latitude and longitude data incorrect in google takeout file