mapbox / geojson-mapnikify Goto Github PK

View Code? Open in Web Editor NEW

38.0 112.0 21.0 450 KB

Transform GeoJSON objects into Mapnik XML stylesheets with embedded GeoJSON data and simplestyle-spec-derived styles.

License: ISC License

JavaScript 100.00%

geojson-mapnikify's Introduction

geojson-mapnikify

Transform GeoJSON objects into Mapnik XML stylesheets with embedded GeoJSON data and simplestyle-spec-derived styles.

install

As a dependency:

npm install --save @mapbox/geojson-mapnikify

As a binary:

npm install -g @mapbox/geojson-mapnikify

api

Assumptions:

GeoJSON is valid, and in EPSG:4326
Styles, if any, are expressed in simplestyle-spec
Mapnik 3.x is the rendering engine

binary

If you install -g, you can use geojson-mapnikify as a binary that takes a single GeoJSON file as an argument and writes a Mapnik XML stylesheet to stdout.

$ geojson-mapnikify test/data/point-retina.geojson > stylesheet.xml
$ geojson-mapnikify test/data/point-retina.geojson retina > stylesheet-retina.xml

`mapnikify(geojson, retina, callback)`

Transform GeoJSON into Mapnik XML.

geojson is a GeoJSON object.
retina is true or false for whether the style should be optimized for 2x rendering.
callback called with (err, xml) where xml is a string

url markers

If your GeoJSON object has one or more features with a marker-url property, mapnikify() will write the images found at the url into a file in a temporary directory and use that path in the Mapnik XML. This uses the needle library to handle the http file fetching.

By default the request will attempt to fetch binary data from the specified url. Mapnikify will use agentkeepalive to speed up requesting multiple images. There is also a default timeout of 5 seconds.

You can customize the defaults passed to needle() . Simply set a custom wrapper defined with needle.defaults . See needle's documentation on defaults for more information. For a quick example, this will set a longer timeout:

var mapnikify = require('mapnikify');
var myRequest = require('needle').defaults({
  timeout: 10000,
  followRedirect: false
});
mapnikify.setRequestClient(myRequest);

mapnikify(geojson, retina, callback);

mapnikify.setRequestClient(null); // return to mapnikify defaults

geojson-mapnikify's People

Contributors

Stargazers

Watchers

Forkers

xkwangy jvrousseau cule ospreyx tcql holmesal tesera twaeng testbigorg miklobit taylorj999 rgwozdz kartotherian brycejohnston isabella232 liubogis wikimedia mapclone myfreebrain

geojson-mapnikify's Issues

agentkeepalive on https?

It looks like agentkeepalive is not used on HTTPS requests - this might be a relic from the past, when there was not support for HTTPS clients in agentkeepalive.

geojson-mapnikify/lib/get.js

Lines 58 to 59 in d40a0b4

 // Don't use keepalive agent for https. 

 params.agent = uri.indexOf('https') === 0 ? null : agent;

This is just something I noticed when reviewing a recent PR, and therefore created this separate issue about it.

@DzmitryU @springmeyer

Support retina

Unable to install this package

I'm unable to install this package on my windows 10 machine. Here's the log after running simply
npm install -g @mapbox/geojson-mapnikify

PS C:\Users\goodh\workspace\goout\MapService\src> npm install -g @mapbox/geojson-mapnikify
npm WARN deprecated queue-async@1.0.7: renamed to d3-queue
npm WARN deprecated protozero@1.5.1: protozero should no longer be used via npm, install instead via https://github.com/mapbox/mason
npm WARN deprecated maki@0.5.0: maki is now namespaced, install @mapbox/maki instead
npm WARN deprecated node-uuid@1.4.8: Use uuid module instead
C:\Users\goodh\AppData\Roaming\npm\geojson-mapnikify -> C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\bin\geojson-mapnikify

> mapnik@3.7.2 install C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik
> node-pre-gyp install --fallback-to-build

node-pre-gyp ERR! Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/mapnik/v3.7.2/node-v57-win32-x64-Release.tar.gz
node-pre-gyp ERR! Pre-built binaries not found for mapnik@3.7.2 and node@8.11.3 (node-v57 ABI, unknown) (falling back to source compile with node-gyp)
'mapnik-config' is not recognized as an internal or external command,
operable program or batch file.
gyp: Call to 'mapnik-config --ldflags' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\configure.js:336:16)
gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "configure" "--fallback-to-build" "--module=C:\\Users\\goodh\\AppData\\Roaming\\npm\\node_modules\\@mapbox\\geojson-mapnikify\\node_modules\\mapnik\\lib\\binding\\mapnik.node" "--module_name=mapnik" "--module_path=C:\\Users\\goodh\\AppData\\Roaming\\npm\\node_modules\\@mapbox\\geojson-mapnikify\\node_modules\\mapnik\\lib\\binding" "--python=C:\\Users\\goodh\\.windows-build-tools\\python27\\python.exe" "--msvs_version=2015"
gyp ERR! cwd C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik
gyp ERR! node -v v8.11.3
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute 'C:\Program Files\nodejs\node.exe C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js configure --fallback-to-build --module=C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik\lib\binding\mapnik.node --module_name=mapnik --module_path=C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik\lib\binding --python=C:\Users\goodh\.windows-build-tools\python27\python.exe --msvs_version=2015' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik\node_modules\node-pre-gyp\lib\util\compile.js:83:29)
node-pre-gyp ERR! stack     at emitTwo (events.js:126:13)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:925:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
node-pre-gyp ERR! System Windows_NT 10.0.17134
node-pre-gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\goodh\\AppData\\Roaming\\npm\\node_modules\\@mapbox\\geojson-mapnikify\\node_modules\\mapnik\\node_modules\\node-pre-gyp\\bin\\node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik
node-pre-gyp ERR! node -v v8.11.3
node-pre-gyp ERR! node-pre-gyp -v v0.6.39
node-pre-gyp ERR! not ok
Failed to execute 'C:\Program Files\nodejs\node.exe C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js configure --fallback-to-build --module=C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik\lib\binding\mapnik.node --module_name=mapnik --module_path=C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik\lib\binding --python=C:\Users\goodh\.windows-build-tools\python27\python.exe --msvs_version=2015' (1)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! mapnik@3.7.2 install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the mapnik@3.7.2 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\goodh\AppData\Roaming\npm-cache\_logs\2018-08-11T08_11_59_364Z-debug.log

I guess it's the same problem as when trying to run this package on RunKit: which also fails.

Use CSV Plugin

The Mapnik CSV plugin uses Mapnik's native GeoJSON parser which is quite a bit faster than OGR. But a bug in Mapnik is causing this commit to break mapbox/tilelive-overlay@989a961, if used with geojson-mapnikify after b65856b.

Going to keep this open until the Mapnik bug is fixed: mapnik/mapnik#2319

Incorrect GeoJSON generation for multipoint geometries

When a multipoint geometry is passed into geojson mapnikify it does not generate the expected properties like marker-path, marker-size, and marker-color. This leads to an empty rendering.

I've pushed a test that includes the correct expected image (and hand fixed XML) that is intentionally failing: https://github.com/mapbox/geojson-mapnikify/compare/multipoint

This test should start passing once the bug is fixed.

XML generation for feature collections is extremely inefficient

Take a geojson like:

{
    "type": "FeatureCollection",
    "features": [
        { "type": "Feature",
            "geometry": {
                "type": "Point",
                "coordinates": [0, 0]
            },
            "properties": {
                "marker-size": "medium",
                "marker-symbol": "bus",
                "marker-color": "#ace"
            }
        }, {
            "geometry": {
                "type": "Point",
                "coordinates": [1, 1]
            },
            "properties": {
                "marker-size": "medium",
                "marker-symbol": "bus",
                "marker-color": "#ace"
            }
        }
    ]
}

Currently this is the XML generated:

<?xml version="1.0" encoding="utf-8"?>
<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
  <Style name="style-0">
    <Rule>
      <PointSymbolizer file="/var/folders/jv/bhkldrpj41jbkpzxgc9ggr_r0000gn/T/73c52cc1d5ce4d4cb149a80c09d6a9db.png" allow-overlap="true" ignore-placement="true"/>
    </Rule>
  </Style>
  <Style name="style-1">
    <Rule>
      <PointSymbolizer file="/var/folders/jv/bhkldrpj41jbkpzxgc9ggr_r0000gn/T/73c52cc1d5ce4d4cb149a80c09d6a9db.png" allow-overlap="true" ignore-placement="true"/>
    </Rule>
  </Style>
  <Layer name="layer-0" srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
    <StyleName>style-0</StyleName>
    <Datasource>
      <Parameter name="type">csv</Parameter>
      <Parameter name="inline">geojson
'{"type":"Point","coordinates":[0,0]}'</Parameter>
    </Datasource>
  </Layer>
  <Layer name="layer-1" srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
    <StyleName>style-1</StyleName>
    <Datasource>
      <Parameter name="type">csv</Parameter>
      <Parameter name="inline">geojson
'{"type":"Point","coordinates":[1,1]}'</Parameter>
    </Datasource>
  </Layer>
</Map>

Instead this xml should be generated and would perform much better as the length of features grows:

<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
  <Style name="style-0">
    <Rule>
      <PointSymbolizer file="/var/folders/jv/bhkldrpj41jbkpzxgc9ggr_r0000gn/T/73c52cc1d5ce4d4cb149a80c09d6a9db.png" allow-overlap="true" ignore-placement="true"/>
    </Rule>
  </Style>
  </Style>
  <Layer name="layer-0" srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
    <StyleName>style-0</StyleName>
    <Datasource>
      <Parameter name="type">csv</Parameter>
      <Parameter name="inline">geojson
'{"type":"Point","coordinates":[0,0]}'
'{"type":"Point","coordinates":[1,1]}'
</Parameter>
    </Datasource>
  </Layer>
</Map>

Add binary

Use Mapnik geojson plugin (rather than CSV)

We should start targeting the Mapnik geojson native plugin, which like the CSV Plugin(#3) uses Mapnik's fast json parsing grammars. But unlike the CSV plugin we know the whole payload is geojson so there is less tokenization cost.

Specifically, after further optimization in Mapnik 3.x, profiling now shows the tokenization + quoting work to prepare to parse rows as geojson geometries is taking longer than parsing the actual geometries:

NOTE: this is with this geojson file with lots of features: https://github.com/mapbox/tilelive-overlay/blob/bench-async/test/data/complex.geojson

The catch is that the geojson plugin only supports featurecollections. So in the case of single geometries, they will need to be wrapped. But this should have minimal overhead and the case of multiple features will then go faster.

Update agentkeepalive version

There was an issue in old version of agentkeepalive version, that might cause error: node-modules/agentkeepalive#35
To avoid that, dependency should be updated to newest version.

[Security] Update `request` dependency

pre-v2.74 releases of request contain a vulnerability due to an outdated dependency on tough-cookie https://nodesecurity.io/advisories/130

	// Don't use keepalive agent for https.
	params.agent = uri.indexOf('https') === 0 ? null : agent;