Giter VIP home page Giter VIP logo

geojson-mapnikify's Issues

Use Mapnik geojson plugin (rather than CSV)

We should start targeting the Mapnik geojson native plugin, which like the CSV Plugin(#3) uses Mapnik's fast json parsing grammars. But unlike the CSV plugin we know the whole payload is geojson so there is less tokenization cost.

Specifically, after further optimization in Mapnik 3.x, profiling now shows the tokenization + quoting work to prepare to parse rows as geojson geometries is taking longer than parsing the actual geometries:

screen shot 2014-08-20 at 3 46 54 pm

NOTE: this is with this geojson file with lots of features: https://github.com/mapbox/tilelive-overlay/blob/bench-async/test/data/complex.geojson

The catch is that the geojson plugin only supports featurecollections. So in the case of single geometries, they will need to be wrapped. But this should have minimal overhead and the case of multiple features will then go faster.

agentkeepalive on https?

It looks like agentkeepalive is not used on HTTPS requests - this might be a relic from the past, when there was not support for HTTPS clients in agentkeepalive.

geojson-mapnikify/lib/get.js

Lines 58 to 59 in d40a0b4

// Don't use keepalive agent for https.
params.agent = uri.indexOf('https') === 0 ? null : agent;

This is just something I noticed when reviewing a recent PR, and therefore created this separate issue about it.

@DzmitryU @springmeyer

XML generation for feature collections is extremely inefficient

Take a geojson like:

{
    "type": "FeatureCollection",
    "features": [
        { "type": "Feature",
            "geometry": {
                "type": "Point",
                "coordinates": [0, 0]
            },
            "properties": {
                "marker-size": "medium",
                "marker-symbol": "bus",
                "marker-color": "#ace"
            }
        }, {
            "geometry": {
                "type": "Point",
                "coordinates": [1, 1]
            },
            "properties": {
                "marker-size": "medium",
                "marker-symbol": "bus",
                "marker-color": "#ace"
            }
        }
    ]
}

Currently this is the XML generated:

<?xml version="1.0" encoding="utf-8"?>
<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
  <Style name="style-0">
    <Rule>
      <PointSymbolizer file="/var/folders/jv/bhkldrpj41jbkpzxgc9ggr_r0000gn/T/73c52cc1d5ce4d4cb149a80c09d6a9db.png" allow-overlap="true" ignore-placement="true"/>
    </Rule>
  </Style>
  <Style name="style-1">
    <Rule>
      <PointSymbolizer file="/var/folders/jv/bhkldrpj41jbkpzxgc9ggr_r0000gn/T/73c52cc1d5ce4d4cb149a80c09d6a9db.png" allow-overlap="true" ignore-placement="true"/>
    </Rule>
  </Style>
  <Layer name="layer-0" srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
    <StyleName>style-0</StyleName>
    <Datasource>
      <Parameter name="type">csv</Parameter>
      <Parameter name="inline">geojson
'{"type":"Point","coordinates":[0,0]}'</Parameter>
    </Datasource>
  </Layer>
  <Layer name="layer-1" srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
    <StyleName>style-1</StyleName>
    <Datasource>
      <Parameter name="type">csv</Parameter>
      <Parameter name="inline">geojson
'{"type":"Point","coordinates":[1,1]}'</Parameter>
    </Datasource>
  </Layer>
</Map>

Instead this xml should be generated and would perform much better as the length of features grows:

<Map srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">
  <Style name="style-0">
    <Rule>
      <PointSymbolizer file="/var/folders/jv/bhkldrpj41jbkpzxgc9ggr_r0000gn/T/73c52cc1d5ce4d4cb149a80c09d6a9db.png" allow-overlap="true" ignore-placement="true"/>
    </Rule>
  </Style>
  </Style>
  <Layer name="layer-0" srs="+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs">
    <StyleName>style-0</StyleName>
    <Datasource>
      <Parameter name="type">csv</Parameter>
      <Parameter name="inline">geojson
'{"type":"Point","coordinates":[0,0]}'
'{"type":"Point","coordinates":[1,1]}'
</Parameter>
    </Datasource>
  </Layer>
</Map>

Unable to install this package

I'm unable to install this package on my windows 10 machine. Here's the log after running simply
npm install -g @mapbox/geojson-mapnikify

PS C:\Users\goodh\workspace\goout\MapService\src> npm install -g @mapbox/geojson-mapnikify
npm WARN deprecated queue-async@1.0.7: renamed to d3-queue
npm WARN deprecated protozero@1.5.1: protozero should no longer be used via npm, install instead via https://github.com/mapbox/mason
npm WARN deprecated maki@0.5.0: maki is now namespaced, install @mapbox/maki instead
npm WARN deprecated node-uuid@1.4.8: Use uuid module instead
C:\Users\goodh\AppData\Roaming\npm\geojson-mapnikify -> C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\bin\geojson-mapnikify

> mapnik@3.7.2 install C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik
> node-pre-gyp install --fallback-to-build

node-pre-gyp ERR! Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/mapnik/v3.7.2/node-v57-win32-x64-Release.tar.gz
node-pre-gyp ERR! Pre-built binaries not found for mapnik@3.7.2 and node@8.11.3 (node-v57 ABI, unknown) (falling back to source compile with node-gyp)
'mapnik-config' is not recognized as an internal or external command,
operable program or batch file.
gyp: Call to 'mapnik-config --ldflags' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\lib\configure.js:336:16)
gyp ERR! stack     at emitTwo (events.js:126:13)
gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Windows_NT 10.0.17134
gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "configure" "--fallback-to-build" "--module=C:\\Users\\goodh\\AppData\\Roaming\\npm\\node_modules\\@mapbox\\geojson-mapnikify\\node_modules\\mapnik\\lib\\binding\\mapnik.node" "--module_name=mapnik" "--module_path=C:\\Users\\goodh\\AppData\\Roaming\\npm\\node_modules\\@mapbox\\geojson-mapnikify\\node_modules\\mapnik\\lib\\binding" "--python=C:\\Users\\goodh\\.windows-build-tools\\python27\\python.exe" "--msvs_version=2015"
gyp ERR! cwd C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik
gyp ERR! node -v v8.11.3
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
node-pre-gyp ERR! build error
node-pre-gyp ERR! stack Error: Failed to execute 'C:\Program Files\nodejs\node.exe C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js configure --fallback-to-build --module=C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik\lib\binding\mapnik.node --module_name=mapnik --module_path=C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik\lib\binding --python=C:\Users\goodh\.windows-build-tools\python27\python.exe --msvs_version=2015' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik\node_modules\node-pre-gyp\lib\util\compile.js:83:29)
node-pre-gyp ERR! stack     at emitTwo (events.js:126:13)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:214:7)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:925:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
node-pre-gyp ERR! System Windows_NT 10.0.17134
node-pre-gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Users\\goodh\\AppData\\Roaming\\npm\\node_modules\\@mapbox\\geojson-mapnikify\\node_modules\\mapnik\\node_modules\\node-pre-gyp\\bin\\node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik
node-pre-gyp ERR! node -v v8.11.3
node-pre-gyp ERR! node-pre-gyp -v v0.6.39
node-pre-gyp ERR! not ok
Failed to execute 'C:\Program Files\nodejs\node.exe C:\Program Files\nodejs\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js configure --fallback-to-build --module=C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik\lib\binding\mapnik.node --module_name=mapnik --module_path=C:\Users\goodh\AppData\Roaming\npm\node_modules\@mapbox\geojson-mapnikify\node_modules\mapnik\lib\binding --python=C:\Users\goodh\.windows-build-tools\python27\python.exe --msvs_version=2015' (1)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! mapnik@3.7.2 install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the mapnik@3.7.2 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\goodh\AppData\Roaming\npm-cache\_logs\2018-08-11T08_11_59_364Z-debug.log

I guess it's the same problem as when trying to run this package on RunKit: which also fails.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.