cschwarz / wkx Goto Github PK

View Code? Open in Web Editor NEW

306.0 21.0 56.0 611 KB

A WKT/WKB/EWKT/EWKB/TWKB/GeoJSON parser and serializer.

License: MIT License

JavaScript 100.00%

wkx's Introduction

wkx

A WKT/WKB/EWKT/EWKB/TWKB/GeoJSON parser and serializer with support for

Point
LineString
Polygon
MultiPoint
MultiLineString
MultiPolygon
GeometryCollection

Examples

The following examples show you how to work with wkx.

var wkx = require('wkx');

//Parsing a WKT string
var geometry = wkx.Geometry.parse('POINT(1 2)');

//Parsing an EWKT string
var geometry = wkx.Geometry.parse('SRID=4326;POINT(1 2)');

//Parsing a node Buffer containing a WKB object
var geometry = wkx.Geometry.parse(wkbBuffer);

//Parsing a node Buffer containing an EWKB object
var geometry = wkx.Geometry.parse(ewkbBuffer);

//Parsing a node Buffer containing a TWKB object
var geometry = wkx.Geometry.parseTwkb(twkbBuffer);

//Parsing a GeoJSON object
var geometry = wkx.Geometry.parseGeoJSON({ type: 'Point', coordinates: [1, 2] });

//Serializing a Point geometry to WKT
var wktString = new wkx.Point(1, 2).toWkt();

//Serializing a Point geometry to WKB
var wkbBuffer = new wkx.Point(1, 2).toWkb();

//Serializing a Point geometry to EWKT
var ewktString = new wkx.Point(1, 2, undefined, undefined, 4326).toEwkt();

//Serializing a Point geometry to EWKB
var ewkbBuffer = new wkx.Point(1, 2, undefined, undefined, 4326).toEwkb();

//Serializing a Point geometry to TWKB
var twkbBuffer = new wkx.Point(1, 2).toTwkb();

//Serializing a Point geometry to GeoJSON
var geoJSONObject = new wkx.Point(1, 2).toGeoJSON();

Browser

To use wkx in a webpage, simply copy a built browser version from dist/ into your project, and use a script tag to include it:

<script src="wkx.js"></script>

If you use browserify for your project, you can simply npm install wkx --save, and just require wkx as usual in your code.

Regardless of which of the preceeding options you choose, using wkx in the browser will look the same:

var wkx = require('wkx');

var geometry = wkx.Geometry.parse('POINT(1 2)');

console.log(geometry.toGeoJSON());

In addition to the wkx module, the browser versions also export buffer, which is useful for parsing WKB:

var Buffer = require('buffer').Buffer;
var wkx = require('wkx');

var wkbBuffer = new Buffer('0101000000000000000000f03f0000000000000040', 'hex');
var geometry = wkx.Geometry.parse(wkbBuffer);

console.log(geometry.toGeoJSON());

wkx's People

Contributors

Stargazers

Watchers

wkx's Issues

The toWkb function does not output the SRID

The bug appears to be in the _writeWkbType function

3D and 4D type flags with no SRID

I'm one of the maintainers of the PHP library creof/wkb-parser. While investigating an issue I noticed your library uses a different set of flags to indicate 3D and 4D geometry when an object has an undefined SRID. I'd like to add support in our library. I'm hoping you can tell me what database uses this format or possibly point me towards documentation on it.

Thanks!

Package

Ever though about building this into a npm package and push. Quite useful code you got here.

Buffer() is Deprecated

On version: 0.5.0, node version 12.13.0.

Warning that shows up:

DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.

Replicated from calling wkx.Geometry.parseGeoJSON(data).toWkb().toString('base64')

No stack trace, but this usually because the code is using new Buffer(), simply swapping it over to Buffer.from() is more than enough

Make default SRID of 4326 in parseGeoJSON optional

There should be a way to opt out from assigning default SRID of 4326 in case of GeoJSON format, here: https://github.com/cschwarz/wkx/blob/master/lib/geometry.js#L254.

toWkb not the same as input

I want to create wkb from wkx Object but function toWkb returns an invalid WKB and it is not the same as parsed.
Example:

wkx.Geometry.parse(new Buffer("0101000000000000000000f03f0000000000000040", "hex")).toWkb().join('')
"1100000000024063000000064"

I tried with Chrome 80.0.3987.149 Is it a bug or not the correct way to create Wkb.

new Buffer("1100000000024063000000064", "hex")
Uncaught ReferenceError: Buffer is not defined
at :1:1

Use alternatives to nodejs modules

Hello there!

First, let me say thanks and I love the wkx package! It works great.

One thing I ran across recently was that wkx will not compile in a react-native based project. As far as I can tell, it looks like this is due to the use of util.inherits and Buffer The react-native ecosystem does not support any nodejs runtime modules, in this case the util module and the Buffer built-in.

So I was wondering if you would be willing to swap out the node stuff for more portable alternatives like node-util-inherits and buffer instead?

If not, I suppose I could fork wkx, but it seems like a change that might benefit others as well without creating a separate repo.

Anyway, thanks for all the great work.

Where does Buffer come from?

Trying to read well known binary and am assuming that the wkx.Gemoetry.parse() function requires a Buffer object rather than a byte array. Not sure where Buffer is defined. Can you provide some guidance?

Where is the "browser version" in "dist/"

To use wkx in a webpage, simply copy a built browser version from dist/ into your project, and use a script tag to include it:

I need to use a browser version for my project but I can't find it.
Thanks for your help!

wkx_py

Hi Christian,

I have implemented a python library to parse twkb and geojson using your wkx parser as reference. Would you be willing to look it over for a code review?

Thanks

add more doc -- not sure about latitude and longitude

It might sounds a bit weird but it is nonetheless true, I always think whether the first item in the coordinates is latitude or longitude

const position = '0101000020E61000003333333333331F409A99999999D94A40'
const buffer = Buffer.from(position, "hex")
const geometry = wkx.Geometry.parse(buffer)
const transformedPosition = geometry.toGeoJSON() 
console.log(
  'lat': transformedPosition.coordinates[0],
  'lng': transformedPosition.coordinates[1]
)

So I really appreciate it if you could mention it in the doc and answer me here. Is my assumption correct.

*I know in google map it is [lat, lng]
*But in PostGIS which is my DB too it is the other way around, [lng, lat]. Also there it says:

In web mapping APIs like Google Maps, spatial coordinates are 🔴often🔴 in order of latitude then longitude.

*And in mongodb it is also `[lat, lng]

Parsing with illegal numbers should fail

We use wkx to validate and parse WKX strings coming as variables in HTTP requests.
We've noticed that it validates the general structure of the WKT when parsing but doesn't validate the numbers.

For example, wkx.Geometry.parse("POINTTTT(1 2)") will fail because there is no POINTTTT in WKT.
But, parsing wkx.Geometry.parse("POINT(a b)") will not fail, instead it will return a point with NaN for numbers:

Point {
  srid: undefined,
  hasZ: false,
  hasM: false,
  x: NaN,
  y: NaN,
  z: undefined,
  m: undefined
}

Move @types/node to devDependency

Is there a particular reason why the @types/node is being specified as a production dependency? I have a typescript project that uses a library that has a dependency on wkx. Because wkx is installing the @types/node module it is playing havoc with my typescript project (i have @types/node included somewhere else and it is causing duplicate identifier issues with the tsc compiler.). Can the module be moved to be only a dev dependency?

Remove unneeded dependencies from being published

Can you change the .gitignore file and add a .npmignore file like the below to this repo to keep the unneeded folders out of the package? See https://docs.npmjs.com/misc/developers#keeping-files-out-of-your-package

.gitignore file

dist/
build/

.npmignore file

tests/
*.test.js

Stack size error when parsing a larger WKT input string

Observed

I observe the following error message when parsing the attached longer WKT input string:

/home/wouter/tmp/geo-test/node_modules/wkx/lib/multipolygon.js:62
        exteriorRing.push.apply(exteriorRing, value.matchCoordinates(options));
                          ^

RangeError: Maximum call stack size exceeded
    at Function.MultiPolygon._parseWkt (/home/wouter/tmp/geo-test/node_modules/wkx/lib/multipolygon.js:62:27)
    at Function.Geometry._parseWkt (/home/wouter/tmp/geo-test/node_modules/wkx/lib/geometry.js:67:33)
    at Function.Geometry.parse (/home/wouter/tmp/geo-test/node_modules/wkx/lib/geometry.js:26:25)
    at Object.<anonymous> (/home/wouter/tmp/geo-test/mwe.js:3:14)
    at Module._compile (node:internal/modules/cjs/loader:1126:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
    at Module.load (node:internal/modules/cjs/loader:1004:32)
    at Function.Module._load (node:internal/modules/cjs/loader:839:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47

Expected

The WKT input string to parse correctly, or a syntax error in case the input string is malformed, but not a memory/allocation-related error.

Steps to reproduce

Download the attached file mwe.txt that contains the input string. (The string is too large to comfortably share as inline text.)
Run the following script and observe the error communitated above:

const fs = require('fs')
const wkx = require('wkx')
const input = fs.readFileSync('mwe.txt', 'utf8')
wkx.Geometry.parse(input)

mwe.txt

How to parse a generic geometry blob to GeoJSON?

My application has a GeoPackage with a feature table that holds a variety of geometry types. I use wkx to parse my geometry blob, so I can convert it to GeoJSON:

wkx.Geometry.parse(r.geom);

But it gives me the following error:

Error parsing geom blob: Error: GeometryType 0 not supported

GeometryType 0 references the generic Geometry data type since my GeoPackage feature table has mixed geometry types. Is it possible for this library to support parsing the generic Geometry type?

If not, is it possible for me to explicitly tell wkx the type when it's parsing it? Something like:

wkx.Geometry.parse(r.geom, "Point");

Conversion for feature collections from TWKB to GeoJSON looks wrong

The following code sample includes a repro. I start with a TWKB string generated using PostGIS, then compare the outputs of two libraries, wkx and twkb.

twkb yields the expected result, with 5 vertices. wkx generates one polygon (instead of two), with 912 vertices (instead of 5).

var wkx = require('wkx');
var twkb = require('twkb');

/* NOTE: generated from the following query:
 * select encode(st_astwkb(
 *   ARRAY['POLYGON((25.9601817282581 48.0424439290235,25.9665092948186 48.0409189127411,25.9689305626008 48.0454493856918,25.9624693171852 48.046795992248,25.9601817282581 48.0424439290235))'::GEOMETRY, 'POLYGON((25.9601817282581 48.0424439290235,25.9665092948186 48.0409189127411,25.9689305626008 48.0454493856918,25.9624693171852 48.046795992248,25.9601817282581 48.0424439290235))'::GEOMETRY]::geometry[]
 *   , ARRAY[123, 456], 5
 *   ), 'base64');
 */
var base64 = 'pgQC9gGQBwEF5PK8Aqi6ygTyCa8C5AOKB4sKjgLJA+cGAQUAAPIJrwLkA4oHiwqOAskD5wY=';

var buf = Buffer.from(base64, 'base64');


console.log('====== twkb:');
var twkbRes = twkb.toGeoJSON(buf);
console.log('twkb num features:', twkbRes.features.length);
console.log('twkb feature ids:', twkbRes.features.map(x => x.id));
console.log('twkb coords per feature:', twkbRes.features.map(x => x.geometry.coordinates[0].length));
console.log(JSON.stringify(twkbRes, null, 2));

console.log('====== wkx:');
var wkxRes = wkx.Geometry.parseTwkb(buf);
console.log('wkx num features:', wkxRes.polygons.length);
console.log('wkx feature ids:', wkxRes.polygons.map(x => x.id));
console.log('wkx coords per feature (ext ring):', wkxRes.polygons.map(x => x.exteriorRing.length));
// console.log(JSON.stringify(wkxRes, null, 2));

Rewrite as es module

Hello, When bundling this library with rollup it throws an error because Geometry is being declared after it's being called. This is due to circular dependencies that the current commonjs format doesn't sort out correctly. Rewriting this library as an es module would fix the problem.

"Expected group end" on valid LineString

Hi,
This LineString

LINESTRING(18.6233900000 -33.8963590000 0, 18.6234010000 -33.8966330000 0, 18.6234090000 -33.8967650000 0, 18.6234420000 -33.8973360000 0, 18.6234490000 -33.8974650000 0, 18.6234790000 -33.8980180000 0, 18.6234860000 -33.8981470000 0, 18.6235040000 -33.8985640000 0)

used like this

let feature = {
      type: 'Feature',
      properties: {},
      geometry: wkx.Geometry.parse(linestring).toGeoJSON()
};

throws this
"Expected group end"

Is it a 3D issue?
Viewing the linestring in WKT view does display correctly

getting invalid geoJSON

I am passing a WKB in hexadecimal format and getting coordinates like this 405319.7331131318, 4668436.547074934 which I think is not valid coordinates.
the procedure I am following is like this

var wkbBuffer = new Buffer('hexadecimal WKB', 'hex');
var geometry = wkx.Geometry.parse(wkbBuffer);
console.log(geometry.toGeoJSON());

Could you please help me out here...

wkx should add closing points to polygons when necessary

Some formats like GeoJSON don't require that a polygon's last point match its first, while others like WKT do.

For example:

var wkx = require("wkx");

var validGeoJSON = {type: 'Polygon', coordinates: [[[-117, 32], [-116, 33], [-115, 34]]]};
var invalidWkt = wkx.Geometry.parseGeoJSON(validGeoJSON).toWkt();

console.log(invalidWkt);
// => "POLYGON((-117 32,-116 33,-115 34))"

Wkx should ensure the presence of closing points when converting to a format that requires it, either by normalizing to closed topologies on input, or conditionally adding on output.

TWKB precision

When using the toTwkb method on any geometry, the default, hardcoded value for precision is 5. I'd like to go even lower but I don't think it's possible without modifying the library's code.

Is customizing precision something that is planned for the future or is this library's goal to keep things simple? I'd really like to use this and I want to know what my options are.

Serializing a Point geometry to EWKT

Excellent library, thanks!

The README.md could be improved with:

var point = new wkx.Point(1, 2);
point.srid = 4326;
var ewktString = point.toEwkt();

because the current:

var ewktString = new wkx.Point(1, 2).toEwkt();

does not make much sense (SRID ending up being undefined, which does not comply with EWKT).

Also, it would be elegant to be able to set the SRID from the constructor...

toWkt output

Hi,

This is the geometry object:

Point {
    srid: undefined, 
    hasZ: true, 
    hasM: false, 
    x: 389671.879, 
    y: 263437.527,
    z: 0
}

When I call .toWkt() it outputs:
POINT Z (389671.879 263437.527 0)

It shouldn't be POINT(389671.879 263437.527 0) ?

Thanks :)

dependencies and devDependencies

Why dependencies and devDependencies have opposite to normal meaning in package.json?
I mean, dependencies should contain what is necessary to work and devDependencies - what is necessary for testing.

My problem is that @types/node package is installed in my app with the last version and that's what I don't need and it adds problems.

Unable to parse WKT string

The wkt POLYGON((-89.31029407 38.72017071,-89.30548017 38.72017071,-89.30421905 38.72418814,-89.31029407 38.72376013,-89.31029407 38.72017071))

causes parse to throw the following error:

_string-trim.js:25 Uncaught RangeError: Maximum call stack size exceeded
    at RegExp.exec (<anonymous>)
    at RegExp.[Symbol.replace] (<anonymous>)
    at String.replace (<anonymous>)
    at ./node_modules/@babel/polyfill/node_modules/core-js/modules/_string-trim.js.exporter.trim (_string-trim.js:25)
    at parseFloat (_parse-float.js:5)
    at WktParser../node_modules/wkx/lib/wktparser.js.WktParser.matchCoordinate (wktparser.js:103)
    at WktParser../node_modules/wkx/lib/wktparser.js.WktParser.matchCoordinates (wktparser.js:112)
    at Function../node_modules/wkx/lib/polygon.js.Polygon._parseWkt (polygon.js:56)
    at Function../node_modules/wkx/lib/geometry.js.Geometry._parseWkt (geometry.js:61)
    at Function../node_modules/wkx/lib/geometry.js.Geometry.parse (geometry.js:26)

via:

wkx.Geometry.parse('POLYGON((-89.31029407 38.72017071,-89.30548017 38.72017071,-89.30421905 38.72418814,-89.31029407 38.72376013,-89.31029407 38.72017071))')

Can you let me know what I'm doing wrong?

Error with WKB parsing

Hi,
I'm trying to convert geoJson to wkb. It doesn't really work (or I don't know what to do with it...).
I do the following:

Get GeoJson.
Parse it into geometry.
Do wkbBuffer = geometry.toWkb();
try to parse the buffer - wkx.Geometry.parse(wkbBuffer);
It gives me an error:

jQuery.Deferred exception: Trying to access beyond buffer length RangeError: Trying to access beyond buffer length

Here's the plnkr: https://plnkr.co/edit/4EyqlbUPDyS81sTHMWWt?p=preview

Can't parse subarray of buffer

MySQL stores geometry with a 4 byte SRID prefix before the WKB (doc). I guess the format is different than PostGIS's EWKB though, as parsing the whole field doesn't produce valid results. Skipping the first for bytes works though, like this:

GeometryParser.parse(Buffer.from(buffer.subarray(4)))

but using just the output from subarray:

GeometryParser.parse(buffer.subarray(4))

produces this error: Uncaught Error: first argument must be a string or Buffer. According to Node, subarray() just creates a new buffer pointing to a subset of the same data. Would be nice to not have to duplicate the data in order to parse it. Fairly minor, I admit.

Some formats don't parse

As far as I can tell, this format is acceptable WKT:

MULTIPOINT((10 40), (40 30), (20 20), (30 10))

However, this doesn't work

> var geometry = wkx.Geometry.parse("MULTIPOINT((10 40), (40 30), (20 20), (30 10))");
Error: Expected coordinate pair
    at WktParser.matchCoordinate (/Users/sacmac/node_modules/wkx/lib/wktparser.js:76:15)
    at WktParser.matchCoordinates (/Users/sacmac/node_modules/wkx/lib/wktparser.js:85:31)
    at Function.MultiPoint._parseWkt (/Users/sacmac/node_modules/wkx/lib/multipoint.js:27:59)
    at Function.Geometry._parseWkt (/Users/sacmac/node_modules/wkx/lib/geometry.js:58:27)
    at Function.Geometry.parse (/Users/sacmac/node_modules/wkx/lib/geometry.js:24:25)
    at repl:1:29
    at REPLServer.defaultEval (repl.js:248:27)
    at bound (domain.js:280:14)
    at REPLServer.runBound [as eval] (domain.js:293:12)
    at REPLServer.<anonymous> (repl.js:412:12)

But this does, without the parentheses surrounding each point pair

> var geometry = wkx.Geometry.parse("MULTIPOINT(10 40, 40 30, 20 20, 30 10)");
undefined

Any chance you could allow parsing of formats like MULTIPOINT((10 40), (40 30), (20 20), (30 10)) ?

What is the expected Format for WKT strings?

Spatial Reference provides multiple WKT string formats but none of those are accepted by wkx.

E.g. the provided OGC WKT string for epsg projection 4326 - wgs 84 is the following:

GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.01745329251994328,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]]

This string is not accepted by wkx. What is the expected WKT format?

Default GeoJSON SRID

First, thanks so much for putting this module together, it's been very helpful!

I wonder if it would make sense to have the srid property of a geometry default to 4326 when no crs property is provided. This is the assumed SRID in the GeoJSON spec, and it looks like in the latest spec they have done away with the crs property entirely.

Cannot parse values that contain numbers in scientific notation

I have a PostGIS query:

SELECT ST_AsText(sections.shape) AS shape
FROM sections

When I run this query, I get results that sometimes look like this:

LINESTRING Z (-0.000687 42.755489 0,8.6e-05 42.75571 0,0.00073 42.756561 0)

Given this shape, following code will fail with Expected coordinates error message.

import { Geometry, LineString } from 'wkx';
const lineString = Geometry.parse(shape) as LineString;

This happens because WKT parses uses regexp that doesn't account for numbers in scientific notation

convert FeatureCollection

Hi, Very nice module.
does it make sense to convert FeatureCollection to twkb. I mean instead of gzip, compress whole geojson file to twkb.
If yes, how can it be done both ways?

wkb to GeoJSON

I store coordinates in wkb format (data get from PostGIS DB, geometry Field the_geom geometry(Geometry,4326)).
Example http://spatialhast.github.io/index.wkb.to.geojson.html. In ' var the_geom' - wkb data of polygonal geometry. In geometryType of wkx.js I receive wrong number of geometry type (536870915). In 'var geojson' - example of GeoJSON data which need receive wrom wkb data.
Possible convert wkb in GeoJSON data (to display on map via Leaflet.js)?