Giter VIP home page Giter VIP logo

node-gdal's Introduction

node-gdal

NPM version Installs Build

Read and write raster and vector geospatial datasets straight from Node.js with this native GDAL binding. GDAL 2.4.4 (GEOS 3.4.2, PROJ 4.8.0) comes bundled, so node-gdal will work straight out of the box. To get started, browse the API Documentation or examples.

$ npm install gdal --save

To link against shared libgdal, install using:

# requires libgdal-dev (debian: sudo apt-get install libgdal-dev)
$ npm install gdal --build-from-source --shared_gdal

Sample Usage

Raster

var gdal = require("gdal");
var dataset = gdal.open("sample.tif");

console.log("number of bands: " + dataset.bands.count());
console.log("width: " + dataset.rasterSize.x);
console.log("height: " + dataset.rasterSize.y);
console.log("geotransform: " + dataset.geoTransform);
console.log("srs: " + (dataset.srs ? dataset.srs.toWKT() : 'null'));

Vector

var gdal = require("gdal");
var dataset = gdal.open("sample.shp");
var layer = dataset.layers.get(0);

console.log("number of features: " + layer.features.count());
console.log("fields: " + layer.fields.getNames());
console.log("extent: " + JSON.stringify(layer.extent));
console.log("srs: " + (layer.srs ? layer.srs.toWKT() : 'null'));

Notes

  • This binding is not async, so it will block node's event loop. Be very careful (or avoid) using it in server code. We recommended using tools like worker-farm to push expensive operations to a seperate process.

Bundled Drivers

AAIGrid, ACE2, ADRG, AIG, AVCBin, AVCE00, AeronavFAA, AirSAR, BLX, BMP, BNA, BT, BYN, CEOS, COASP, COSAR, CPG, CSV, CTG, CTable2, DERIVED, DGN, DIMAP, DIPEx, DOQ1, DOQ2, DTED, DXF, E00GRID, ECRGTOC, EDIGEO, EHdr, EIR, ELAS, ENVI, ERS, ESAT, ESRI Shapefile, ESRIJSON, FAST, FIT, FujiBAS, GFF, GML, GPSBabel, GPSTrackMaker, GPX, GRASSASCIIGrid, GRIB, GS7BG, GSAG, GSBG, GSC, GTX, GTiff, GenBin, GeoJSONSeq, GeoJSON, GeoRSS, Geoconcept, HF2, HFA, HTF, IDA, ILWIS, INGR, IRIS, ISCE, ISIS2, ISIS3, Idrisi, JAXAPALSAR, JDEM, JPEG, KMLSUPEROVERLAY, KML, KRO, L1B, LAN, LCP, LOSLAS, Leveller, MAP, MEM, MFF2, MFF, MVT, MapInfo File, Memory, NDF, NGSGEOID, NITF, NTv1, NTv2, NULL, NWT_GRC, NWT_GRD, OGR_GMT, OGR_PDS, OGR_SDTS, OGR_VRT, OpenAir, OpenFileGDB, PAux, PCIDSK, PDS4, PDS, PGDUMP, PNG, PNM, REC, RMF, ROI_PAC, RPFTOC, RRASTER, RS2, RST, R, S57, SAGA, SAR_CEOS, SDTS, SEGUKOOA, SEGY, SGI, SIGDEM, SNODAS, SRP, SRTMHGT, SUA, SVG, SXF, TIL, TSX, Terragen, TopoJSON, UK .NTF, USGSDEM, VICAR, VRT, WAsP, XPM, XPlane, XYZ, ZMap

Contributors

This binding is a collaboration between Natural Atlas and Mapbox. Its contributors are Brandon Reavis, Brian Reavis, Dane Springmeyer, Zac McCormick, and others.

Before submitting pull requests, please update the tests and make sure they all pass.

$ make test # test against bundled gdal
$ make test-shared # test against shared gdal

Alternate Fork

For an alternate fork that's more bleeding-edge at the moment, check out gdal-next and evaluate which project works best for your needs. The module-alias package can be useful if gdal is deep in your dependency tree and you don't want to update references everywhere.

License

Copyright © 2015–2021 Natural Atlas, Inc. & Contributors

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

node-gdal's People

Contributors

algunenano avatar brandonreavis avatar brianreavis avatar bryanburgers avatar dtudury avatar eduardogspereira avatar jdesboeufs avatar mapsam avatar mojodna avatar rclark avatar sanjay900 avatar springmeyer avatar tomhughes avatar tschaub avatar wilhelmberg avatar yhahn avatar zhm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

node-gdal's Issues

Expose gdal.version

We should expose the GDAL version to Javascript. This would be really useful for debugging. It would also allow people to do version comparisons too.

console.dir(gdal.version)
// "1.11.0"

Homebrew Errors

It looks like the brew install of gdal is being a little janky at times on Travis. cc: @yhahn

Unlinking /usr/local/Cellar/gdal/1.11.0... 173 symlinks removed
Warning: gdal-1.11.0 already installed, it's just not linked
$ if [[ ${TARGET} == 'SHARED' ]]; then npm install --build-from-source --shared_gdal; fi;
- 
> [email protected] install /Users/travis/build/naturalatlas/node-gdal
> node-pre-gyp install --fallback-to-build
-mand not found
gyp: Call to 'gdal-config --libs' returned exit status 127. while trying to load binding.gyp

https://travis-ci.org/naturalatlas/node-gdal/jobs/30369120

Add band and dataset attribute: size

Raster bands and datasets should have a size attribute instead of the methods GetXSize(), GetYSize(), GetRasterXSize(), and GetRasterYSize().

Example usage:

band.size
> {x: 256, y: 256}

First discussed here: #10

AppVeyor pre-gyp publishing

Now that we have Windows builds completing on AppVeyor (#19)...

I think the final step on getting the preliminary first version(s) out the door (#36) is getting AppVeyor to publish the pre-gyp package when "[publish binary]" is in the commit message like on Travis. I'm assuming we can model this off node-mapnik's appveyor.yml.

I'd jump on this, but don't have the credentials for the bucket. cc: @springmeyer @BergWerkGIS @yhahn @rclark Can one of you guys tackle this whenever you're free?

Creating layer fields

Right now you can create fields like this:

var field = new gdal.FieldDefn("name", gdal.OFTString);
lyr.fields.add(field);

It would be nice to support cleaner ways like these:

lyr.fields.create("name", gdal.OFTString);
lyr.fields.create("name", "string");

To do this, the enum OGRFieldType should be mapped to strings instead of integers like we did with the GDALColorInterp enum. This would also make debugging easier.

Finally, it would be nice to be able to set a new layer's fields based on an object you intend to write to the layer. I am not sure of the best naming for this method, but something like:

lyr.fields.fromJSON({
   id: 0,
   county_name: "Park",
   state_abbr: "WY"
});

Thankfully, I don't see why this method couldn't be implemented in JS instead of C++.

gdal install error

Installing fine locally, but erroring in Travis.

> [email protected] install /home/travis/build/mapbox/mapnik-omnivore/node_modules/gdal
> node-gyp rebuild
gyp http GET http://nodejs.org/dist/v0.10.26/node-v0.10.26.tar.gz
gyp http 200 http://nodejs.org/dist/v0.10.26/node-v0.10.26.tar.gz
/bin/sh: 1: gdal-config: not found
gyp: Call to 'gdal-config --libs' returned exit status 127.
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/home/travis/.nvm/v0.10.26/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:337:16)
gyp ERR! stack     at ChildProcess.EventEmitter.emit (events.js:98:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:797:12)
gyp ERR! System Linux 2.6.32-042stab079.5
gyp ERR! command "node" "/home/travis/.nvm/v0.10.26/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/travis/build/mapbox/mapnik-omnivore/node_modules/gdal
gyp ERR! node -v v0.10.26
gyp ERR! node-gyp -v v0.12.2
gyp ERR! not ok

npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is most likely a problem with the gdal package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp rebuild
npm ERR! You can get their info via:
npm ERR!     npm owner ls gdal
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 2.6.32-042stab079.5
npm ERR! command "/home/travis/.nvm/v0.10.26/bin/node" "/home/travis/.nvm/v0.10.26/bin/npm" "install"
npm ERR! cwd /home/travis/build/mapbox/mapnik-omnivore
npm ERR! node -v v0.10.26
npm ERR! npm -v 1.4.3
npm ERR! code ELIFECYCLE

Use random file names in tests which create files

The tests are looking great, and its amazing to see no crashes when I tried this devilish trick:

node ./node_modules/.bin/_mocha & node ./node_modules/.bin/_mocha & node ./node_modules/.bin/_mocha

Basically running the above is a cheap way to simulate some concurrent access to files to see if any i/o failures present crashes. I can't trigger any but I do see:

․  1) Dataset setProjection() should set projection (WKT):
     Error: Error opening dataset
      at Context.<anonymous> (/Users/dane/projects/node-gdal/test/dataset.test.js:45:18)

Which appears to occur randomly because of the writing of data/dem_azimuth50_pa.tmp1.img. Ideally a random name could be used to keep this passing even if tests are run concurrently.

Pixel access

Currently node-gdal is unable to write and write pixel data to and from bands. I envision we will want to expose them with C++ like:

  • band.pixels.get(x, y) : value
  • band.pixels.set(x, y, value)
  • band.pixels.read(x, y, width, height, [type]) : NativeArray
    • x : integer
    • y : integer
    • width : integer
    • height : integer
    • type : GDALDataType
  • band.pixels.write(data, x, y, width, height, [type])

Block Access
I am not sure yet about the best way to expose efficient block access with the ReadBlock()/WriteBlock() methods.

Option 1: Interpret the arguments passed to pixels.read()/pixels.write(), and if they match up with the block boundaries, use the ReadBlock()/WriteBlock() method.
Option 2: Add separate methods, which don't require passing the block size/offset each time

  • band.pixels.writeBlock(data, x, y)
  • band.pixels.readBlock(x, y) : NativeArray

ndarray

Next is the issue of whether or not we automatically turn NativeArrays into ndarrays. This would add a dependency, but it would likely be what most users would want. If we decide to do this, it should be a simple JS wrapper around the C++ methods that return NativeArrays (or accept them as arguments).

If users decide to do it on their own it would look something like this:

var w = h = 128; 
var pixels = ndarray(band.pixels.read(0, 0, w, h), [w, h]);
/// ...
band.pixels.write(pixels.data, 0, 0, w, h);

Datasources being garbage collected when layer is still alive

An example script where a layer's datasource goes out of scope and may or may not be GC'ed

function test(){
    var ds = ogr.open("example.shp");
    return ds.getLayer(0);
}
var n = test().getFeatureCount(); //may or may not throw "Layer already destroyed" error depending on when node performs garbage collection

Although it's an uncommon way to write a GDAL script, we should fix this problem because it would be a nightmare for a JS programmer to debug. Your script could be running just fine until node decides to garbage collect, and then your layer gets destroyed and your script throws a "Layer already destroyed" error somewhere.

Adding a ds property to a Layer object should fix it. V8's AddImplicitReferences() (link) might fix it as well.

(Problem also applies to GDAL datasets/bands)

Consider targeting GDAL 2.0

http://trac.osgeo.org/gdal/wiki/rfc46_gdal_ogr_unification includes some forward thinking improvements and it might be worth targeting this version. This makes me wonder about whether node-ogr should be collapsed into node-gdal, and also about whether it might make sense to wait to gypify gdal because so much might be moving with the build system and using gdal's own build system would be less work.

Fix/silence "vc120.pdb" build warnings on Windows

In the AppVeyor logs, there are hundreds of these warnings:

warning LNK4099: PDB 'vc120.pdb' was not found with 'libgdal.lib(cpl_multiproc.obj)' or
at 'C:\projects\node-gdal\build\Release\vc120.pdb';
linking object as if no debug info
[C:\projects\node-gdal\build\gdal.vcxproj]

CoordinateTransformation error

Are you familiar with the following error?
CoordinateTransformation parameter already destroyed

Error is triggered by this line: https://github.com/mapbox/mapnik-omnivore/blob/raster/lib/datasourceProcessor.js#L332

It's an odd error because it works fine within my application, but throws in Travis when tests are running.

I can't find much on it in GDAL documentation or basic web search. I found it commented out in node-gdal here: https://github.com/naturalatlas/node-gdal/blob/master/src/gdal_common.hpp

Interested if you've run into this before or have any thoughts on what could be throwing it. Thanks!

Deprecate getDriverByName()

gdal.getDriverByName(name);
gdal.getDriver(index);

Change: Have getDriver determine the appropriate GDAL method internally based on the type of the argument. This simplifies usage to:

gdal.getDriver(name);
gdal.getDriver(index);

This should also be done on OGR.

Reduce build times on AppVeyor (optimize gyp files)

@rclark @brianreavis
Another thought on build times on AppVeyor (referencing #19 (comment)):

Probably optimizing the gyp files could be another approach.

At the moment there are a lot of gyp files, e.g. for the formats.
These result in 71 Visual Studio solutions and 71 libraries.

Maybe reducing those could speed up things.
I also think less gyp files would be easier to maintain.

Including a lot of files (sources, headers) is as easy as it gets.

What do you think?
Perhaps @springmeyer could chime in with his knowledge about gyp?

"equals" vs "equal" method

This is a really minor nitpick.

Feature.prototype.equal(Feature f)
Geometry.prototype.equals(Geometry g)

@brandonreavis Is this fine to standardize to one or the other? I'd lean toward "equal".

Prototype async function with error reporting

My sense is that async functions can wait: once sync bindings are in place with solid tests and some releases are out in the wild then selectively exposing some critical functions as async would be worthwhile.

But we should figure out now how we think async functions will work. In particular error reporting will be important to discuss. This came up in the comment stream in #13.

The questions:

  • Can CPLSetErrorHandler be set, used, and unset safely per async function: so is is safe to use while in the libuv threadpool?
  • Can CPLSetThreadLocalConfigOption be used similarly to allow GDAL config options to be modified per async function?
  • If the answers to the above issues are no, then what options do we have for catching and reporting meaningful errors when async functions hit GDAL errors - Is this a case where the C++ API instead of the C API should be used?

Namespace Binding Code

To not conflict with GDAL and other dependencies (if we're statically linking them).

libgdal_ogr_kml_frmt.lib(kmlnode.obj) : error LNK2005: "public: __thiscall Feature::Feature(void)" (??0Feature@@QAE@XZ) already defined in gdal_feature.obj [C:\projects\node-gdal\build\gdal.vcxproj]
     Creating library C:\projects\node-gdal\build\Release\gdal.lib and object C:\projects\node-gdal\build\Release\gdal.exp
C:\projects\node-gdal\build\Release\gdal.node : fatal error LNK1169: one or more multiply defined symbols found [C:\projects\node-gdal\build\gdal.vcxproj]

Definition 1: https://github.com/naturalatlas/node-gdal/blob/master/src/gdal_feature.hpp#L17
Definition 2: https://github.com/naturalatlas/node-gdal/blob/master/deps/libgdal/gdal/ogr/ogrsf_frmts/kml/kmlutility.h#L60

Segfaults when accessing closed datasets

I love the idea of solid GDAL bindings for Node. Having done this for Python in Rasterio, I'd like to warn you about a category of trouble for GDAL bindings: null dataset and band pointers. For example:

> gdal = require('..')
> ds = gdal.open('/Users/sean/code/rasterio/rasterio/tests/data/RGB.byte.tif', 0)
{}
> ds.getRasterXSize()
791
> ds.close()
undefined
> ds.getRasterXSize()
Segmentation fault: 11

Eliminating such gotchas like Rasterio has would make node-gdal quite popular and useful, I think.

Ability to build standalone binaries against GDAL SDK

Appveyor builds are limited to ~30 minutes, which is not enough time to compile both the internally bundled gdal and the bindings.

So, I'm going to take a stab at:

  • Adding a ./scripts/build_against_sdk.sh script in the same genre as https://github.com/mapnik/node-mapnik/blob/master/scripts/build_against_sdk.sh. It's not pretty but it allows an external libgdal and all headers to be downloaded on the fly.
  • I'll use https://github.com/mapnik/mapnik-packaging to post tarballs of GDAL deps and a libgdal shared library to s3 on demand. Given the Mapnik 2.3.x SDK includes this we could start by using that, but ideally we'd use a trimmed down version without Mapnik/boost/etc and only the minimal things needed for GDAL.
  • Will first get this working for unix, then move to windows

Simplification of API regarding collections

I tried to do an audit on the main GDAL/OGR features that represent arrays / collections. This is partially related to #10. Here are some thoughts on what we can and can't do with them.

Standardized Language

  • "delete", "remove", "unset" → "remove"
  • "add", "push", "create" → "add"
  • "length", "count" → "count"

Consideration: Potentially-Async Features

While GDAL is not particularly thread-safe at the moment, we shouldn't corner ourselves into assuming certain features will always have to be synchronous.

GDAL Wiki: Is the GDAL library thread-safe?

No. GDAL is not completely thread safe.

However for GDAL 1.3.0 much work has been done on making some common scenarios thread safe. In particular for the situation where many threads are reading from GDAL datasets at once should work as long as no two threads access the same GDALDataset object at the same time. However, in this scenario, no threads can be writing to GDAL while others are reading or chaos may ensue.

Also, while the GDAL core infrastructure is now thread-safe for this specific case, only a few drivers have been vetted to be thread safe. It is intended that work will continue on improving GDAL's thread safety in future versions.

For the features that are potentially async, we can't get too clever like we could if we assumed it was sync:

// syntactically-nice array-like objects won't work:
rasterband.overviews[index] // getter for getOverview
rasterband.overviews.length // getter for getOverviewCount

Proposals

RasterBand Collections (potentially-async)

gdal.Dataset.prototype.getRasterBand(i)
gdal.Dataset.prototype.getRasterCount()

This could be normalized to:

dataset.bands.count()
dataset.bands.get(index)
dataset.bands.forEach(callback)

Layer Collections (potentially-async)

ogr.Datasource.prototype.getLayerCount()
ogr.Datasource.prototype.getLayer(i)
ogr.Datasource.prototype.deleteLayer(i)
ogr.Datasource.prototype.createLayer(string)
ogr.Datasource.prototype.getLayerByName(i)

This could be normalized to:

datasource.layers.count()
datasource.layers.get(name|index)
datasource.layers.remove(name|index)
datasource.layers.add(layer)
datasource.layers.forEach(callback)

Feature Collections (potentially-async)

ogr.Layer.prototype.getFeatureCount()
ogr.Layer.prototype.getFeature(i)
ogr.Layer.prototype.getNextFeature()
ogr.Layer.prototype.setFeature(i)
ogr.Layer.prototype.createFeature()
ogr.Layer.prototype.deleteFeature(i)

This could be normalized to:

layer.features.count()
layer.features.get(index)
layer.features.remove(index)
layer.features.add(feature)
layer.features.forEach(callback)

Driver Collections

gdal/ogr.getDriver()
gdal/ogr.getDriverCount

This could be simplified to:

[gdal|ogr].drivers.count()
[gdal|ogr].drivers.get(name|index)

Field Collections

ogr.Feature.prototype.getFieldCount()
ogr.Feature.prototype.getFieldIndex()
ogr.Feature.prototype.getFieldAs[X]()
ogr.Feature.prototype.isFieldSet(i)
ogr.Feature.prototype.unsetField(i)
ogr.Feature.prototype.getField(i)
ogr.Feature.prototype.setField(i)
ogr.Feature.prototype.getFields()
ogr.Feature.prototype.setFields()

This could be simplified to:

feature.fields.count()
feature.fields.reset({})
feature.fields.set({})
feature.fields.set(key|index, value)
feature.fields.isset(key|index)
feature.fields.remove(key|index)
feature.fields.get(key|index, [type])
feature.fields.toJSON()

This removes the need for all of these:

- ogr.Feature.prototype.getFieldAsInteger()
- ogr.Feature.prototype.getFieldAsDouble()
- ogr.Feature.prototype.getFieldAsString()
- ogr.Feature.prototype.getFieldAsIntegerList()
- ogr.Feature.prototype.getFieldAsDoubleList()
- ogr.Feature.prototype.getFieldAsStringList()
- ogr.Feature.prototype.getFieldAsBinary()
- ogr.Feature.prototype.getFieldAsDateTime()

Field Definition Collections

ogr.FeatureDefn.prototype.getFieldCount()
ogr.FeatureDefn.prototype.getFieldDefn(i)
ogr.FeatureDefn.prototype.addFieldDefn()
ogr.FeatureDefn.prototype.deleteFieldDefn(i)
ogr.FeatureDefn.prototype.reorderFieldDefns()

This could be simplified to:

layer.def.fields.count()
layer.def.fields.add(field_defn)
layer.def.fields.remove(index)
layer.def.fields.get(index)

// "mapping" could be a map of keys to indexes, or indexes to indexes
layer.def.fields.reorder(mapping)

TL;DR:

  • Decide on consistent language to be used across the binding ("add", "remove", "count")
  • Have better encapsulation of functionality. Example: GDAL groups layer CRUD methods in with all the other DataSource methods that don't really have to do with layers. Given a data source is a representation of a collection of layers + some other things, I think it makes sense to have layers be its own thing in DataSource. I think it's cleaner & more intuitive.

Package GEOS

Used by many of the OGR geometry methods. If GEOS is not included, these methods will return null.

Geometry.prototype.isSimple()
Geometry.prototype.isValid()
Geometry.prototype.centroid()
Geometry.prototype.boundary()
Geometry.prototype.buffer()
// etc...

Cleaning up geometry objects

The geometry class could really use some cleaning.

Right now its like this: (very similar to python binding)

var pt = new ogr.Geometry(ogr.wkbPoint);
pt.setX(10);
pt.setY(20);

//scale it by 2
pt.setX(pt.getX() * 2)
pt.setY(pt.getY() * 2)

But it would be much more useful like this:

var pt = new ogr.Point(10, 20);

//scale it by 2
pt.x *= 2;
pt.y *= 2;

Return the units string from getLinearUnits()

Currently, node-gdal does not expose the units of a dataset's spatial reference system, only a numerical coefficient.

> var gdal = require('./lib/gdal.js')
undefined
> var ds = gdal.open('/Users/sean/code/rasterio/rasterio/tests/data/RGB.byte.tif')
undefined
> ds.srs.isProjected()
true
> ds.srs.getLinearUnits()
1

OSR's OGRSpatialReference::GetLinearUnits method returns a scale factor as a number and a pointer to the actual units string ("m" or "ft" or whatever). Should node-gdal's GetLinearUnits() return a pair of values like ['m', 1.0] (these are the method's defaults) or an object like {'units': 'm', 'coefficient': 1.0}?

SpatialReference import methods should be static

As discussed with @springmeyer, @sgillies, and @brianreavis, we should make all of spatial reference import methods be static methods that can be used to construct a new spatial reference.

//before change
var srs = new SpatialReference();
srs.importFromEPSG(4326);

//after change
var srs = SpatialReference.fromEPSG(4326);

Methods to change

setFromUserInput() -> fromUserInput()
importFromWKT() -> fromWKT()
importFromProj4() -> fromProj4()
importFromEPSG() -> fromEPSG()
importFromEPSGA() -> fromEPSGA()
importFromWMSAUTO() -> fromWMSAUTO()
importFromXML() -> fromXML()
importFromURN() -> fromURN()
importFromMICoordSys() -> fromMICoordSys()

Make isnan check portable

gdal_rasterband.cpp
..\src\gdal_rasterband.cpp(391): error C2039: 'isnan' : is not a member of 'std' [C:\projects\node-gdal\build\gdal.vcxproj]
..\src\gdal_rasterband.cpp(391): error C3861: 'isnan': identifier not found [C:\projects\node-gdal\build\gdal.vcxproj]

gdaltransform and distance capabilities

Does node-gdal currently have bindings for these?

If not, would be awesome to have this capability. Mainly interested in gdaltransform so that I can convert two points in the source's native pixelsize to mercator-friendly points. Documented here if you're interested.

After obtaining the converted points, I want to measure the distance between them using GDAL's OGRGeometry::Distance function. Is this the same as what is set up here?

/cc @brianreavis @brandonreavis

getStatistics() failing on VRT file

Calling band.getStatistics() on node-gdal/test/data/sample.vrt, and getting the following error:
Error: Error getting statistics

Though, the function is working fine for my .tiff sources.

Adding libpq.a support to bundled gdal gyp build

Have discussed with @brianreavis that is reasonable to compile your own libpq.a and statically link it to be able to support talking to postgres servers on various systems. At least this has worked for me for shipping Mapnik's postgis.input plugin without external shared library dependencies.

The things I've learned along the way:

  • Ubuntu has patched postgres so that the default unix socket is different. This means that users must connect via tcp by providing host=localhost and unix sockets don't work unless this patch is applied (which I recommend): Details at mapnik/mapnik-packaging#130.
  • I've not found a way (yet) to use postgres's main build scripts to build only libpq. So, instead I just build everything and accept failures for some parts of postgres that don't compile because libpq.a still builds fine. This is why I use the -i -k here. Figured this out: mapnik/mapnik-packaging@c3c5bb9
  • I've not looked into what it would take to gyp'ify just libpq. We could dodge this problem if we said only builds done with #19 would support postgres connections. But that would be limiting.

Geometry, SpatialReference export methods should follow toX() style

Related to #22. In Javascript, most objects have...

object.toString()
object.toJSON()

To be in line with this, we should change:

- SpatialReference.prototype.exportToWKT()
+ SpatialReference.prototype.toWKT()
- SpatialReference.prototype.exportToPrettyWKT()
+ SpatialReference.prototype.toPrettyWKT()
- SpatialReference.prototype.exportToProj4()
+ SpatialReference.prototype.toProj4()
- SpatialReference.prototype.exportToXML()
+ SpatialReference.prototype.toXML()

- Geometry.prototype.exportToKML()
+ Geometry.prototype.toKML()
- Geometry.prototype.exportToGML()
+ Geometry.prototype.toGML()
- Geometry.prototype.exportToJSON()
+ Geometry.prototype.toJSON()
- Geometry.prototype.exportToWKT()
+ Geometry.prototype.toWKT()

Shim FindFieldIndex for GDAL < 1.11.0

CXX(target) Release/obj.target/gdal/src/collections/layer_fields.o
../src/collections/layer_fields.cpp: In static member function ‘static v8::Handle<v8::Value> LayerFields::indexOf(const v8::Arguments&)’:
../src/collections/layer_fields.cpp:109:48: error: ‘class OGRLayer’ has no member named ‘FindFieldIndex’
../src/collections/layer_fields.cpp:110:1: warning: control reaches end of non-void function [-Wreturn-type]
make: *** [Release/obj.target/gdal/src/collections/layer_fields.o] Error 1
make: Leaving directory `/home/travis/build/naturalatlas/node-gdal/build'
gyp ERR! build error 

The only function that needs to be affected seems to be layer_fields.cpp:

Handle<Value> LayerFields::indexOf(const Arguments& args)

drivers.get() should return null when not found

Currently, this throws "Error retrieving driver":

gdal.drivers.get('awfag')

It'd be nicer if it returned null to prevent try/catch clutter. Checking to see if a driver existed could be as simple as:

var exists = !!gdal.drivers.get('awfawf');

Interface style: attributes vs functions?

Typically we're asking for both the height and width of a raster dataset, not just one or the other. Shall we aim for a size attribute that gives us a width, height pair instead of exposing GDAL's GetXSize() and GetYSize() methods? Bonus: this dodges the confusion I always feel when I try to remember whether the name is GetXSize() (the band method) or GetRasterXSize() (the dataset method).

Related: my comment on designing the right interface now as a future proofing measure: #3 (comment).

Set GDAL_DATA runtime config option

By calling srs.importFromEPSG(4326) I just discovered a problem that we will have if we are trying to make node-gdal work right out of the box.

At runtime, GDAL looks for csv files that map EPSG codes to projections. It looks in the folder specified by the environment variable GDAL_DATA which is normally set to "/usr/share/gdal/data"

"deps/libgdal/gdal/data" is the folder with the data that it needs.

Thankfully we should be able package this folder along with the pre-built library and then set the GDAL_DATA path when the module is loaded like:

CPLSetConfigOption("GDAL_DATA", "path/to/gdal/data");

Details here: http://trac.osgeo.org/gdal/wiki/ConfigOptions

Add layer, dataset, and geometry attribute: srs

A srs attribute should replace the following methods:

  • dataset.getProjectionRef()
  • layer.getSpatialRef()
  • geometry.getSpatialReference()
  • geometry.assignSpatialReference()

dataset.getProjectionRef() returns a string, but I think we should turn that into a SpatialReference object so it behaves like everything else.

First discussed here: #10

Setting geometry type of new layers

Right now to create a layer you need to pass the wkbGeometryType like:

var lyr = ds.layers.create('layername', srs, gdal.wkbPoint);

Since we have removed the need to use the wkbGeometryType enum everywhere else in the binding, it would be nice to remove it here too. We could pass the geometry constructor like:

var lyr = ds.layers.create('layername', srs, gdal.Point);

Raster Band type info

Wondering if you plan to include more band info in order to answer questions like:

  • does the file have 4 bands that represent rgba pixels
  • does it just have one band that represents infrared

/cc @brandonreavis @brianreavis

Change getConfigOption/setConfigOption to match API design

For the rest of the binding we've tried to encapsulate like functionality. The only thing that feels a little foreign still are config options:

gdal.getConfigOption(key)
gdal.setConfigOption(key, value)

This could change to:

gdal.config.get(key)
gdal.config.set(key, value)

nodata returning as NaN

I am currently obtaining the nodata property using band.getNoDataValue()
and it consistently returns NaN

Update documentation to instruct user to install gdal separately

I added gdal as a dependency in my package.json file. Running npm install, I got the following errors:

/bin/sh: 1: gdal-config: not found
gyp: Call to 'gdal-config --libs' returned exit status 127. while trying to load binding.gyp
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:340:16)
gyp ERR! stack     at ChildProcess.EventEmitter.emit (events.js:98:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:807:12)
gyp ERR! System Linux 3.13.0-30-generic
gyp ERR! command "node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/justin/prog/jiminy/node_modules/gdal
gyp ERR! node -v v0.10.28
gyp ERR! node-gyp -v v0.13.0
gyp ERR! not ok 
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1

After looking through the issues, I ran

sudo apt-get -q update
sudo apt-get -q install libgdal1-dev

as brianreavis suggested in Issue #25. This solved the issue, and npm was able to install gdal successfully.

The README currently reads: "GDAL 1.11.0 (GEOS 3.4.2, Proj.4 4.8.0) comes bundled, so node-gdal will work straight out of the box." Based on my experience, this doesn't seem true...if it is the case that this package no longer comes bundled with GDAL, then the README should be updated.

If this is indeed the case, I am happy to update the README. Can a maintainer please confirm?

Old node-pre-gyp bundled in new release

Getting the following invalid error for node-pre-gyp from within gdal...
screen shot 2014-07-29 at 5 23 57 pm

I took a look in the gdal tarball to see what version of node-pre-gyp was actually built, and looks like version 0.5.13 was bundled into the new release, instead of ~0.5.17

Mind running another release with the newer version?

/cc @brianreavis @brandonreavis

Compile in debug mode if [publish binary] isn't present

It's nearly impossible to get to the source of a segfault if it happens on Travis/AppVeyor but can't be repro'd locally. To do this well, we should compile node-gdal (and gdal, cc: @BergWerkGIS) in debug mode and attach a segfault handler:

$ npm install https://github.com/ddopson/node-segfault-handler/archive/master/node-segfault-handler.tar.gz
// lib/gdal.js (at the top)
try {
    var SegfaultHandler = require('segfault-handler');
    SegfaultHandler.registerHandler();
} catch(e) {}

cc: @brandonreavis

Windows: build with "build-gdal"

@springmeyer @brianreavis
I've again been trying to build node-gdal with the libs that are created by build-gdal.

I'm hitting the following error:

 Creating library C:\projects\node-gdal\build\Release\gdal.lib and object C:\projects\node-gdal\build\Release\gdal.exp
gdal.lib(cpl_odbc.obj) : error LNK2001: unresolved external symbol _SQLRemoveDriver@12 [C:\projects\node-gdal\build\gdal.vcxproj] gdal.lib(cpl_odbc.obj) : error LNK2001: unresolved external symbol _SQLInstallerError@20 [C:\projects\node-gdal\build\gdal.vcxpro j]
gdal.lib(cpl_odbc.obj) : error LNK2001: unresolved external symbol _SQLInstallDriverEx@28 [C:\projects\node-gdal\build\gdal.vcxpr oj]
libpq.lib(fe-connect.obj) : error LNK2001: unresolved external symbol __imp__FreeCredentialsHandle@4 [C:\projects\node-gdal\build \gdal.vcxproj]
libpq.lib(fe-connect.obj) : error LNK2001: unresolved external symbol __imp__DeleteSecurityContext@4 [C:\projects\node-gdal\build \gdal.vcxproj]
libpq.lib(fe-auth.obj) : error LNK2001: unresolved external symbol _AcquireCredentialsHandleA@36 [C:\projects\node-gdal\build\gda l.vcxproj]
libpq.lib(fe-auth.obj) : error LNK2001: unresolved external symbol _InitializeSecurityContextA@48 [C:\projects\node-gdal\build\gd al.vcxproj]
libpq.lib(fe-auth.obj) : error LNK2001: unresolved external symbol _FreeContextBuffer@4 [C:\projects\node-gdal\build\gdal.vcxproj ]
C:\projects\node-gdal\build\Release\gdal.node : fatal error LNK1120: 8 unresolved externals [C:\projects\node-gdal\build\gdal.vcx proj]

This suggests to add odbc32.lib odbccp32.lib as additional dependencies.
But, as far as I can tell, these are already added, see here and here.

Any idea, what's missing?

BTW:
I cannot add labels for issues.
What do you think about a 'Windows' label?

Build error on Windows x64

When building on Windows with 64bit I get the following error when gdal.lib is about to be created.
All the dependent libs built successfully and I've already uploaded them to S3.

Any ideas?

On AppVeyor, line 557

Local:
node-gdal-windows-x64

Caching wrapped objects

There never should be multiple JS objects wrapping the same OGRLayer, OGRDatasource, OGRSpatialReference, GDALDataset, or GDALRasterBand.

// --- example ---
var ds = ogr.open('test.shp');
var a = ds.getLayer(1);
var b = ds.getLayer(1);
//currently, a and b are different objects wrapping the same layer

This will be a problem when we want to properly destroy each wrapped layer when a datasource is destroyed (outlined here: #5). Without caching the wrapped objects, we would have to maintain a list of JS objects for each OGRLayer pointer and properly set each pointer to null when a datasource is destroyed.

The solution using persistent handles and a map is demonstrated here: http://create.tpsitulsa.com/wiki/V8/Persistent_Handles

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.