Giter VIP home page Giter VIP logo

geoarrays.jl's Introduction

Hi there 👋

  • 🔭 I'm a GeoData scientist at Deltares and an external PhD candidate at the 3d geoinfo group at the TU Delft working on elevation modelling.
  • 🌱 I'm the author of several open-source packages in Julia related to geospatial processing, such as GeoArrays, GeoDataFrames and LazIO.
  • ⚡ In my free time I tinker with open data and open-source software, but I also read, climb, boulder, mountaineer and ride a motorbike.
  • 💬 You can get in touch with me by email at [email protected], Twitter (@3vetion), and LinkedIn.

geoarrays.jl's People

Contributors

alex-s-gardner avatar crghilardi avatar dependabot[bot] avatar dilumaluthge avatar dsantra92 avatar evetion avatar github-actions[bot] avatar juliatagbot avatar krostir avatar maarten-keijzer avatar mauro3 avatar visr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

geoarrays.jl's Issues

coords for 3D

Should this work:

julia> GeoArrays.coords(GeoArrays.GeoArray(rand(2,3,4)), [1,1,1])
ERROR: DimensionMismatch("expected input array of length 2, got length 3")
Stacktrace:
 [1] dimension_mismatch_fail(::Type{T} where T, ::Array{Int64,1}) at /home/mauro/.julia/packages/StaticArrays/1g9bq/src/convert.jl:18
 [2] convert at /home/mauro/.julia/packages/StaticArrays/1g9bq/src/convert.jl:23 [inlined]
 [3] StaticArray at /home/mauro/.julia/packages/StaticArrays/1g9bq/src/convert.jl:7 [inlined]
 [4] coords(::GeoArray{Float64}, ::Array{Int64,1}) at /home/mauro/.julia/dev/GeoArrays/src/geoarray.jl:52
 [5] top-level scope at REPL[225]:1

?

In GeoArrays.read, ERROR: UndefVarError: allregister not defined

I'm trying to read in a file with GeoArrays.read and I'm getting an error that allregister isn't defined.

ERROR: UndefVarError: allregister not defined
Stacktrace:
 [1] read(::String) at /root/.julia/packages/GeoArrays/SV5MC/src/io.jl:13
 [2] top-level scope at untitled-97594d953fa6b8541389f5f1dda0f0a5:4

It has to do with GDAL.allregister() in GeoArrays.Read()

I'm using Julia 1.4-rc2:

julia> versioninfo()
Julia Version 1.4.0-rc2.0
Commit b99ed72c95 (2020-02-24 16:51 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) E-2176M  CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 2
  JULIA_EDITOR = atom  -a

I'm wondering if this issue could have something to do with that? It also may very well be an issue in GDAL.jl itself, but I figured posting an issue here was a good place to start.

Thanks for any help!

Plots example from README is broken. coords method conflict

The plotting example from the README.md appears to be broken: https://github.com/evetion/GeoArrays.jl#plotting

julia> using Plots
julia> fn = download("https://github.com/yeesian/ArchGDALDatasets/blob/master/pyrasterio/RGB.byte.tif?raw=true")
julia> ga = GeoArrays.read(fn)
julia> plot(ga)
ERROR: UndefVarError: coords not defined
Stacktrace:
 [1] macro expansion
   @ ~/.julia/packages/GeoArrays/oqARi/src/plot.jl:10 [inlined]
 [2] apply_recipe(plotattributes::AbstractDict{Symbol, Any}, ga::GeoArray)
   @ GeoArrays ~/.julia/packages/RecipesBase/qpxEX/src/RecipesBase.jl:289
 [3] _process_userrecipes!(plt::Any, plotattributes::Any, args::Any)
   @ RecipesPipeline ~/.julia/packages/RecipesPipeline/7ijBv/src/user_recipe.jl:36
 [4] recipe_pipeline!(plt::Any, plotattributes::Any, args::Any)
   @ RecipesPipeline ~/.julia/packages/RecipesPipeline/7ijBv/src/RecipesPipeline.jl:70
 [5] _plot!(plt::Plots.Plot, plotattributes::Any, args::Any)
   @ Plots ~/.julia/packages/Plots/9C6z9/src/plot.jl:208
 [6] plot(args::Any; kw::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
   @ Plots ~/.julia/packages/Plots/9C6z9/src/plot.jl:91
 [7] top-level scope

It appears that there may be a conflict with the coords method

julia> coords
WARNING: both Plots and GeoArrays export "coords"; uses of it in module Main must be qualified
ERROR: UndefVarError: coords not defined

Julia version: 1.7.1
GeoArrays version: 0.6.0
Plots version: 1.25.6

Support `stepsize` in `getindex`

As the subsetting with ga[1:2:100, 1:2:100] will yield incorrect scaling, the linear transformation should be adjusted accordingly.

return Array{T} instead of Array{Union{T, Missing}} when applicable

Hi, quick Q,
in DataFrames there's a lot of effort going into detecting whether you've got missing values, and if you don't, just return Array{T} rather than Array{Union{T, Missing}}. Do you have similar plans here? Or is it so rare that rasters don't have missing values that it isn't worth it?

GeoArrays produces positively y spaced rasters by default

I am writing tiff using the following scripts:

begin
    ga = GeoArray(rand(360, 150))
    bbox!(ga, (min_x=-180.0, min_y=-60.0, max_x=180.0, max_y=90.0))
    epsg!(ga, 4326)  # in WGS84
    GeoArrays.write!("test.tif", ga)    
end

When reading in R, the crsTransform is error. The latitude range is changed to [-59, 91].

library(raster)
b <- brick("test.tif")
Warning message:
In .rasterFromGDAL(x, band = band, objecttype, ...) :
  data seems flipped. Consider using: flip(x, direction='y')

b
class      : RasterBrick 
dimensions : 150, 360, 54000, 1  (nrow, ncol, ncell, nlayers)
resolution : 1, 1  (x, y)
extent     : -180, 180, -59, 91  (xmin, xmax, ymin, ymax)
crs        : +proj=longlat +datum=WGS84 +no_defs 
source     : N:/Research/GEE_repos/GEE-latest/test.tif 
names      : layer 

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Suggest renaming`interpolate!` to `fill!`

My intuition for interpolate! is a function that can retrieve geoarray values between exact indices. For this reason I would suggest renaming interpolate! to fill!.

OutOfMemoryError using interpolate!

I am getting an OutOfMemoryError() when using interpolate!

I have read through this old issue and read through the GeoStats documentation without success.

I have tried so far:

  1. removing CopyMapper() parameter
  2. replacing CopyMapper() with SimpleMapper()

I have also confirmed the test_interpolate.jl works fine, so I am not sure if there is a limit to how sparse a layer is before it just doesn't work.

The example below reproduces the issue but I also tested it using a few other smaller datasets with the same results.

Gelukkig Nieuwjaar!

#from PointCloudRasterizers README
julia> using PointCloudRasterizers

julia> using LazIO

julia> using GeoArrays

julia> using Statistics

julia> lazfn = joinpath(dirname(pathof(LazIO)), "..", "test/libLAS_1.2.laz")
"/home/crgxcf/.julia/packages/LazIO/IC3Cp/src/../test/libLAS_1.2.laz"

julia> pointcloud = LazIO.open(lazfn)
LazDataset of /home/crgxcf/.julia/packages/LazIO/IC3Cp/src/../test/libLAS_1.2.laz with 497536 points.

julia> cellsizes = (1.,1.)
(1.0, 1.0)

julia> raster_index = index(pointcloud, cellsizes)
[ Info: Indexing into a grid of (5001, 5001)
Building raster index..100%|█████████████████████████████████████████████████████████████████████| Time: 0:00:00
PointCloudRasterizers.PointCloudIndex(LazDataset of /home/crgxcf/.julia/packages/LazIO/IC3Cp/src/../test/libLAS_1.2.laz with 497536 points.
, 5001x5001x1 Array{Int64,3} with AffineMap([1.0 0.0; 0.0 1.0], [1.44e6, 375000.03]) and WKT , [134, 125, 115, 5106, 5096, 5087, 5077, 10069, 10059, 10049    39976, 44988, 44998, 14992, 14982, 9970, 9959, 4948, 4937, 4927], Tuple{Int64,Int64}[(0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0)    (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0), (0, 0)], [1 5002  25000000 25005001; 2 5003  25000001 25005002;  ; 5000 10001  25004999 25010000; 5001 10002  25005000 25010001])

julia> raster = reduce(raster_index, field=:Z, reducer=median)
Reducing points...100%|██████████████████████████████████████████████████████████████████████████| Time: 0:00:02
5001×5001×1 GeoArray{Union{Missing, Float64}}:
[:, :, 1] =
 missing  missing  missing  missing  missing  missing    missing  missing  missing  missing  missing  missing
 missing  missing  missing  missing  missing  missing     missing  missing  missing  missing  missing  missing
 missing  missing  missing  missing  missing  missing     missing  missing  missing  missing  missing  missing
                                                                                                         
 missing  missing  missing  missing  missing  missing     missing  missing  missing  missing  missing  missing
 missing  missing  missing  missing  missing  missing     missing  missing  missing  missing  missing  missing
 missing  missing  missing  missing  missing  missing    missing  missing  missing  missing  missing  missing

#get a sense of how sparse the layer is
julia> count(ismissing,raster)
24513458

julia> count(!ismissing,raster)
496543

julia> using GeoStats

julia> GeoArrays.interpolate!(raster, Kriging())
ERROR: OutOfMemoryError()
Stacktrace:
 [1] Array at ./boot.jl:406 [inlined]
 [2] lhs(::OrdinaryKriging{GaussianVariogram{Float64,Distances.Euclidean}}, ::Array{Float64,2}) at /home/crgxcf/.julia/packages/KrigingEstimators/3HifF/src/estimators.jl:96
 [3] fit(::OrdinaryKriging{GaussianVariogram{Float64,Distances.Euclidean}}, ::Array{Float64,2}, ::Array{Union{Missing, Float64},1}) at /home/crgxcf/.julia/packages/KrigingEstimators/3HifF/src/estimators.jl:69
 [4] solve_exact(::EstimationProblem{RegularGridData{Float64,2},RegularGrid{Float64,2},CopyMapper{Nothing,Nothing}}, ::Symbol, ::Dict{Symbol,NamedTuple}) at /home/crgxcf/.julia/packages/KrigingEstimators/3HifF/src/solvers/kriging.jl:255
 [5] solve(::EstimationProblem{RegularGridData{Float64,2},RegularGrid{Float64,2},CopyMapper{Nothing,Nothing}}, ::Kriging) at /home/crgxcf/.julia/packages/KrigingEstimators/3HifF/src/solvers/kriging.jl:153
 [6] interpolate!(::GeoArray{Union{Missing, Float64}}, ::Kriging, ::Int64, ::Symbol) at /home/crgxcf/.julia/dev/GeoArrays/src/interpolate.jl:17
 [7] interpolate!(::GeoArray{Union{Missing, Float64}}, ::Kriging) at /home/crgxcf/.julia/dev/GeoArrays/src/interpolate.jl:7
 [8] top-level scope at REPL[11]:1

Support metadata

And especially also things like name, a scaling_factor or offset on each band.

GeoArrays Plot example not producing the suggested output

I am currently investigating how to plot 2D heatmap as function of 2D x and y coordinate field within the Plots.jl framework (see here) and came across your last example from the README. There, you plot a 2D map in apparently rotated coordinate. Trying to reproduce the code snippet

# Plot a GeoArray
julia> using Plots
julia> fn = download("https://github.com/yeesian/ArchGDALDatasets/blob/master/pyrasterio/RGB.byte.tif?raw=true")
julia> ga = GeoArrays.read(fn)
julia> plot(ga)

# or plot a band other than the first one
julia> plot(ga, band=2)

I could not manage to get the same output as depicted on the GeoArrays README. The figure I obtained was different (low-res) and not rotated.

ranges produces wrong length of vector

For this example ranges produces wrong length of y vector... can't quite figure out why

url = "https://s3-eu-west-1.amazonaws.com/download.agisoft.com/gtg/us_nga_egm2008_1.tif"
HTTP.download(url, abspath("junk.tif"))

julia> ga = GeoArrays.read(abspath("junk.tif"); masked=false)
21601x10801x1 ArchGDAL.RasterDataset{Float32, ArchGDAL.IDataset} with AffineMap([0.016666666666666666 0.0; 0.0 -0.016666666666666666], [-180.00833333333333, 90.00833333333334]) and CRS GEOGCRS["WGS 84",ENSEMBLE["World Geodetic System 1984 ensemble",MEMBER["World Geodetic System 1984 (Transit)"],MEMBER["World Geodetic System 1984 (G730)"],MEMBER["World Geodetic System 1984 (G873)"],MEMBER["World Geodetic System 1984 (G1150)"],MEMBER["World Geodetic System 1984 (G1674)"],MEMBER["World Geodetic System 1984 (G1762)"],MEMBER["World Geodetic System 1984 (G2139)"],ELLIPSOID["WGS 84",6378137,298.257223563,LENGTHUNIT["metre",1]],ENSEMBLEACCURACY[2.0]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433]],CS[ellipsoidal,3],AXIS["geodetic latitude (Lat)",north,ORDER[1],ANGLEUNIT["degree",0.0174532925199433]],AXIS["geodetic longitude (Lon)",east,ORDER[2],ANGLEUNIT["degree",0.0174532925199433]],AXIS["ellipsoidal height (h)",up,ORDER[3],LENGTHUNIT["metre",1]],USAGE[SCOPE["Geodesy. Navigation and positioning using GPS satellite system."],AREA[...],BBOX[-90,-180,90,180]],ID["EPSG",4979]]

julia> x0, y0 = GeoArrays.ranges(ga)
(-180.0:0.016666666666666666:180.0, 90.0:-0.016666666666666666:-89.98333333333332)

julia> length(x0)
21601

julia> length(y0)
10800

julia> size(ga)
(21601, 10801, 1)

Interpolation methods on GeoArray types

I have been working on writing some existing LiDAR ground filtering methods in Julia. The one I am working on now uses nearest neighbor interpolation to fill in the grid after selecting a minimum.

After making a GeoArray with PointCloudRasterizers.jl I have tried different functions in GridInterpolations.jl, Interpolations.jl, and NearestNeighbor.jl but typically run into method errors like ERROR: MethodError: no method matching interpolate(::GeoArray{Union{Missing, Float64}})

Before going any further, I wanted to open an issue for visibility. Not sure if I am missing something where GeoArray should work since it already subtypes AbstractArray or if some glue code needs to be written to make everything work together.

return indices as CartesianIndex

If indices were returned as CartesianIndex then they can be used to directly index info a geoarray when dealing with Vector of points

Make dims a type parameter

Why not have the dimensions as a type parameter? That is what AbstractArrays usually have.

mutable struct GeoArray{T<:Union{Real, Union{Missing, Real}}, N} <: AbstractArray{T, N}
    A::AbstractArray{T, N}
    f::AffineMap
    crs::AbstractString
end

(and why is it a "mutable" struct?)

Subsetting GeoArray

When a GeoArray is subset by index, only the array is returned. However, a subset of a GeoArray - either spatial or band - is a GeoArray.

Here is an example.

fn = download("https://bitbucket.org/earthobservation/julia-remote-sensing/raw/ccea113bf43ba71d4d1c6f02e060fd2923c982b2/data/KR_S2B_MSIL2A_20210614_20.tif")
geoarray = GeoArrays.read(fn)

ga_band = geoarray[:,:,7] # Matrix{UInt16}

The subset is an array.

I have written a quick and dirty - I am a Julia beginner - function to make the subset as a GeoArray.

function ga_subset(ga, bands, ind)
	afm = ga.f
	r_min = ind[1]
	r_max = ind[3]
	c_min = ind[2]
	c_max = ind[4]
	sub_afm_l = afm.linear
	sub_a = ga.A[r_min:r_max,c_min:c_max,bands]
	sub_afm_t = GeoArrays.coords(ga, [r_min,c_min])
	sub_f = CoordinateTransformations.AffineMap(sub_afm_l, sub_afm_t)
	sub_crs = ga.crs
	sub_ga = GeoArray(sub_a, sub_f, sub_crs)
    return sub_ga
end

This works for row and column and band subsets. It could be rewritten also to use geographical coordinates (compute index from coords).

ga_sub = ga_subset(geoarray, [1,7], [200,200,400,300]);
typeof(ga_sub) # 201x101x2 Array{UInt16, 3} with AffineMap([20.0 0.0; 0.0 -20.0], [448980.0, 5.12002e6]) ...

GeoArray slicing is of course complex, but for simple subsetting adding methods for band slicing and spatial slicing could be beneficial.

Support for warp?

Are there any intentions for GeoArrays to support warping between projections?

Crop returns incorrect result

This example returns a 100x0x1 GeoArray when I believe it should return a 100x100x1 GeoArray, no?

ga = GeoArray(rand(100,100))
bbox!(ga, (min_x=.5, min_y=.5, max_x=100.5, max_y=100.5))  
extent = bbox(ga)
foo = crop(ga, extent)

using GeoArrays v0.8.3

Type conversion

I'm finding that in my workflow I need to convert Int16 GeoArrays to Floa32 when applying some filters to the data. The only way I can figure to do this is to create an entirely new GeoArray. A convert! function would be helpful for myself and maybe others.

Read single band from raster

When working with large rasters, e.g. with satellite images that can be GB in size, it is useful to be able to read only one band (or a selection of them) to GeoArray. Now the best option is to read the whole dataset and subset by band.

As suggested by @visr ArchGDAL has the ArchGDAL.getband() that could be exported to GeoArrays. GeoArrays.jl could add a method getband(::GeoArray, i::Int)::GeoArray to enable reading by band.

See also the conversation at Julia Discourse: GeoArrays band math.

BBox is Float or Int

Currently, BBox is defined as Float64.

bbox::NamedTuple{(:min_x, :min_y, :max_x, :max_y),Tuple{Float64, Float64, Float64, Float64}})

This is mostly true, but BBox can also be Int64. Or a combination of them (each element can be either Int or Float). GeoArrays does not handle this.

julia> GeoArrays.bbox_to_affine((10,10), (min_x=10, min_y=10, max_x=100, max_y=100))
ERROR: MethodError: no method matching bbox_to_affine(::Tuple{Int64, Int64}, ::NamedTuple{(:min_x, :min_y, :max_x, :max_y), NTuple{4, Int64}})
Closest candidates are:
  bbox_to_affine(::Tuple{Integer, Integer}, ::NamedTuple{(:min_x, :min_y, :max_x, :max_y), NTuple{4, Float64}})

Plot example doesn't work in Julia 1.6.4

On my machine (MacOS)

julia> using GeoArrays

julia> using Plots

julia> fn = download("https://github.com/yeesian/ArchGDALDatasets/blob/master/pyrasterio/RGB.byte.tif?raw=true")
"/var/folders/cq/80lc4bfd5qg0k4q1j8zc67lh0000gn/T/jl_KTUIxD"

julia> ga = GeoArrays.read(fn)
791x718x3 Array{Union{Missing, UInt8}, 3} with AffineMap([300.0379266750948 0.0; 0.0 -300.041782729805], [101985.0, 2.826915e6]) and CRS PROJCS["UTM Zone 18, Northern Hemisphere",GEOGCS["Unknown datum based upon the WGS 84 ellipsoid",DATUM["Not_specified_based_on_WGS_84_spheroid",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-75],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH]]

julia> plot(ga)
ERROR: UndefVarError: coords not defined
Stacktrace:
[1] macro expansion
@ ~/.julia/packages/GeoArrays/oqARi/src/plot.jl:10 [inlined]
[2] apply_recipe(plotattributes::AbstractDict{Symbol, Any}, ga::GeoArray)
@ GeoArrays ~/.julia/packages/RecipesBase/qpxEX/src/RecipesBase.jl:289
[3] _process_userrecipes!(plt::Any, plotattributes::Any, args::Any)
@ RecipesPipeline ~/.julia/packages/RecipesPipeline/Bxu2O/src/user_recipe.jl:36
[4] recipe_pipeline!(plt::Any, plotattributes::Any, args::Any)
@ RecipesPipeline ~/.julia/packages/RecipesPipeline/Bxu2O/src/RecipesPipeline.jl:70
[5] _plot!(plt::Plots.Plot, plotattributes::Any, args::Any)
@ Plots ~/.julia/packages/Plots/qbc7U/src/plot.jl:208
[6] plot(args::Any; kw::Any)
@ Plots ~/.julia/packages/Plots/qbc7U/src/plot.jl:91
[7] plot(args::Any)
@ Plots ~/.julia/packages/Plots/qbc7U/src/plot.jl:85
[8] top-level scope
@ REPL[8]:1

===

Changing line 10 in plot.jl from:

coords = coords(ga, Vertex())

to

coords = GeoArrays.coords(ga, Vertex())

fixes the issue

Coalesce complete GeoArray

Currently, it's hard to dump all missing values from a GeoArray and change the type.
Ideally, we have coalesce(ga, Inf) or similar.

Allow ranges in getindex

Hi! geoarray[1,1] works and returns the contents of all bands at position 1,1. However ga[1:10,1:10] errors. Therefore I propose to introduce:

Base.getindex(ga::GeoArray, I::Vararg{<:AbstractRange, 2}) = getindex(ga.A, I..., :)

Edit: Actually I would boil down the whole rest to:

Base.getindex(ga::GeoArray, I) = getindex(ga.A, I)

Upstream changes break GeoArrays

Problem description

GeoArrays no longer compiles due to breaking changes upstream (JuliaEarth/GeoStatsBase.jl#174).

The problem lies in the use of the internal library GeoStatsBase in interpolate.jl, which has recently been reorganised.

Workaround

Fixing TableTransforms.jl at 1.15.0 allows compilation again, but is obviously not a long-term solution.

Instead of using GeoStatsBase, the main library GeoStats.jl should be used.

Stacktrace

julia> using GeoArrays
[ Info: Precompiling GeoArrays [2fb1d81b-e6a0-5fc5-82e6-8e06903437ab]
WARNING: could not import TableTransforms.ColSpec into GeoStatsBase
WARNING: could not import TableTransforms.Col into GeoStatsBase
WARNING: could not import TableTransforms.colspec into GeoStatsBase
WARNING: could not import TableTransforms.choose into GeoStatsBase
ERROR: LoadError: UndefVarError: `ColSpec` not defined
Stacktrace:
  [1] top-level scope
    @ ~/.julia/packages/GeoStatsBase/hN0RT/src/transforms/detrend.jl:5
  [2] include(mod::Module, _path::String)
    @ Base ./Base.jl:457
  [3] include(x::String)
    @ GeoStatsBase ~/.julia/packages/GeoStatsBase/hN0RT/src/GeoStatsBase.jl:5
  [4] top-level scope
    @ ~/.julia/packages/GeoStatsBase/hN0RT/src/transforms.jl:14
  [5] include(mod::Module, _path::String)
    @ Base ./Base.jl:457
  [6] include(x::String)
    @ GeoStatsBase ~/.julia/packages/GeoStatsBase/hN0RT/src/GeoStatsBase.jl:5
  [7] top-level scope
    @ ~/.julia/packages/GeoStatsBase/hN0RT/src/GeoStatsBase.jl:56
  [8] include
    @ ./Base.jl:457 [inlined]
  [9] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
    @ Base ./loading.jl:2049
 [10] top-level scope
    @ stdin:3
in expression starting at ~/.julia/packages/GeoStatsBase/hN0RT/src/transforms/detrend.jl:5
in expression starting at ~/.julia/packages/GeoStatsBase/hN0RT/src/transforms.jl:14
in expression starting at ~/.julia/packages/GeoStatsBase/hN0RT/src/GeoStatsBase.jl:5
in expression starting at stdin:3
ERROR: LoadError: Failed to precompile GeoStatsBase [323cb8eb-fbf6-51c0-afd0-f8fba70507b2] to "~/.julia/compiled/v1.9/GeoStatsBase/jl_OK8pRv".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2300
  [3] compilecache
    @ ./loading.jl:2167 [inlined]
  [4] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1805
  [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1660
  [6] macro expansion
    @ ./loading.jl:1648 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1611
  [9] include(mod::Module, _path::String)
    @ Base ./Base.jl:457
 [10] include(x::String)
    @ GeoArrays ~/.julia/packages/GeoArrays/Is81p/src/GeoArrays.jl:1
 [11] top-level scope
    @ ~/.julia/packages/GeoArrays/Is81p/src/GeoArrays.jl:17
 [12] include
    @ ./Base.jl:457 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2049
 [14] top-level scope
    @ stdin:3
in expression starting at ~/.julia/packages/GeoArrays/Is81p/src/interpolate.jl:1
in expression starting at ~/.julia/packages/GeoArrays/Is81p/src/GeoArrays.jl:1
in expression starting at stdin:3
ERROR: Failed to precompile GeoArrays [2fb1d81b-e6a0-5fc5-82e6-8e06903437ab] to "~/.julia/compiled/v1.9/GeoArrays/jl_eJEmzF".
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:2300
 [3] compilecache
   @ ./loading.jl:2167 [inlined]
 [4] _require(pkg::Base.PkgId, env::String)
   @ Base ./loading.jl:1805
 [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
   @ Base ./loading.jl:1660
 [6] macro expansion
   @ ./loading.jl:1648 [inlined]
 [7] macro expansion
   @ ./lock.jl:267 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1611

add methods for arithmetic operators?

I've been working on a project where I calculate two GeoArray layers using PointCloudRasterizers and then calculate a difference.
I can use layer1 - layer2 syntax and it calculates successfully but converts to a standard Array type and casting it as a GeoArray afterwards resets the coordinate information.
It would be nice to be able to do band math operations and have the coordinate info and everything get passed through automatically.
My example is simple and I know all layers are the same info, so I have not thought through how difficult it becomes if there are differences.If someone can point me to an example or give me some guidance on what this might look like I can try and put it together for a PR.

#example from PointCloudRasterizers README,  file from GeoArrays readme doesn't exist?

lazfn = joinpath(dirname(pathof(LazIO)), "..", "test/libLAS_1.2.laz")
pointcloud = LazIO.open(lazfn)
cellsizes = (1.,1.)
raster_index = index(pointcloud, cellsizes)

raster_maximum = reduce(raster_index, field=:Z, reducer=maximum)
raster_minimum = reduce(raster_index, field=:Z, reducer=minimum)

#original affine info from raster_minimum.f
AffineMap([1.0 0.0; 0.0 -1.0], [375000.03, 380000.03])

#calculate difference layer
raster_range = raster_maximum - raster_minimum #5000×5000×1 Array{Union{Missing, Float64},3}
#just casting it to GeoArray resets coord info
GeoArray(raster_range).f #AffineMap([1.0 0.0; 0.0 1.0], [0.0, 0.0])

#I can manually construct it afterwards, but that seems very manual
GeoArray(raster_range,raster_minimum.f,"")

KeyError: key Int64 not found on Int Array type

As reported by @vernimmen

julia> ga = GeoArray(rand(Int, (100,100)))
100x100x1 Array{Int64,3} with AffineMap([1.0 0.0; 0.0 1.0], [0.0, 0.0]) and undefined CRS

julia> GeoArrays.write!("test.tif", ga)
ERROR: KeyError: key Int64 not found
Stacktrace:
 [1] getindex at ./dict.jl:477 [inlined]
 [2] #unsafe_create#30 at /Users/evetion/.julia/packages/ArchGDAL/FiLcr/src/dataset.jl:200 [inlined]
 [3] create(::GeoArrays.var"#27#28"{GeoArray{Int64},Int64}, ::String; kwargs::Base.Iterators.Pairs{Symbol,Any,NTuple{6,Symbol},NamedTuple{(:driver, :width, :height, :nbands, :dtype, :options),Tuple{ArchGDAL.Driver,Int64,Int64,Int64,DataType,Array{String,1}}}}) at /Users/evetion/.julia/packages/ArchGDAL/FiLcr/src/context.jl:213
 [4] write!(::String, ::GeoArray{Int64}, ::Nothing, ::String) at /Users/evetion/.julia/packages/GeoArrays/95hSL/src/io.jl:80
 [5] write!(::String, ::GeoArray{Int64}) at /Users/evetion/.julia/packages/GeoArrays/95hSL/src/io.jl:61
 [6] top-level scope at REPL[35]:1

Abstracting projections

I chunked out your CRS code and projection conversions into a few packages. The Base package is to provide types so that CRS metadata can be a lightweight, standard addition to any spatial array, without GDAL. The main package provides your conversions as Base.convert() methods between the projection types (which is a little more Julian and modular), with a GDAL dep.

The Base package could go into the GeoArrayBase.jl but it could also be used in other contexts, so I'm trying to keep things focussed for now. Let me know if you want push rights, half of it is your code.

https://github.com/rafaqz/CoordinateReferenceSystems.jl/blob/master/src/CoordinateReferenceSystems.jl

https://github.com/rafaqz/CoordinateReferenceSystemsBase.jl/blob/master/src/CoordinateReferenceSystemsBase.jl

Installation error

(v1.3) pkg> add GeoArrays
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package GeoStatsBase [323cb8eb]:
 GeoStatsBase [323cb8eb] log:
 ├─possible versions are: [0.2.0-0.2.6, 0.3.0-0.3.6, 0.4.0-0.4.3, 0.5.0, 0.6.0-0.6.4, 0.7.0-0.7.1, 0.8.0-0.8.7, 0.9.0-0.9.5, 0.10.0-0.10.1] or uninstalled
 ├─restricted by compatibility requirements with StaticArrays [90137ffa] to versions: [0.2.0-0.2.6, 0.3.0-0.3.6, 0.4.0-0.4.3, 0.6.4, 0.7.0-0.7.1, 0.8.0-0.8.7, 0.9.0-0.9.5, 0.10.0-0.10.1] or uninstalled
 │ └─StaticArrays [90137ffa] log:
 │   ├─possible versions are: [0.8.0-0.8.3, 0.9.0-0.9.2, 0.10.0, 0.10.2-0.10.3, 0.11.0-0.11.1, 0.12.0-0.12.3] or uninstalled
 │   ├─restricted to versions 0.12 by GeometryBasics [5c1252a2], leaving only versions 0.12.0-0.12.3
 │   │ └─GeometryBasics [5c1252a2] log:
 │   │   ├─possible versions are: 0.2.13 or uninstalled
 │   │   ├─restricted to versions * by GeoJSONTables [b51d8a2e], leaving only versions 0.2.13
 │   │   │ └─GeoJSONTables [b51d8a2e] log:
 │   │   │   ├─possible versions are: 0.1.0 or uninstalled
 │   │   │   └─GeoJSONTables [b51d8a2e] is fixed to version 0.1.0
 │   │   └─GeometryBasics [5c1252a2] is fixed to version 0.2.13
 │   └─restricted to versions 0.12.3 by an explicit requirement, leaving only versions 0.12.3
 ├─restricted by compatibility requirements with DataFrames [a93c6f00] to versions: [0.2.0-0.2.6, 0.3.0-0.3.6, 0.4.0-0.4.3, 0.10.0-0.10.1] or uninstalled
 │ └─DataFrames [a93c6f00] log:
 │   ├─possible versions are: [0.11.7, 0.12.0, 0.13.0-0.13.1, 0.14.0-0.14.1, 0.15.0-0.15.2, 0.16.0, 0.17.0-0.17.1, 0.18.0-0.18.4, 0.19.0-0.19.4, 0.20.0-0.20.2, 0.21.0-0.21.2] or uninstalled
 │   ├─restricted to versions * by GeoDataFrames [fbc9b913], leaving only versions [0.11.7, 0.12.0, 0.13.0-0.13.1, 0.14.0-0.14.1, 0.15.0-0.15.2, 0.16.0, 0.17.0-0.17.1, 0.18.0-0.18.4, 0.19.0-0.19.4, 0.20.0-0.20.2, 0.21.0-0.21.2]
 │   │ └─GeoDataFrames [fbc9b913] log:
 │   │   ├─possible versions are: 0.0.0 or uninstalled
 │   │   └─GeoDataFrames [fbc9b913] is fixed to version 0.0.0
 │   └─restricted to versions 0.21.0 by an explicit requirement, leaving only versions 0.21.0
 └─restricted by compatibility requirements with GeoArrays [2fb1d81b] to versions: [0.5.0, 0.6.0-0.6.4, 0.7.0-0.7.1, 0.8.3-0.8.7] — no versions left
   └─GeoArrays [2fb1d81b] log:
     ├─possible versions are: [0.1.0-0.1.7, 0.2.0-0.2.2, 0.3.0-0.3.1] or uninstalled
     ├─restricted to versions * by an explicit requirement, leaving only versions [0.1.0-0.1.7, 0.2.0-0.2.2, 0.3.0-0.3.1]
     ├─restricted by compatibility requirements with RecipesBase [3cdcf5f2] to versions: [0.1.0-0.1.4, 0.2.2, 0.3.0-0.3.1] or uninstalled, leaving only versions: [0.1.0-0.1.4, 0.2.2, 0.3.0-0.3.1]
     │ └─RecipesBase [3cdcf5f2] log:
     │   ├─possible versions are: [0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0, 1.0.0-1.0.1] or uninstalled
     │   ├─restricted to versions * by GeoDataFrames [fbc9b913], leaving only versions [0.4.0, 0.5.0, 0.6.0, 0.7.0, 0.8.0, 1.0.0-1.0.1]
     │   │ └─GeoDataFrames [fbc9b913] log: see above
     │   ├─restricted by compatibility requirements with GeoInterface [cf35fbd7] to versions: [0.6.0, 0.7.0, 0.8.0, 1.0.0-1.0.1]
     │   │ └─GeoInterface [cf35fbd7] log:
     │   │   ├─possible versions are: [0.4.0-0.4.1, 0.5.0-0.5.4] or uninstalled
     │   │   ├─restricted to versions 0.4-0.5 by LibGEOS [a90b1aa1], leaving only versions [0.4.0-0.4.1, 0.5.0-0.5.4]
     │   │   │ └─LibGEOS [a90b1aa1] log:
     │   │   │   ├─possible versions are: 0.6.4 or uninstalled
     │   │   │   └─LibGEOS [a90b1aa1] is fixed to version 0.6.4
     │   │   ├─restricted to versions 0.5 by GeoMakie [db073c08], leaving only versions 0.5.0-0.5.4
     │   │   │ └─GeoMakie [db073c08] log:
     │   │   │   ├─possible versions are: 0.1.14 or uninstalled
     │   │   │   └─GeoMakie [db073c08] is fixed to version 0.1.14-DEV
     │   │   └─restricted to versions 0.5.4 by an explicit requirement, leaving only versions 0.5.4
     │   └─restricted by compatibility requirements with Plots [91a5bcdd] to versions: 1.0.0-1.0.1
     │     └─Plots [91a5bcdd] log:
     │       ├─possible versions are: [0.12.1-0.12.4, 0.13.0-0.13.1, 0.14.0-0.14.2, 0.15.0-0.15.1, 0.16.0, 0.17.0-0.17.4, 0.18.0, 0.19.0-0.19.3, 0.20.0-0.20.6, 0.21.0, 0.22.0-0.22.5, 0.23.0-0.23.2, 0.24.0, 0.25.0-0.25.3, 0.26.0-0.26.3, 0.27.0-0.27.1, 0.28.0-0.28.4, 0.29.0-0.29.9, 1.0.0-1.0.14, 1.1.0-1.1.4, 1.2.0-1.2.6, 1.3.0-1.3.7, 1.4.0-1.4.3] or uninstalled
     │       └─restricted to versions 1.0.14 by an explicit requirement, leaving only versions 1.0.14
     └─restricted by compatibility requirements with ArchGDAL [c9ce4bd3] to versions: [0.1.4-0.1.7, 0.2.0-0.2.2, 0.3.0] or uninstalled, leaving only versions: [0.1.4, 0.2.2, 0.3.0]
       └─ArchGDAL [c9ce4bd3] log:
         ├─possible versions are: [0.1.0, 0.2.0-0.2.2, 0.3.0-0.3.3, 0.4.0-0.4.1] or uninstalled
         └─restricted to versions 0.3.2 by an explicit requirement, leaving only versions 0.3.2

GeoArrays.read(ga, masked=false) should probably be default

In all of my workflows I need to set masked=false when using GeoArrays.read otherwise I get really slow read times. Also masked=false results in a change in Type as missings are added. For this reason I think masked=false should be set as the default

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.