Render large spatial datasets in shiny leaflet about auunconf HOT 13 OPEN

Hi,

Thanks for getting in touch. I wouldn't really recommend using theses functions as there has been a lot of work done on solving this problem since the unconf. @tim-salabim has been doing some amazing on rendering large spatial data sets in leaflet (see https://github.com/tim-salabim/leaflet.glify; both polygon and point data supported I think). @SymbolixAU has been doing fantastic work too (see https://github.com/SymbolixAU/mapdeck; though this uses Deck.gl and not leaflet). So, I would recommend trying out those packages and see if they can do what you need.

Cheers,

Jeff

from auunconf.

Hi, i have tested your package. I think it is in the good way. I haves tested it with a 60K+ row spatialLineDataframe. Wanted to share with you my experience with a large data frame.

the geom.simplify() took 3min to disaggregate and give me a 1.1Gb list (started with a 37.5Mb Large SpatialLines) with the same variables as in your example.

• Loading the layer with the simplified data is "faster" than with original dataframe.
• Switching zoom levels is really faster than using directly leaflet and the original data frame.
• But dragging the map is still long (may be because it clear all the layer depending on a group and not if the layer are still in the window)

from auunconf.

Hi,

any news about this package project? I found it super awesome and was thinking about maybe working on something like that at some point if nothing exists...

from auunconf.

Have you tried mapview? Also see this proposal for its future direction:
https://github.com/environmentalinformatics-marburg/mapview_toolchain/blob/master/mapview_interactive_data_manipulation.Rmd.

On Wed, Aug 17, 2016, 02:23 Joel Gombin [email protected] wrote:

Hi,

any news about this package project? I found it super awesome and was
thinking about maybe working on something like that at some point if
nothing exists...

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#38 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD6tb4D-r8xdo0gRxWVeL_cY-K-iAz3Mks5qgeQXgaJpZM4IHFnx
.

Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia

from auunconf.

Hi,

yes, I use mapview on a regular basis but I'm not sure it offers special tools for viewing very large spatial datasets?
Also, the roadmap you point to only mentions very briefly the issue of large dataset.
In the meantime I stumbled across your leafier repo; did you abandon its development?

from auunconf.

I'd be interested in contributing to an R package to render large spatial
datasets in shiny leaflet.

How can we move this ahead?

On Tue, Aug 16, 2016 at 1:32 PM, Joel Gombin [email protected]
wrote:

Hi,

yes, I use mapview on a regular basis but I'm not sure it offers special
tools for viewing very large spatial datasets?
Also, the roadmap you point to only mentions very briefly the issue of
large dataset.
In the meantime I stuùbled across your leafier repo; did you abandon its
development?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#38 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGiO7FwK45V3Jmbl_OCHmLsCZHsgS7_Wks5qgfRHgaJpZM4IHFnx
.

from auunconf.

Hi,

Thanks for showing an interest in this. We had a working prototype by the end of the unconference but I haven't had time to develop this into a package.

We ended up scrapping the tiles because we couldn't work out how to make that interactive. So, we ended up simplifying the vector data to various tolerances (using rgeos::gSimplify) and then clipping to based on the user's view port (using rgeos::gIntersection) and sending that to leaflet for rendering.

The prototype should be in the leafier repo. Let me know if you have any questions.

from auunconf.

I just found the mapview R package @mdsumner mentioned:

https://github.com/environmentalinformatics-marburg/mapview

from auunconf.

Correct me if I'm wrong, but there's no actual code in the leafier repo, is it? (except for the geom.subset function)
Thanks for the information anyway! i'll keep thinking about this tile thing!

from auunconf.

This leafier yeah?

https://github.com/ropenscilabs/leafier

Thanks, I'll have a look. I'm not experienced enough with the browser side,
but it seems to me a lot could be done to link the web tool straight to the
GDAL functions, which are inherently geared to on-demand read. Reading into
R is probably not that helpful, but writing the hooks between GDAL and the
browser tool is probably reasonable. Ultimately mapview should do that,
simply provide a "viewport" request back to registered data sources, and
the geo-spatial library/database does the rest. Also there are non-R tools
that already do this for leaflet afaik.

Of course R is great for prototyping this kind of thing though, and there
are tools like raster that make it possible to minimize the middle-man
overhead. (rgdal has these on-demand functions too but far less user
friendly).

On Wed, 17 Aug 2016 at 14:08 Joel Gombin [email protected] wrote:

Correct me if I'm wrong, but there's no actual code in the leafier repo,
is it? (except for the geom.subset function)
Thanks for the information anyway! i'll keep thinking about this tile
thing!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#38 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD6tb2QTOq8pJrfkeWCDdZenVYfgoVZ0ks5qgok0gaJpZM4IHFnx
.

Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia

from auunconf.

Yeah , sorry I forgot we didn't put it under the master branch

here's the prototype:

https://github.com/ropenscilabs/leafier/tree/amy-jeff-dev/leaflet_test

EDIT: turns out we did - and I just can't read haha

from auunconf.

Ah thanks, I was looking at the paleo13/leafier repo!

from auunconf.

Mr. Hanson,

Do your functions (geom.subset and geom.simplify), work with points? The code is specifically for polygons. I attempted to modify them to get them to work with points, but have as yet been unsuccessful.

# define functions
geom.subset.points <- function(spgeom, top, left, bottom, right ){
coords <- data.frame(y = c(left, right, right, left ), x =c(top, top, bottom, bottom) )
poly <- sp::SpatialPoints(coords)
polys <- sp::SpatialPoints( list(poly) , ID =0)
polys.spatial <- SpatialPoints(list(polys), proj4string = spgeom@proj4string)
inter <- rgeos::gIntersection(spgeom, polys.spatial, byid =TRUE)
}

from auunconf.