rspatial / luna Goto Github PK
View Code? Open in Web Editor NEWsatellite remote sensing tools
License: GNU General Public License v3.0
satellite remote sensing tools
License: GNU General Public License v3.0
Strictly following the workflow of https://rspatial.org/terra/modis/2-download.html leads to empty HDF files being downloaded.
Opening the HDF file in Editor shows the following text:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Unauthorized</title>
</head><body>
<h1>Unauthorized</h1>
<p>This server could not verify that you
are authorized to access the document
requested. Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
</body></html>
Using the same credentials works, when manually downloading the files in EarthData.
Does anybody else experience this issue or is it a local issue?
I'm looking forward to your replies.
Sometimes file downloads getting stuck in midway. Need ctrl+c
to escape and move to next download. One example situation shown below.
Possible solutions and new feature request:
download
functioncontent-length
from the httr::post
request and attempt to re-download if the numbers don't match|======================================================================| 100%
|================== | 25%^C
Warning messages:
1: In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2004.06.17/MYD13Q1.A2004169.h18v07.006.2015154155539.hdf
2: In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2009.10.08/MYD13Q1.A2009281.h18v07.006.2015195132705.hdf
3: In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2009.10.08/MYD13Q1.A2009281.h18v08.006.2015195135818.hdf
4: In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2009.10.08/MYD13Q1.A2009281.h17v07.006.2015195135725.hdf
5: In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2009.10.08/MYD13Q1.A2009281.h17v08.006.2015195135815.hdf
|======================================================================| 100%
|======================================================================| 100%
|============ | 18%^C
Warning message:
In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2013.05.17/MYD13Q1.A2013137.h17v07.006.2015260174324.hdf
|======================================================================| 100%
Hello @rhijmans
I would like to download the MODIS product "MOD10A1", over a long time period for an Italian region.
Where I proceed to download the data, I get this error:
Errore in getNASA(product = product, start_date = start, end_date = end, :
The requested product is not available for this product version or server
Thanks a lot for your help!
Lorenzo Bernicchi
This is the very simple script I used to download the data:
install.packages('luna', repos='https://rspatial.r-universe.dev')
library(terra)
library(luna)
product <- "MOD10A1"
productInfo(product)
#It redirects to a non existing web page
luna::getProducts("MOD10A1")
# provider concept_id short_name version
#2675 NSIDC_ECS C1000001167-NSIDC_ECS MOD10A1 6
#53928 NSIDC_ECS C183614535-NSIDC_ECS MOD10A1 3
#56156 NSIDC_ECS C50772655-NSIDC_ECS MOD10A1 4
#56841 NSIDC_ECS C92711294-NSIDC_ECS MOD10A1 5
start <- "2024-03-01"
end <- "2024-03-31"
FVG <- terra::vect("C:\\Users\\Loren\\Desktop\\Global_models_WVCs\\FVG_3035.shp")
mf <- getNASA(
product = product,
start_date = start,
end_date = end,
aoi = FVG,
download = T,
version = "6",
path = "C:\\Users\\Loren\\Desktop\\Snow_Cover_Raw_Data",
username = "lorenzo_bernicchi",
password = "PisaMerda1996!",
server = "NSIDC_ECS"
)
Running the example code for getLandsat() returns "No results found". This happens for every query I run with getLandsat
.
product <- "Landsat_8_OLI_TIRS_C1"
sdate <- "2019-05-01"
edate <- "2019-05-30"
area <- c(33, 34, -5, -4)
dir <- tempdir()
f <- getLandsat(product, start_date=sdate, end_date=edate,
aoi=area, download=FALSE, path=dir)
> [1] No results found
Thanks!
It would be great if there are some examples of download_ee and download_ers in the document.
Hi,
I just installed luna
to download AVHRR data, and I keep getting the following error:
> library('luna') # for avhrr, modis, sentinel, and landsat data
>
> getAVHRR(start_date = '1981-06-24',
+ end_date = '1981-06-24',
+ path = 'data/ndvi-rasters/avhrr',
+ overwrite = FALSE,
+ update = FALSE)
Creating index of available AVHRR files on 2023-07-02
Downloading AVHRR tile for 1981-06-24
Error in match.arg(method, c("auto", "internal", "wininet", "libcurl", :
'arg' must be NULL or a character vector
The error occurs with both the GitHub version (remotes::install_github("rspatial/luna")
) and the version from install.packages("luna", repos = "https://rspatial.r-universe.dev")
. I believe it's because download.file
doesn't have a default method
and getAVHRR
can't pass the method
argument since the download.file
call is missing a ...
.
Cheers,
Stefano
Hi there,
I'm using luna to pull MODIS data. I had a getModis() script that worked as expected yesterday, but today I'm getting the following errors:
In .cmr_download(urls, path, username, password, overwrite) :
failure: https://e4ftl01.cr.usgs.gov//DP106/MOLT/MOD13Q1.006/2010.06.26/MOD13Q1.A2010177.h11v04.006.2015208204334.hdf
2: In .cmr_download(urls, path, username, password, overwrite) :
failure: https://e4ftl01.cr.usgs.gov//DP136/MOLT/MOD13Q1.061/2010.06.26/MOD13Q1.A2010177.h11v04.061.2021168170216.hdf
Two files do download to the expected location, but now when i try to turn the files into raster objects using the rast() function from the terra package, I get a new error:
Error: [rast] cannot open file: /Users/ryan/Desktop/untitled folder/MOD13Q1.A2010177.h11v04.061.2021168170216.hdf
In addition: Warning message:
`/Users/jim/Desktop/untitled folder/MOD13Q1.A2010177.h11v04.061.2021168170216.hdf' not recognized as a supported file format. (GDAL error 4)
Is this a common suite of errors? I'm trying to figure out if the download error is coming from a bug in luna or a problem with the download site. I haven't changed the code, so I don't think I've messed something up on my end, but that's entirely possible. Is there a dependency I'm forgetting to load other than luna and terra?
Thanks for your time.
Hi
I'm trying to access data for MOD44B, but only version 006. However, the getModis()
function does not respect the version argument, such that also version 061 gets returned. I think this is coming from the fact that the argument is not being passed on to the .searchGranules()
function.
getModis(
product = "MOD44B"
, start_date = "2011-01-01"
, end_date = "2012-01-01"
, aoi = extent(c(20, 28, -22, -16))
, server = "LPDAAC_ECS"
, version = "006" # doesn't do anything
)
Sentinel 2
Source | Sensor/Version | Description/URL | Download format | Notes |
---|---|---|---|---|
Copernicus Open Access Hub | L1C/L2A | API Hub | SAFE | Alternate European mirrors available |
AWS | L1C | AWS Registry | SAFE/TIFF | Not free(?) |
USGS | L1C | EarthExplorer | SAFE (tiled) | Does it work with CMR? |
Hi, I tried to download MOD09A1 files using getNASA. I only got files of 27bytes and cannot read it with rast() from terra. However, if I download the same file from website they are bigger and I can read it with terra. I followed the steps from https://rspatial.org/modis/modis.pdf.
Could anyone help me?
Hi, first I wanted to thank you for the package. I also wanted to notify an issue. LP DAAC recently updated some of its products including the one that I am working with, "MOD17A3" (Primary productivity data). I tried to use the updated "MOD17A3H" version 061 in getModis. However, it does not seem to retrieve any products (it returned "No results found"). I am not experiencing this issues when I try other products and keep the rest of the arguments.
Thanks for making the incredibly useful image processing tools! I noticed one issue for modis_mask(). According to the official instructions, the bit-string should be read from right to left and the bit-word should be read from left to right (see below). But in modis_mask(), it just reads from left to right for the bit-string given the user-specified start-end bits.
For example, if QA integer is 9057 then 16-bit binary code is 0 0 1 0 0 0 1 1 0 1 1 0 0 0 0 1. If I were to set reject 9-12 of 0110, then modis_mask() will identify this pixel for rejection and eventually users may accidently lost one 'good' pixel.
Hi, when I try to get MODIS or Landsat data I get the following error:
Error in curl::curl_fetch_memory(url, handle = handle) :
Failed initialization
And trying to download AVHRR data gives me this error:
Error in open.connection(x, "rb") :
Timeout was reached: [www.ncei.noaa.gov] Resolving timed out after 10000 milliseconds
Am I doing something wrong? Thanks
Hi,
Just working through your tutorial https://youtu.be/5b2xhqlH49I and got the following message.
fmod <- getModis(product = "MOD09A1", start, end,aoi = aoi) Error in lenth(aoi) : could not find function "lenth"
My workaround was aoi = raster::extent(aoi)
Cheers,
Julian.
It would be great if RStoolbox::mesma()
https://github.com/bleutner/RStoolbox/blob/master/R/mesma.R
or similar could be included in terra.
Sentinel 1
Source | Sensor/Version | Description/URL | Download format | Notes |
---|---|---|---|---|
Copernicus Open Access Hub | Level 0/1/2 | API Hub | SAFE | Alternate European mirrors available |
Hi,
I am trying to install luna on R but I keep getting this error:
`Downloading GitHub repo rspatial/luna@HEAD
โ checking for file 'C:\Users\admin\AppData\Local\Temp\RtmpoH5VLt\remotes25e81737276d\rspatial-luna-66da15b/DESCRIPTION' (821ms)
Session Info:
`> sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)
Matrix products: default
locale:
[1] LC_COLLATE=English_Kenya.1252 LC_CTYPE=English_Kenya.1252 LC_MONETARY=English_Kenya.1252
[4] LC_NUMERIC=C LC_TIME=English_Kenya.1252
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base
other attached packages:
[1] fasterize_1.0.3 doParallel_1.0.16 iterators_1.0.13 foreach_1.5.1 tmaptools_3.1-1
[6] maditr_0.8.2 geodata_0.3-5 terra_1.4-22 tmap_3.3-2 basf_0.2.0
[11] sf_1.0-5 data.table_1.14.2 pacman_0.5.1
loaded via a namespace (and not attached):
[1] Rcpp_1.0.7 lattice_0.20-45 prettyunits_1.1.1 ps_1.6.0 png_0.1-7
[6] class_7.3-19 rprojroot_2.0.2 digest_0.6.29 utf8_1.2.2 V8_3.6.0
[11] R6_2.5.1 e1071_1.7-9 ggplot2_3.3.5 pillar_1.6.4 rlang_0.4.12
[16] curl_4.3.2 geojsonlint_0.4.0 callr_3.7.0 raster_3.5-9 htmlwidgets_1.5.4
[21] munsell_0.5.0 proxy_0.4-26 compiler_4.1.2 rmapshaper_0.4.5 pkgconfig_2.0.3
[26] base64enc_0.1-3 pkgbuild_1.3.1 htmltools_0.5.2 tibble_3.1.6 httpcode_0.3.0
[31] codetools_0.2-18 XML_3.99-0.8 jsonvalidate_1.3.2 fansi_0.5.0 viridisLite_0.4.0
[36] withr_2.4.3 crayon_1.4.2 crul_1.2.0 grid_4.1.2 jsonlite_1.7.2
[41] lwgeom_0.2-8 gtable_0.3.0 lifecycle_1.0.1 DBI_1.1.2 magrittr_2.0.1
[46] units_0.7-2 scales_1.1.1 KernSmooth_2.23-20 cli_3.1.0 leafsync_0.1.0
[51] remotes_2.4.2 leaflet_2.0.4.1 sp_1.4-6 ellipsis_0.3.2 vctrs_0.3.8
[56] RColorBrewer_1.1-2 tools_4.1.2 dichromat_2.0-0 leafem_0.1.6 glue_1.6.0
[61] crosstalk_1.2.0 processx_3.5.2 abind_1.4-5 fastmap_1.1.0 colorspace_2.0-2
[66] stars_0.5-5 classInt_0.4-3 `
Kindly assist. Thanks
Could pansharpening (eg. the one in gdal_pansharpen.py) be implemented?
I'm trying to install the luna package, but am receiving an error stating it is not available for my version of R.
Will it be updated for new versions of R?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.