Sometimes file downloads getting stuck in midway. Need ctrl+c to escape and move to next download. One example situation shown below.

Possible solutions and new feature request:

1. Add a time-out in the download function
2. Compare size of downloaded file and content-length from the httr::post request and attempt to re-download if the numbers don't match
3. Use another alternative to the progress bar showing download status

|======================================================================| 100%
|================== | 25%^C
Warning messages:
1: In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2004.06.17/MYD13Q1.A2004169.h18v07.006.2015154155539.hdf
2: In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2009.10.08/MYD13Q1.A2009281.h18v07.006.2015195132705.hdf
3: In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2009.10.08/MYD13Q1.A2009281.h18v08.006.2015195135818.hdf
4: In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2009.10.08/MYD13Q1.A2009281.h17v07.006.2015195135725.hdf
5: In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2009.10.08/MYD13Q1.A2009281.h17v08.006.2015195135815.hdf
|======================================================================| 100%
|======================================================================| 100%
|============ | 18%^C
Warning message:
In cmr_download(urls = fileurls, path = path, username = cred$user, :
failure:https://e4ftl01.cr.usgs.gov//DP107/MOLA/MYD13Q1.006/2013.05.17/MYD13Q1.A2013137.h17v07.006.2015260174324.hdf

|======================================================================| 100%

Hello @rhijmans

I would like to download the MODIS product "MOD10A1", over a long time period for an Italian region.
Where I proceed to download the data, I get this error:

Errore in getNASA(product = product, start_date = start, end_date = end,  : 
  The requested product is not available for this product version or server

Thanks a lot for your help!
Lorenzo Bernicchi

This is the very simple script I used to download the data:

install.packages('luna', repos='https://rspatial.r-universe.dev')

library(terra)
library(luna)

product <- "MOD10A1"
productInfo(product)
#It redirects to a non existing web page
luna::getProducts("MOD10A1")
#       provider            concept_id short_name version
#2675  NSIDC_ECS C1000001167-NSIDC_ECS    MOD10A1       6
#53928 NSIDC_ECS  C183614535-NSIDC_ECS    MOD10A1       3
#56156 NSIDC_ECS   C50772655-NSIDC_ECS    MOD10A1       4
#56841 NSIDC_ECS   C92711294-NSIDC_ECS    MOD10A1       5

start <- "2024-03-01"
end <- "2024-03-31"

FVG <- terra::vect("C:\\Users\\Loren\\Desktop\\Global_models_WVCs\\FVG_3035.shp")

mf <- getNASA(
  product = product,
  start_date = start,
  end_date = end,
  aoi = FVG,
  download = T,
  version = "6",
  path = "C:\\Users\\Loren\\Desktop\\Snow_Cover_Raw_Data",
  username = "lorenzo_bernicchi",
  password = "PisaMerda1996!",
  server = "NSIDC_ECS"
)

I'm trying to install the luna package, but am receiving an error stating it is not available for my version of R.
Will it be updated for new versions of R?

Thanks for making the incredibly useful image processing tools! I noticed one issue for modis_mask(). According to the official instructions, the bit-string should be read from right to left and the bit-word should be read from left to right (see below). But in modis_mask(), it just reads from left to right for the bit-string given the user-specified start-end bits.

For example, if QA integer is 9057 then 16-bit binary code is 0 0 1 0 0 0 1 1 0 1 1 0 0 0 0 1. If I were to set reject 9-12 of 0110, then modis_mask() will identify this pixel for rejection and eventually users may accidently lost one 'good' pixel.

Running the example code for getLandsat() returns "No results found". This happens for every query I run with getLandsat.

product <- "Landsat_8_OLI_TIRS_C1"
sdate <- "2019-05-01" 
edate <- "2019-05-30"
area <- c(33, 34, -5, -4)
dir <- tempdir()
f <- getLandsat(product, start_date=sdate, end_date=edate,
            aoi=area, download=FALSE, path=dir)
> [1] No results found

Thanks!

Hi

I'm trying to access data for MOD44B, but only version 006. However, the getModis()function does not respect the version argument, such that also version 061 gets returned. I think this is coming from the fact that the argument is not being passed on to the .searchGranules() function.

getModis(
    product    = "MOD44B"
  , start_date = "2011-01-01"
  , end_date   = "2012-01-01"
  , aoi        = extent(c(20, 28, -22, -16))
  , server     = "LPDAAC_ECS"
  , version    = "006" # doesn't do anything
)

Hi there,

I'm using luna to pull MODIS data. I had a getModis() script that worked as expected yesterday, but today I'm getting the following errors:

In .cmr_download(urls, path, username, password, overwrite) :
failure: https://e4ftl01.cr.usgs.gov//DP106/MOLT/MOD13Q1.006/2010.06.26/MOD13Q1.A2010177.h11v04.006.2015208204334.hdf

2: In .cmr_download(urls, path, username, password, overwrite) :
failure: https://e4ftl01.cr.usgs.gov//DP136/MOLT/MOD13Q1.061/2010.06.26/MOD13Q1.A2010177.h11v04.061.2021168170216.hdf

Two files do download to the expected location, but now when i try to turn the files into raster objects using the rast() function from the terra package, I get a new error:

Error: [rast] cannot open file: /Users/ryan/Desktop/untitled folder/MOD13Q1.A2010177.h11v04.061.2021168170216.hdf
In addition: Warning message:
`/Users/jim/Desktop/untitled folder/MOD13Q1.A2010177.h11v04.061.2021168170216.hdf' not recognized as a supported file format. (GDAL error 4)

Is this a common suite of errors? I'm trying to figure out if the download error is coming from a bug in luna or a problem with the download site. I haven't changed the code, so I don't think I've messed something up on my end, but that's entirely possible. Is there a dependency I'm forgetting to load other than luna and terra?

Thanks for your time.

Hi, when I try to get MODIS or Landsat data I get the following error:

Error in curl::curl_fetch_memory(url, handle = handle) : 
  Failed initialization

And trying to download AVHRR data gives me this error:

Error in open.connection(x, "rb") : 
  Timeout was reached: [www.ncei.noaa.gov] Resolving timed out after 10000 milliseconds

Am I doing something wrong? Thanks

It would be great if there are some examples of download_ee and download_ers in the document.

Strictly following the workflow of https://rspatial.org/terra/modis/2-download.html leads to empty HDF files being downloaded.

Opening the HDF file in Editor shows the following text:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Unauthorized</title>
</head><body>
<h1>Unauthorized</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
</body></html>

Using the same credentials works, when manually downloading the files in EarthData.
Does anybody else experience this issue or is it a local issue?

I'm looking forward to your replies.

Hi, first I wanted to thank you for the package. I also wanted to notify an issue. LP DAAC recently updated some of its products including the one that I am working with, "MOD17A3" (Primary productivity data). I tried to use the updated "MOD17A3H" version 061 in getModis. However, it does not seem to retrieve any products (it returned "No results found"). I am not experiencing this issues when I try other products and keep the rest of the arguments.

Hi,
Just working through your tutorial https://youtu.be/5b2xhqlH49I and got the following message.

fmod <- getModis(product = "MOD09A1", start, end,aoi = aoi) Error in lenth(aoi) : could not find function "lenth"

My workaround was aoi = raster::extent(aoi)

Cheers,
Julian.

Hi,

I just installed luna to download AVHRR data, and I keep getting the following error:

> library('luna') # for avhrr, modis, sentinel, and landsat data
> 
> getAVHRR(start_date = '1981-06-24',
+          end_date = '1981-06-24',
+          path = 'data/ndvi-rasters/avhrr',
+          overwrite = FALSE,
+          update = FALSE)
Creating index of available AVHRR files on 2023-07-02 
Downloading AVHRR tile for 1981-06-24 
Error in match.arg(method, c("auto", "internal", "wininet", "libcurl",  : 
  'arg' must be NULL or a character vector

The error occurs with both the GitHub version (remotes::install_github("rspatial/luna")) and the version from install.packages("luna", repos = "https://rspatial.r-universe.dev"). I believe it's because download.file doesn't have a default method and getAVHRR can't pass the method argument since the download.file call is missing a ....

Cheers,
Stefano

It would be great if RStoolbox::mesma()
https://github.com/bleutner/RStoolbox/blob/master/R/mesma.R
or similar could be included in terra.

Could pansharpening (eg. the one in gdal_pansharpen.py) be implemented?

Sentinel 1

• Provider: ESA
• Versions: Level 0/1/2

Sentinel 2

• Provider: ESA
• Versions: L1C, L2A

Hi,

I am trying to install luna on R but I keep getting this error:

`Downloading GitHub repo rspatial/luna@HEAD
√ checking for file 'C:\Users\admin\AppData\Local\Temp\RtmpoH5VLt\remotes25e81737276d\rspatial-luna-66da15b/DESCRIPTION' (821ms)

• preparing 'luna': (856ms)
  √ checking DESCRIPTION meta-information ...
• cleaning src
• installing the package to process help pages
  -----------------------------------
• installing source package 'luna' ...
  ** using staged installation
  ** libs
  "C:/rtools40/mingw64/bin/"g++ -std=gnu++11 -I"F:/Folders/Programs/R-4.1.2/include" -DNDEBUG -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -Wall -pedantic -fdiagnostics-color=always -c modisbits.cpp -o modisbits.o
  C:/rtools40/mingw64/bin/g++ -shared -s -static-libgcc -o luna.dll tmp.def modisbits.o -LF:/Folders/Programs/R-4.1.2/bin/x64 -lR
  installing to C:/Users/admin/AppData/Local/Temp/RtmpAbdlqb/Rinst466c5f3713ce/00LOCK-luna/00new/luna/libs/x64
  ** R
  ** inst
  ** byte-compile and prepare package for lazy loading
  Error in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]) :
  namespace 'terra' 1.4-22 is being loaded, but >= 1.5.3 is required
  Calls: ... withCallingHandlers -> loadNamespace -> namespaceImport -> loadNamespace
  Execution halted
  ERROR: lazy loading failed for package 'luna'
• removing 'C:/Users/admin/AppData/Local/Temp/RtmpAbdlqb/Rinst466c5f3713ce/luna'
  -----------------------------------
  ERROR: package installation failed
  Error: Failed to install 'luna' from GitHub:
  System command 'Rcmd.exe' failed, exit status: 1, stdout & stderr were printed`

Session Info:

`> sessionInfo()
R version 4.1.2 (2021-11-01)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)

Matrix products: default

locale:
[1] LC_COLLATE=English_Kenya.1252 LC_CTYPE=English_Kenya.1252 LC_MONETARY=English_Kenya.1252
[4] LC_NUMERIC=C LC_TIME=English_Kenya.1252

attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base

other attached packages:
[1] fasterize_1.0.3 doParallel_1.0.16 iterators_1.0.13 foreach_1.5.1 tmaptools_3.1-1
[6] maditr_0.8.2 geodata_0.3-5 terra_1.4-22 tmap_3.3-2 basf_0.2.0
[11] sf_1.0-5 data.table_1.14.2 pacman_0.5.1

loaded via a namespace (and not attached):
[1] Rcpp_1.0.7 lattice_0.20-45 prettyunits_1.1.1 ps_1.6.0 png_0.1-7
[6] class_7.3-19 rprojroot_2.0.2 digest_0.6.29 utf8_1.2.2 V8_3.6.0
[11] R6_2.5.1 e1071_1.7-9 ggplot2_3.3.5 pillar_1.6.4 rlang_0.4.12
[16] curl_4.3.2 geojsonlint_0.4.0 callr_3.7.0 raster_3.5-9 htmlwidgets_1.5.4
[21] munsell_0.5.0 proxy_0.4-26 compiler_4.1.2 rmapshaper_0.4.5 pkgconfig_2.0.3
[26] base64enc_0.1-3 pkgbuild_1.3.1 htmltools_0.5.2 tibble_3.1.6 httpcode_0.3.0
[31] codetools_0.2-18 XML_3.99-0.8 jsonvalidate_1.3.2 fansi_0.5.0 viridisLite_0.4.0
[36] withr_2.4.3 crayon_1.4.2 crul_1.2.0 grid_4.1.2 jsonlite_1.7.2
[41] lwgeom_0.2-8 gtable_0.3.0 lifecycle_1.0.1 DBI_1.1.2 magrittr_2.0.1
[46] units_0.7-2 scales_1.1.1 KernSmooth_2.23-20 cli_3.1.0 leafsync_0.1.0
[51] remotes_2.4.2 leaflet_2.0.4.1 sp_1.4-6 ellipsis_0.3.2 vctrs_0.3.8
[56] RColorBrewer_1.1-2 tools_4.1.2 dichromat_2.0-0 leafem_0.1.6 glue_1.6.0
[61] crosstalk_1.2.0 processx_3.5.2 abind_1.4-5 fastmap_1.1.0 colorspace_2.0-2
[66] stars_0.5-5 classInt_0.4-3 `

Kindly assist. Thanks

Hi, I tried to download MOD09A1 files using getNASA. I only got files of 27bytes and cannot read it with rast() from terra. However, if I download the same file from website they are bigger and I can read it with terra. I followed the steps from https://rspatial.org/modis/modis.pdf.
Could anyone help me?