Comments (6)
Thanks, I would like to look into what goes wrong here, however, your example fails with
Rects$BR_Lat = Locations$Lat
#> Error in Rects$BR_Lat = Locations$Lat : object 'Rects' not found
since Rects
is not defined. Could you please update your example, so that I can run the exact query that you ran?
from getspatialdata.
Sorry about that. Here's a small change to it:
Locations = data.frame(39.9,-105.65)
colnames(Locations)=c("Lat","Lon")
cell_dim = .1
Rects=Locations
Rects$BR_Lat = Locations$Lat
Rects$BR_Lon = Locations$Lon
Rects$TR_Lat = Rects$BR_Lat+cell_dim
Rects$TR_Lon= Rects$BR_Lon
Rects$TL_Lat= Rects$TR_Lat
Rects$TL_Lon= Rects$TR_Lon - cell_dim
Rects$BL_Lat= Rects$BR_Lat
Rects$BL_Lon=Rects$TL_Lon
Rects=Rects[,3:10]
a<-matrix(as.numeric(as.character(Rects[1, ])), ncol=2, byrow=TRUE)
set_aoi(a)
view_aoi() #view AOI in viewer, which will look like this:
start_records = "2020-06-01"
end_records = "2020-07-05"
#records = get_records(time_range = c(start_records, end_records), products = c("LANDSAT_8_C1", "Sentinel-2", "LANDSAT_ETM_C1"))
records = get_records(time_range = c(start_records, end_records), products = c("LANDSAT_8_C1"))
records_filtered <- records[which(records$level %in% c("l1")),] #filter by Level
records_filtered <- records_filtered[as.numeric(records_filtered$cloudcov) <= 70, ] #filter by clouds
#order any data that is not already available
test_for_availability = check_availability(records_filtered)
tryCatch(records_filtered <- order_data(test_for_availability), error = function(e) message("No New Records Ordered: ", e))
#test for all data availability for 60 minutes else proceed with current data
time_waited=0
cycle=60
#try for an hour before just going with all avaialble now
while(!all(test_for_availability$download_available==TRUE && time_waited<3600)){
Sys.sleep(cycle)
test_for_availability = check_availability(records_filtered)
time_waited=time_waited+cycle
}
records_filtered = test_for_availability[test_for_availability$download_available==TRUE,]
records_filtered <-records_filtered[order(records_filtered$cloudcov),]
get_data(records = records_filtered[1,])
get_data(records = records_filtered[2,])
from getspatialdata.
Thanks. When I run your code, view_aoi()
on your AOI results in
#> Warning message:
#> In sf::st_is_longlat(x) :
#> bounding box has potentially an invalid value range for longlat data
The reason appears to be that longitude and latitude have been confused in a<-matrix(as.numeric(as.character(Rects[1, ])), ncol=2, byrow=TRUE)
. When defining the AOI as a matrix, set_aoi
expects the column order to be longitude, latitude (when I run your code, a
has the opposite column order). I am not sure how you were able to query for records with that AOI since this resulted in zero records for me.
This however works fine for me and I do not get a 404
library(getSpatialData)
library(sf)
# using your corner coordinates to define AOI with column order (lon, lat)
aoi <- cbind(c(-105.65, -105.65, -105.75, -105.75), #longitude
c(39.9, 40.0, 40.0, 39.9)) # latitude
aoi <- rbind(aoi, aoi[1,]) # close the polygon
aoi <- st_sfc(st_polygon(list(aoi)), crs = 4326)
set_aoi(aoi)
view_aoi()
set_archive("archive/")
start_records = "2020-06-01"
end_records = "2020-07-05"
login_USGS("user")
records = get_records(time_range = c(start_records, end_records), products = c("LANDSAT_8_C1"))
records <- records[which(records$level %in% c("l1")),] #filter by Level
records <- records[as.numeric(records$cloudcov) <= 70, ] #filter by clouds
#order any data that is not already available
records <- check_availability(records)
get_data(records)
#> Receiving MD5 checksums...
#> Assembling dataset URLs...
#> [Dataset 1/2 | File 1/38] Download dataset 'LC08_L1TP_034032_20200616_20200626_01_T1 (l1)' to
#> '/home/xxxx/xxxxx/tmp/gSD//LANDSAT_8_C1//LC08_L1TP_034032_20200616_20200626_01_T1/LC08_L1TP_034032_20200616_20200626_01_T1_B6.TIF'...
#> |=====================================================================================| 100%
Side note: I see you build a little while loop to check for orders to be ready for download. You can also simply use the argument wait_to_complete = TRUE
in order_data()
to make the function run until the last order is complete.
from getspatialdata.
I used your code and verified I successfully get the data. However, I think this is an issue that has to do with timing.
I changed the aoi and timing slightly to get the most recent Landsat data near me. One I did this I received the same 404:
library(getSpatialData)
library(sf)
# using your corner coordinates to define AOI with column order (lon, lat)
aoi <- cbind(c(-115.65, -115.65, -105.75, -105.75), #longitude
c(39.9, 45.0, 45.0, 39.9)) # latitude
aoi <- rbind(aoi, aoi[1,]) # close the polygon
aoi <- st_sfc(st_polygon(list(aoi)), crs = 4326)
set_aoi(aoi)
view_aoi()
set_archive("archive/")
start_records = "2020-07-13"
end_records = "2020-07-14"
login_USGS("user")
records = get_records(time_range = c(start_records, end_records), products = c("LANDSAT_8_C1"))
records <- records[which(records$level %in% c("l1")),] #filter by Level
records <- records[as.numeric(records$cloudcov) <= 70, ] #filter by clouds
#order any data that is not already available
records <- check_availability(records)
get_data(records)
##>Error in .get(index_url) :
##>Not Found (HTTP 404). Failed to process request.
from getspatialdata.
I've discovered the root cause of this issue. There is a mismatch between the products which are available from USGS versus from AWS. When using awscli, I can verify that products (from 2020-08-03), which are available through NASA Earth Explorer, are not yet available on the aws pds.
AWS states: "New Landsat 8 scenes are added regularly as soon as they are available."
Clearly, this is not very truthful wording from Amazon.
from getspatialdata.
I'll close this issue for now because I don't wish the task of interpreting the Earth Explorer API documentation upon anyone. However, I think the right solution to most problems is avoiding AWS.
from getspatialdata.
Related Issues (20)
- Retrieving Incidence angle of records
- getLandsat_records fails to connect to server (HTTP 500) HOT 1
- Missing values are not allowed in subscripted assignments of data frames.
- getLandsat_records - product group, *tmp* error HOT 2
- error with getLandsat_records HOT 4
- Got the error while trying to install package HOT 2
- dir.create(tmp_dir) : invalid 'path' argument HOT 1
- Error in getSentinel_records
- not able to fetch today.. it was working previously
- Error in getting sentinel records HOT 4
- Landsat download problem: error in evaluating the argument 'y' in selecting a method for function 'intersect': non-character object(s) HOT 1
- USGS EROS API 1.4.0 expired - requests invalid HOT 8
- Sentinel order_data() HOT 2
- get_records(): Some products invalid HOT 2
- Missing "summary" column of 'records' for Landsat products HOT 4
- Why does set_archive() add another string to my archive directory?
- view_previews() error
- login_earthdata() HOT 4
- No Sentinel-3 WFR/WRR data
- Problems with downloading Landsat data
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from getspatialdata.