timcera / tsgettoolbox Goto Github PK

The xindex/yindex options in ldas have always been problematic. The xindex/yindex in the NLDAS shapefile actually have to have 1 subtracted to be used with the LDAS web services. Haven't found GLDAS shapefile. Maybe just remove so there is no confusion?

What happens with NLDAS is that if you try with a --lat/--lon that appears to be on land, but NLDAS decided that the cell was over ocean, the NLDAS gives a 404 error. Need to perhaps test to see if --lat/--lon is valid?

odo has requirement for networkx > 1 and < 2. Looking at the odo code it supports seamlessly either, so change in 'setup.py'.

Hi

I am trying to download the historical ndbc data and I always get the error that it is a BadGzipFile.

My code is

df=tsgettoolbox.ndbc(station='pfdc1',table='ocean',startUTC='2020-01-01', endUTC='2020-05-01')

The error is
BadGzipFile: Not a gzipped file (b'<!')

Thank You
~A

I see on the GLDAS documentation that Solar Radiation data is also available. However it looks like I cannot download it using tsgettoolbox. I tried using the variable name based on the NLDAS documentation and got this error.

HTTPError: 404 Client Error: Not Found for url: https://hydro1.gesdisc.eosdis.nasa.gov/daac-bin/access/timeseries.cgi?variable=GLDAS2%3AGLDAS_NOAH025_3H_v2.1%3ASwnet_tavg&location=GEOM%3APOINT%2881.84%2C+25.43%29&startDate=2015-01-01T00&endDate=2019-01-31T00&type=asc2

Thanks for creating this! I'm getting all zeros for land cover, in a location (the desert) that should not have zeros. I'm getting the same results in a Jupyter notebook as from terminal.

#Get land cover for point in Sahara (googled "lat long Sahara")
lat = 23.4162
lon = 25.6628
product = "MCD12Q1"

#https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mcd12q1_v006
#Land Cover Type 1: Annual International Geosphere-Biosphere Programme (IGBP) classification
#Valid range 1 to 17
#band = "Land_Cover_Type_1"#this is not recognized by the function; error message indicates accepted bands
band = "LC_Type1"
startdate = "2001-01-01"
enddate = "2002-01-01"
output = tsgettoolbox.modis(lat=lat, 
                            lon=lon, 
                            product=product, 
                            band=band, 
                            startdate=startdate, 
                            enddate=enddate)
output

Datetime,MCD12Q1_LC_Type1:class
2001-01-01,0
2002-01-01,0
2003-01-01,0
2004-01-01,0
2005-01-01,0
2006-01-01,0
2007-01-01,0
2008-01-01,0
2009-01-01,0
2010-01-01,0
2011-01-01,0

I get the same results when I try a few other locations in the world and a few other bands. I also notice that I'm getting back results for dates outside the start and end date I'd indicated. Any advice much appreciated, recognizing this may be user error!

Especially NLDAS. Need to make really clear that tsgettoolbox is not making any time zone modifications, and delivers the time zone of the source organization. Example: NLDAS is always and everywhere UTC, where NWIS is the local time zone of the station.

I was downloading a bunch of NLDAS data and I was running into an inconsistent error.

Example code

    tsget.ldas_nldas_fora(
        lat=29.3125,
        lon=-81.6875,
        variables=[
            "NLDAS_FORA0125_H.002:DLWRFsfc",
            "NLDAS_FORA0125_H.002:DSWRFsfc",
            "NLDAS_FORA0125_H.002:PEVAPsfc",
            "NLDAS_FORA0125_H.002:SPFH2m",         
            "NLDAS_FORA0125_H.002:TMP2m",
        ],
        startDate=str(datetime(1979, 1, 1, 0, 0)),
    )

If I printed out the resp variable I would receive this message
“This server is temporarily unable to service your request due to either high I/O processing or your IP address has reached the limit of concurrent connections. Please review/reduce the number of concurrent connections you are submitting, and try again.\r\nIf you feel you have received this message in error, please contact GES DISC user services”

When I added the max_workers argument=1, then the error went away. Looking into the ar.retrieve_binary function, max_workers defaults to 8.
resp = ar.retrieve_binary(urls, kwds,max_workers=1)

Let me know what you think about this change.

I tried following command

df = tsgettoolbox.ldas(xindex=1031, yindex=360,
                           variable='GLDAS:GLDAS_NOAH025_3H.001:precip')

And got following error

Traceback (most recent call last):

File "<ipython-input-46-be91ad9ffc6c>", line 3, in <module>
    variable='GLDAS:GLDAS_NOAH025_3H.001:precip')

File "C:\ProgramData\Anaconda3\lib\site-packages\tstoolbox\tsutils.py", line 858, in onCall
  validator_func[valid](funcname, argname, nargs, nvar, vlen)

File "C:\ProgramData\Anaconda3\lib\site-packages\tstoolbox\tsutils.py", line 788, in _vrange
  if nargs[1] is None:

IndexError: list index out of range

I am seeing the following error when downloading the instantaneous values.

"Cannot infer dst time from 2019-11-03 01:00:00, try using the 'ambiguous' argument" Depending upon the USGS station, this date changes.

Sincerely
~A

https://cida.usgs.gov/thredds/dodsC/mows/pe.html