albertotb / get-gfs Goto Github PK

First of all, Thank you for useful repo :D. But I have got "<class 'pydap.model.StructureType'>' object has no attribute 'shape'" while running this command line : get_gfs_hist.py -t 0 10 -x -10 10 -y -10 10 -c example_conf_hist.json 20200505 00
How can I fix this ?

ps. I created conda environment from your YML file.

Hi, hope you're doing well.

I've trying to download data from the historical server from the following coordinates (-32.77946054543407, -70.9403806772067), so i've been running the next line:

python get_gfs_hist.py -t 0 10 -x -71 -69 -y -33 -31 -c example_conf_hist.json 20200101 00

but i've got the following error:
Downloading 20200101 00... [20200101 00] AttributeError: 'str' object has no attribute 'data'
Traceback (most recent call last):
File "get_gfs_hist.py", line 347, in main
verbose=args.verbose,
File "get_gfs_hist.py", line 234, in save_dataset
for time in time_list
File "get_gfs_hist.py", line 234, in
for time in time_list
File "get_gfs_hist.py", line 79, in get_sequential
for var in dataset
File "get_gfs_hist.py", line 79, in
for var in dataset
AttributeError: 'str' object has no attribute 'data'

So... i was wondering if i can get any help here with this download.
Thanks in advance.

Hi @albertotb ,

Could you please advice what I should do to combat the error message below?

Thanks

(py2) C:\get-gfs-master>conda env create -f environment.yml
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:

• libedit==3.1.20181209=hc058e9b_0
• libstdcxx-ng==9.1.0=hdf63c60_0
• sqlite==3.28.0=h7b6447c_0
• intel-openmp==2019.4=243
• readline==7.0=h7b6447c_5
• python==2.7.16=h9bab390_0
• mkl==2019.4=243
• ncurses==6.1=he6710b0_1
• zlib==1.2.11=h7b6447c_3
• mkl_random==1.0.2=py27hd81dba3_0
• libgcc-ng==9.1.0=hdf63c60_0
• numpy-base==1.16.4=py27hde5b4d6_0
• pandas==0.24.2=py27he6710b0_0
• libgfortran-ng==7.3.0=hdf63c60_0
• libffi==3.2.1=hd88cf55_4
• tk==8.6.8=hbc83047_0
• openssl==1.1.1c=h7b6447c_1
• numpy==1.16.4=py27h7e9f1db_0
• mkl_fft==1.0.12=py27ha843d7b_0

Dear Alberto,
I have used your notebook to extract wind and pressure data from:
gfs = f"https://nomads.ncep.noaa.gov/dods/gfs_0p25_1hr"
I have produced a netcdf file with u10m, v10m and prmsl (pressure at sea surface level) using:
with xr.open_dataset(url) as ds:
(
ds[var1]
.isel(time=slice(*time))
.sel(lat=slice(*lat), lon=lon)
.to_netcdf(f"{date}_{run:02d}_uv10prmsl.nc")
)

I woiuld like to add the description of the variables contained in the netcdf file par example:
u10m: x-component of the wind at 10m
u10m: y-componet of the wind at 10m
prmsl: atmospheric pressure at sea surface level.

How to proceed please?

Thank you.

Hi Alberto

Not sure if you are still supporting this, but if you are, there seems to have been a change to the ftp location.

• the path 'https://nomads.ncep.noaa.gov:9090/dods' no longer seems to be valid.
• However you can still get access to the real time server files through 'https://nomads.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/'

Hi Alberto,

Would it possible to download GRIB files instead of CSV's using a similar methodology?

Thanks,
Diego

Dear Alberto,

Could you please help with the following? When testing the real time server script (get_gfs.py) the data is downloaded without any problems.

However, when testing the script for the historical data (get_gfs_hist.py), with the following instruction:

(get-gfs) D:\VLAB\GFS\get-gfs-master>python get_gfs_hist.py -t 0 10 -x -10 10 -y -10 10 -c example_conf_hist.json 20191005 00 -v

I'm getting the following error, related to the 'shape' attribute. Have you ever seen this error? I tried to update the pydap library, but I'm getting the same error.

Thanks in advance!

Regards,

Diego Souza

[20191005 00] AttributeError: '<class 'pydap.model.StructureType'>' object has no attribute 'shape'
Traceback (most recent call last):
  File "D:\Users\dsouza\miniconda3\envs\get-gfs\lib\site-packages\pydap\model.py", line 382, in __getattr__
    return self[attr]
  File "D:\Users\dsouza\miniconda3\envs\get-gfs\lib\site-packages\pydap\model.py", line 425, in __getitem__
    return self._getitem_string(key)
  File "D:\Users\dsouza\miniconda3\envs\get-gfs\lib\site-packages\pydap\model.py", line 404, in _getitem_string
    return self._dict[quote(key)]
KeyError: 'shape'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Users\dsouza\miniconda3\envs\get-gfs\lib\site-packages\pydap\model.py", line 235, in __getattr__
    return self.attributes[attr]
KeyError: 'shape'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\VLAB\GFS\get-gfs-master\get_gfs_hist.py", line 233, in main
    save_dataset(hour, date, var_config, args.time, args.lat, args.lon, fname, verbose=args.verbose)
  File "D:\VLAB\GFS\get-gfs-master\get_gfs_hist.py", line 170, in save_dataset
    data_list = [get_general(file, time, var_config, lat_idx,
  File "D:\VLAB\GFS\get-gfs-master\get_gfs_hist.py", line 170, in <listcomp>
    data_list = [get_general(file, time, var_config, lat_idx,
  File "D:\VLAB\GFS\get-gfs-master\get_gfs_hist.py", line 105, in get_general
    var_data = [ pd.DataFrame((np.concatenate((var_w.data, var_e.data),
  File "D:\VLAB\GFS\get-gfs-master\get_gfs_hist.py", line 106, in <listcomp>
    axis=len(var_w.shape)-1)
  File "D:\Users\dsouza\miniconda3\envs\get-gfs\lib\site-packages\pydap\model.py", line 384, in __getattr__
    return DapType.__getattr__(self, attr)
  File "D:\Users\dsouza\miniconda3\envs\get-gfs\lib\site-packages\pydap\model.py", line 237, in __getattr__
    raise AttributeError(
AttributeError: '<class 'pydap.model.StructureType'>' object has no attribute 'shape'

Hi, I've been trying to install and run this but haven't had any luck.

• I'm using Pycharm w Anaconda on a Windows 10 pc,
• I've created a new project
• I've downloaded all the files from github and placed them in the project folder
• I've created and activated the conda env using environment.yml

But after that I'm lost.

• I've tried typing the ./get_gfs.py ... into the terminal, but that didn't like the './'
• I've tried without the './' in terminal, but it didn't seem to do anything
• I've tried running the get_gfs script but its got no inputs

Any chance for a more detailed how to guide for beginners? would be much appreciated

Hi, I am trying to create an algorithm that will check every day if the 00 o'clock weather has already come out, select the parameters, trim to a certain spatial range and download it. For this I would like to use this source: https://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_0p25_1hr.pl

Could I hope for some simple example

Dear @albertotb , Dear All

I have installed the get-gfs and I lauched the commande line:
/get_gfs.py -s 1 -r 0.25 -t 0 48 -x 9 11 -y 35 37 -p 0 2 -c example_conf.json 20240530 00

I got a file "20240530_00"

What is the format of this file?
How we can obtain a netcdf or grib format file?

Thank you.