ufs-community / ufs_utils Goto Github PK
View Code? Open in Web Editor NEWUtilities for the NCEP models.
License: Other
Utilities for the NCEP models.
License: Other
Update to read in the monthly aerosol climatology, time interpolate to the run time, then horizontally and vertically interpolate it to the model grid.
The climo file is QNWFA_QNIFA_SIGMA_MONTHLY.dat.nc and located on Hera:
/scratch1/BMC/gsd-fv3-dev/Judy.K.Henderson/test/gw_ccpp_v16b/sorc/aeroconv.fd/INPUT
In August, Jeff B. found that chgres_cube
can fail at model points near the poles when using RAP data as input. He fixed it by changing the pole method from ESMF_POLEMETHOD_NONE
to ESMF_POLEMETHOD_ALLAVG
in the call to ESMF_FieldRegridStore
(in routine surface.F90). Jim A. is having the same problem when using chgres_cube
to initialize a C768 nested run using gaussian nemsio input data.
This problem likely also exists in the sfc_climo_gen
program.
There are some strange missing values (see the 0.9969E+37 values below) in the two netcdf coordinate variables (lon and lat) in the orography files for the regional FV3SAR and HAFS configurations, e.g., C768_oro_data.tile7.halo0.nc. The geolon and geolat variables, however, look normal.
Not sure whether these missing coordinate lon/lat values matter or not. But it might be better to fill them with reasonable values or just use npx, npy as coordinate variables.
BTW, when checking the lon, lat values in the sfc_data.tile7.nc and gfs_data.tile7.nc files (generated by chgres), they all look normal though.
Variable: f
Type: file
filename: oro_data
path: oro_data.nc
file global attributes:
dimensions:
lon = 2560
lat = 2160
variables:
float lon ( lon )
cartesian_axis : X
float lat ( lat )
cartesian_axis : Y
float geolon ( lat, lon )
float geolat ( lat, lon )
float slmsk ( lat, lon )
float land_frac ( lat, lon )
float orog_raw ( lat, lon )
float orog_filt ( lat, lon )
float stddev ( lat, lon )
float convexity ( lat, lon )
float oa1 ( lat, lon )
float oa2 ( lat, lon )
float oa3 ( lat, lon )
float oa4 ( lat, lon )
float ol1 ( lat, lon )
float ol2 ( lat, lon )
float ol3 ( lat, lon )
float ol4 ( lat, lon )
float theta ( lat, lon )
float gamma ( lat, lon )
float sigma ( lat, lon )
float elvmax ( lat, lon )
Coordinates:
lon:
337.1 337.1 337.0 337.0 337.0 336.9
336.9 336.9 336.9 336.8 336.8 336.8
336.8 336.7 336.7 336.7 336.7 336.6
336.6 336.6 336.6 336.5 336.5 336.5
336.4 336.4 336.4 336.4 336.3 336.3
336.3 336.3 336.2 336.2 336.2 336.2
336.1 336.1 336.1 336.0 336.0 336.0
...
246.3 246.3 246.3 246.2 246.2 246.2
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
...
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
lat:
44.88 44.86 44.84 44.82 44.79 44.77
44.75 44.73 44.70 44.68 44.66 44.64
44.61 44.59 44.57 44.54 44.52 44.50
44.48 44.45 44.43 44.41 44.39 44.36
...
-6.162 -6.186 -6.209 -6.233 -6.256 -6.280
-6.303 -6.327 -6.350 -6.374 -6.398 -6.421
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37 0.9969E+37
This issue/task is to import the regional_grid capability into the UFS_UTILS repository.
The regional_grid capability is Jim Purser's method to create a much more uniform (in terms of cell size) regional grid for the FV3 standalone regional (SAR) configuration.
Port the repo to the new Orion machine.
Also, remove all references to WCOSS-Phase 1/2, which is being turned off by the end of April.
NCEPLIBS was recompiled and moved to a new location on Jet in mid-November, and ufs_utils no longer compiles.
In sorc/chgres_cube.fd/CMakeLists.txt AVX2 specific flag is set.
This is not portable. For example on Hera I see a lot of warnings like:
../preproc/sorc/chgres_cube.fd/model_grid.F90(141): remark #15009: model_grid_mp_define_input_grid_gaussian_ has been targeted for automatic cpu dispatch
../preproc/sorc/chgres_cube.fd/model_grid.F90(582): remark #15009: model_grid_mp_define_input_grid_gfs_grib2_ has been targeted for automatic cpu dispatch
../preproc/sorc/chgres_cube.fd/model_grid.F90(370): remark #15009: model_grid_mp_define_input_grid_mosaic_ has been targeted for automatic cpu dispatch
We should not use cpu specific flags unless we are cross-compiling, for example on WCOSS Cray or Jet, and in such specific cases those flags should be added conditionally based on user specified option.
The CHGRES packed with NCEPLIBS (github.com:NOAA-EMC/UFS_UTILS.git, @8b8db58, Nov 11 19:08:52 2019) triggers error when it tries to process GFS data.
- CALL FieldScatter FOR INPUT GRID TEMPERATURE.
- FATAL ERROR: READING TEMPERATURE RECORD.
- IOSTAT IS: -31
MPT ERROR: Rank 0(g:0) received signal SIGSEGV(11).
Process ID: 43767, Host: r12i0n32, Program: /glade/scratch/turuncu/ufs-mrweather-app-workflow/bld/chgres_cube.exe
MPT Version: HPE MPT 2.19 02/23/19 05:30:09
and the error trace is
MPT: rc=<error reading variable: Cannot access memory at address 0x0>,
MPT: .tmp.STRING.len_V$7=14336928)
MPT: at /glade/work/turuncu/UFS/NCEP_LIBS_ALL/UFS_UTILS/sorc/chgres_cube.fd/utils.f90:11
MPT: #8 0x00000000004590fa in input_data::read_input_atm_gaussian_file (localpet=0)
MPT: at /glade/work/turuncu/UFS/NCEP_LIBS_ALL/UFS_UTILS/sorc/chgres_cube.fd/input_data.F90:1254
MPT: #9 0x0000000000435a5f in input_data::read_input_atm_data (localpet=0)
MPT: at /glade/work/turuncu/UFS/NCEP_LIBS_ALL/UFS_UTILS/sorc/chgres_cube.fd/input_data.F90:147
MPT: #10 0x000000000041092a in atmosphere::atmosphere_driver (localpet=0)
MPT: at /glade/work/turuncu/UFS/NCEP_LIBS_ALL/UFS_UTILS/sorc/chgres_cube.fd/atmosphere.F90:145
MPT: #11 0x0000000000435839 in chgres ()
MPT: at /glade/work/turuncu/UFS/NCEP_LIBS_ALL/UFS_UTILS/sorc/chgres_cube.fd/chgres.F90:78
The same data and the namelist can be processed with the external CHGRES installation used in the prototype system. This version has last commit from George Gayno on Fri Sep 6 10:29:16 2019 and hash is @947145c.
I also test to install NCEP LIBS with Intel 18.0.5 and system provided NetCDF (4.7.1) but it does not help.
The following four CMakeLists.txt files support Intel only. This needs to be fixed asap.
sorc/global_chgres.fd/CMakeLists.txt:
sorc/chgres_cube.fd/CMakeLists.txt:
sorc/emcsfc_ice_blend.fd/CMakeLists.txt:
sorc/emcsfc_snow2mdl.fd/CMakeLists.txt:
See also #43 for a first mention of this issue (but not for a solution).
Hera is the replacement for Theia. As such, this repo must be ported and tested on that platform.
UFS_UTILS is the only submodule of the NCEPLIBS umbrella build that uses a branch name release/ufs_release_1.0
for the UFS public release branch. All others just have ufs_release_1.0
. Should this be changed, i.e, should a branch ufs_release_1.0
be created from release/ufs_release_1.0
and the latter then deleted? Note that in this case the .gitmodules
file in the NCEPLIBS umbrella repository should be updated (because it contains the branch name).
See also NOAA-EMC/NCEPLIBS#12.
${NETCDF_INCLUDES}
${MPI_Fortran_INCLUDE_PATH}
${NETCDF_INCLUDES_F90}
are specified twice as include directories:
Here: https://github.com/NOAA-EMC/UFS_UTILS/blob/8934efa6474d95ff0dbf5764f26d3389d85e7bf5/sorc/chgres_cube.fd/CMakeLists.txt#L20 and here
https://github.com/NOAA-EMC/UFS_UTILS/blob/8934efa6474d95ff0dbf5764f26d3389d85e7bf5/sorc/chgres_cube.fd/CMakeLists.txt#L33
I think these two target_include_directories
can be merged.
We have received a request for documentation of chgres_cube output files (gfs_data.nc, sfc_data.nc, and gfs_bndy*.nc files), including the fields contained within. The fields themselves (at least in the SAR) are a function of the external model and the suite definition file used by CCPP.
The current suite of CHGRES_CUBE regression tests does not test the new GRIB2 option. Add a GRIB2 test.
In the current NSST foundation temperature (Tf) analysis, a few sea ice related issues need to be resolved:
Tf analysis for the water-ice mixed grids (ice concentration > 15%)
The Tf evolution in prediction mode due to two SST climatology updates: (1) Seasonal tendency; (2) Relaxation to climatology
In this ticket,
During a recent update the 'develop' (3ad7d83), the load of 'prod_util' was inadvertently removed. As a result, NDATE is undefined in the GDAS initialization utility scripts. NDATE is used when processing GFS v15 data.
Regression tests currently exist for chgres_cube
. Tests are needed for other repository components. In particular: global_cycle
, the grid generation programs, and the emcsfc
programs.
The nemsio_cvt
utility may be used to query the file 'endianness' of a nemsio file, and to change the 'endianness' of a nemsio file. The latter function is not working. After talking with Jun, we decided it is not worth fixing if no one is using it.
The global-workflow, regional-workflow and UPP groups have verified that they do not use this utility. I can do a search in our operational log files. If it not used, then it should be removed from the repository.
Support GEFS V12.
Per request from GEFS group, create 'release' branch from 1b76994 of 'develop'
v1 of the public release used CMake to build chgres_cube
. Use CMake to build all programs in the repository. Remove the old build system.
Support GFS v16.
sorc/sfc_climo_gen.fd is incomplete. And since it's not compiled in release/public-v1 branch it should be removed.
Julie Schramm reported a problem with the snow field for a regional grid. The odd snow values were occurring at open water points. I was able to reproduce the problem using the head of 'develop' (4643e2c). I also noticed bad ice depth values at land points.
I ran a quick test that zeroes out these surface fields after the call to FieldCreate. That appears to have fixed it. To be safe, I will initialize all surface fields after FieldCreate to a proper value.
Dusan used chgres_cube to initialize a C96 global run using FV3GFS gaussian nemsio data as input (2019111600 cycle). The model broke during on the first time step during a computation of surface stress. In his words:
I recompiled the model with debug flags and found that model crashes in
the very first physics step, in moninedmf.f (subroutine hedmf_run) at this line:
411 do i=1,im
412 ustar(i) = sqrt(stress(i))
413 enddo
At one point stress(i) is negative number. It is computed in GFS_surface_composites.F90 here:
472 stress(i) = cice(i)*stress_ice(i) + (one-cice(i))*stress_ocn(i) !<--- cice(i) must not be > 1.0
At that point, the sea ice fraction was slightly more than one (1.0000004). When he reset that point to exactly one, the model ran with no issues.
Determine why this is happening in chgres and add checks to prevent it.
Current using two UFS_UTILS branches for all functionality (dtc/develop and feature/chgres_grib2 in https://github.com/NCAR/UFS_UTILS)
set FNMSKH=${FNMSKH:-${FIXam}/global_slmask.t1534.3072.1536.grb}
instead of
FNMSKH=${FNMSKH:-${FIXam}/seaice_newland.grb}
to use high-res land-sea mask in global_chgres.sh and global_cycle.sh for interpolation of climatological SST etc.
This is needed to reduce lake temperature biases for GFS.v16 development.
Each of these utilities is envisioned to be part of UFS_UTILS prior to the SRW App Release (list below still needs to be firmed up). Given that, each utility should have a chapter in the UFS-UTILS documentation to be hosted through ReadtheDocs.
make_hgrid
orog
regional_grid
global_equiv_resol
shave
filter_topo
chgres_cube: already exists - simply need to update as necessary for SAR
GFS v16 will output gaussian history files in netcdf format instead of the current nemsio format. Update chgres_cube accordingly.
To facilitate building on multiple platforms, explore using CMake and Spack.
/home/dusan/simple-ufs/src/preproc/sorc/chgres_cube.fd/write_data.F90:1167:31:
1167 | WRITE(OUTFILE, '(A, I1, A)'), 'out.atm.tile', tile, '.nc'
| 1
Warning: Legacy Extension: Comma before i/o item list at (1)
/home/dusan/simple-ufs/src/preproc/sorc/chgres_cube.fd/write_data.F90:1637:33:
1637 | WRITE(OUTFILE, '(A, I1, A)'), 'out.sfc.tile', tile, '.nc'
| 1
Warning: Legacy Extension: Comma before i/o item list at (1)
[100%] Linking Fortran executable chgres_cube.exe
chgres should include metadata in it's output to identify the source of it's input, this data should be enough to reproduce the contents of the file.
Russ had problems running the model when using tiled, warm restart data as input to chgres. He noticed large vertical velocities in the log file. He wrote:
The C384L127 forecast from the chgres'd C768L127 files failed.
I noticed the following in the forecast job log file:
W max = 3842.734 min = -2243.542
Before adi: W max = 3842.734 min = -2243.542
Because chgres works with both the old spectral model (which output omega) and fv3 (which outputs 'w'), logic was added to the model to convert from omega to 'w' depending on the source of the chgres'd data. The source is identified via a global attribute. The conversion is done in routine routine external_ic.F90. The problem is that the logic does thinks data from warm restart files is 'omega', not 'w'. After discussing with Fanglin and Jun, it was decided to have chgres zero out 'w' for tiled input data instead of updating routine external_ic.F90. According to Fanglin:
Changing resolution from high-res tiles to low_res tiles may produce
strong w that the low-res model can’t sustain. I think it is better to zero it out.
When I run make install
I get the following files in the install directory:
my_install_dir/
├── bin
│ └── chgres_cube.exe
└── include_4
└── include_4
├── atmosphere.mod
├── grib2_util.mod
├── input_data.mod
├── model_grid.mod
├── program_setup.mod
├── search_util.mod
├── static_data.mod
└── surface.mod
3 directories, 9 files
There is no need for include_4 directory under include_4.
In fact, all those .mod files are not necessary in the install tree. They are internal to chgres_cube, and chgres_cube is not a library.
I suggest the following lines to be removed from sorc/chgres_cube.fd/CMakeLists.txt
:
The GDAS initialization scripts must be updated to support v16 cycled experiments - with and without the Incremental Analysis Update (IAU).
A license and other documents were added to the public release branch/tag. Add these documents to 'develop'.
Seeing odd behavior when running in regional mode. My comment from a previous issue (#91).
Discovered when running the grid_gen regression test from develop (270f9dc). It happens with the 'regional' regression test. If I fix the rank of phis
in routine FV3_zs_filter (should be 3, not 4) and print out the value of phis(1,1), I get this difference in file C96_oro_data.tile7.halo4.nc when compared the regression test baseline:
Variable Group Count Sum AbsSum Min Max Range Mean StdDev
orog_filt / 396 103.033 318.697 -25.4385 26.2191 51.6576 0.260184 2.97099
If I add another print statement for phis(1,2), I get this difference:
Variable Group Count Sum AbsSum Min Max Range Mean StdDev
orog_filt / 370 -0.375389 68.313 -2.69452 3.75537 6.44989 -0.00101457 0.484862
So there is likely some kind of memory leak going on with the 'regional' option.
Originally posted by @GeorgeGayno-NOAA in #91 (comment)
On June 1st, the directory name 'project' was changed to 'HFIP'.
On June 30th, the /lfs3 disk will be replace by /lfs4.
Update the repo as required.
The NEMSIO utilities such as nemsio_read
, nemsio_get
, nemsio_chgdate
, mkgfsnemsioctl
belong with the NCEPLIBS-nemsio as a change to the library API most certainly will need an update to these utilities.
Associated ctests
should be added to ensure these utilities and the library are in agreement.
Matt Pyle requested a release branch to support HREFv3.
HREF uses chgres_cube.
During testing for the public release, the model failed (in debug mode) when using coldstart files prepared from GFS GRIB2 files. The failure - in the nsst_module - indicates a too cold SST. Investigate.
See this issue for more details: ufs-community/ufs-mrweather-app#87
Adding lake maker from Shan Sun and Ning Wang: generate lake fraction and depth on the FV3 grid, add them to oro_data, and adjust land_frac and slmsk, such that land_frac and slmsk are consistent with lake_frac.
The nst_tf_chg program does not work with GFS v15 nemsio files. When I try running it with the 'fv3gfs_chgres.sh' script, I get errors:
ERROR: the size of recname is not equal to the total number of the fields in the file!
readrecv for gfilei for zm Status = -42
writerecv for gfileo for zm Status = -73
readrecv for gfilei for ifd Status = -42
writerecv for gfileo for ifd Status = -73
The program will also need to work with GFS v16 gaussian netcdf files.
When I compile chgres_cube using debug compile flags (on Hera using Intel compiler), I get the following syntax errors:
/scratch2/NCEPDEV/fv3-cam/Dusan.Jovic/sufs/simple-ufs-aero/src/preproc/sorc/chgres_cube.fd/program_setup.f90(280): error #6631: A non-optional actual argument must be present when invoking a procedure with an explicit interface. [RC]
call error_handler("FOR GRIB2 DATA, PLEASE PROVIDE GRIB2_FILE_INPUT_GRID")
----------^
compilation aborted for /scratch2/NCEPDEV/fv3-cam/Dusan.Jovic/sufs/simple-ufs-aero/src/preproc/sorc/chgres_cube.fd/program_setup.f90 (code 1)
and
/scratch2/NCEPDEV/fv3-cam/Dusan.Jovic/sufs/simple-ufs-aero/src/preproc/sorc/chgres_cube.fd/input_data.F90(5532): error #6631: A non-optional actual argument must be present when invoking a procedure with an explic
it interface. [RC]
call error_handler("ERROR USING MISSING_VAR_METHOD. PLEASE SET VALUES IN" // &
---------^
/scratch2/NCEPDEV/fv3-cam/Dusan.Jovic/sufs/simple-ufs-aero/src/preproc/sorc/chgres_cube.fd/input_data.F90(5564): error #6631: A non-optional actual argument must be present when invoking a procedure with an explic
it interface. [RC]
call error_handler("reading soil levels. File must have 4 soil levels.")
---------^
compilation aborted for /scratch2/NCEPDEV/fv3-cam/Dusan.Jovic/sufs/simple-ufs-aero/src/preproc/sorc/chgres_cube.fd/input_data.F90 (code 1)
Users are asking how to coldstart a full global parallel (80 enkf members and the high res member) using FV3GFS GDAS data. The only GDAS data available in HPSS is tiled restart files. Therefore, the fv3gfs_chgres.sh
script, which runs the old serial chgres, will not work. Need to come up with a replacement to that script that uses chgres_cube. The script could also be designed for users who only want to run a free forecast.
Create regression tests for chgres_cube. At a minimum, the tests should...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.