Validation of the Climate Analysis Forecast Ensemble dataset (CAFE) often involves comparison against the Australian Gridded Climate Data dataset (AGCD).
This repository contains details of the code, data processing steps,
and software environment used to process the original AGCD dataset
(maintained by the Bureau of Meteorology and hosted at /g/data/zv2/agcd/
)
into a format consistent with the CAFE dataset
(maintained by CSIRO and hosted at /g/data/xv83/dcfp/
).
This essentially involves converting the original AGCD netCDF files
to Zarr format and then regridding to the CAFE spatial grid.
The processed files are stored at:
/g/data/ia39/AGCD/post-processed/data/
The following Python scripts contain the code used to process the AGCD data:
nc_to_zarr.py
: converts the netCDF AGCD data files to zarr formatregrid_agcd_to_cafe.py
: regrids AGCD data to the CAFE grid
The Makefile
contains the instructions to build and run the sequence of commands
that were used to execute the Python scripts and produce each data file:
agcd_v2_precip_total_r005_monthly_1900-2020.zarr.zip
make zarr TIMESCALE=monthly
- Converts the version 2 monthly AGCD data (0.05 degree grid) to zarr format
agcd_v2_precip_total_cafe-grid_monthly_1900-2020.zarr.zip
make regrid TIMESCALE=monthly
- Regrids the monthly AGCD data (0.05 degree grid) to the CAFE grid
agcd_v1_precip_total_r005_daily_1900-2020.zarr.zip
make zarr TIMESCALE=daily
- Converts the version 2 daily AGCD data (0.05 degree grid) to zarr format
agcd_v1_precip_total_cafe-grid_daily_1900-2020.zarr.zip
make regrid TIMESCALE=daily
- Regrids the daily AGCD data (0.05 degree grid) to the CAFE grid
To find out exactly what commands were executed,
run any of those make commands with the -n
(dry run) and -B
(force make)
options (e.g. make -n -B make zarr TIMESCALE=monthly
)
or open any of the data files and look at the global history attribute.
The following Python libraries are dependencies for running the code:
xarray
, netCDF4
, dask
, xesmf
, gitpython
and cmdline_provenance
.
When the Makefile
executes a Python command it uses the conda environment
at /g/data/e14/dbi599/miniconda3/envs/agcd
,
which has all the dependencies (and their dependencies) installed.
The complete list of packages installed in that environment is shown in environment.yml
.
If you'd like to recreate that exact software environment
(you don't need to in order to run the Makefile
),
run the following at the command line:
$ conda env create -f environment.yml
Questions or comments are welcome at the GitHub repostory
associated with the code:
https://github.com/AusClimateService/agcd/issues