Difference lat-lon maps only show half for several obs datasets about e3sm_diags HOT 7 OPEN

@xylar I'm glad to report: your fix conda-forge/cdms2-feedstock#87 works, and thanks for adding a check for testing conda-forge/cdms2-feedstock#89.
With esmfy correctly imported by cdms, the regriding behavior gets back to normal.

from e3sm_diags.

After some troubleshooting, it ruled out change in cartopy as the cause. Looking at the intermediate files being saved, data already missing half after regridding. It suggests regriding with changes in cdms/esmf is the cause. I tried to roll back to older version/builds of cdms and esmf, but ran into numpy, alias deprecation problems. Pinning for old version is not ideal for coordinating with e3sm-unified. For a immediate solution, it might make sense to convert longitude from [-180, 180) to [0, 360) online or offline for input dateset being impacted to work around this problem.

from e3sm_diags.

Found an old cdms issue pointing out the same behavior.
CDAT/cdms#371
I'm following the approach suggested in this post to wrap the longitude with [0,360), since cdms somehow failed to do it upon open datasets.

from e3sm_diags.

The regriding step also produce warning as follows:

/global/cfs/cdirs/e3sm/zhang40/conda_envs/edv290_numpy/lib/python3.10/site-packages/cdms2/avariable.py:1289: Warning:
avariable.regrid: regridTool = 'esmf' but your version does not
seems to be built with esmf, will switch to regridTool = 'libcf'

  warnings.warn(message, Warning)

It seems cdms is not properly built with esmf. Not sure if this is relevant to the regriding issue here.

from e3sm_diags.

@chengzhuzhang, I'm going to see if conda-forge/cdms2-feedstock#87 fixes this. I patched the imports in cdms2 to hopefully bring in esmpy, since the package has been renamed since v8.4.0 from ESMF to esmpy.

from e3sm_diags.

@chengzhuzhang, could you add a test somewhere in E3SM diags where you test for ESMFRegrid? It could just be as simple as:

from regrid2 import ESMFRegrid

but you might want to have a try/except ImportError where you give an explanation about this meaning that an incompatible version of cdsm2 is installed that is not able to import ESMPy.

Such an error would tell us right away that there's a problem, rather than noticing it only when plots are half missing.

from e3sm_diags.

Great suggestion. I think it makes sense to add it as one of the CI/CD tests.

from e3sm_diags.