render MOM6 example with correct mesh geometry about pangeo-example-notebooks HOT 9 CLOSED

With your new data, this works:

qm = hv.QuadMesh((ds.geolon[0].values, ds.geolat[0].values, ds.SST[0].values))
qm

from pangeo-example-notebooks.

The absence of the 2D coordinates and mesh data in the output files is very much an annoyance. It is possible for users to specify to add this data but they have to do so explicitly for each and every file which is very cumbersome. However, we do normally write the 2D coordinates and mesh to a "static" data file (in the above example this is referred to by the associated_files attribute). The fragile part of this is that the data and coordinates are in separate files and prone to be disconnected or mixed up.

I totally agree that we need to fix the data first. The short-term solution is to add the 2D mesh data to the same files using the cumbersome approach. In the longer term, I think we need a way to indicate within the netCDF file which variables are the associated mesh coordinates. Currently the CF convention has an attribute "coordinates" which is explicitly for cell center coordinates. I think we should propose to CF an attribute such as "mesh_coordinates" to indicate which data defines the grid for each variable. (This proposal is off thread so I will take it up elsewhere).

@gustavo-marques How hard would it be for you to add geolon, geolat, geolon_c, geolat_c to each of the files in the diag_table that you use for this example?

from pangeo-example-notebooks.

Thanks @adcroft for catching this and thanks @rabernat for proposing a solution to it. I will look into fix the problem (hopefully today) by either 1) uploading the directories to the existing zarr group; or 2) adding the lat/lon arrays into the netCDF files.

from pangeo-example-notebooks.

I've 2D added lon/lat arrays, but open_zarr interprets them as variables:

I tried append/concat, but this is still an open issue. @rabernat, any ideas?

Another approach is to follow @adcroft 's suggestion, but for that I will have to re-run the model since the original data has been purged.

from pangeo-example-notebooks.

@gustavo-marques: I guess you have resolved this?

When I open 'pangeo-data/MOM6.cesm/', I now see the correct coordinates. However, the example notebook is now broken because it points to 'pangeo-data/MOM6.CESM/', which no longer exists.

FYI, one work around would have been to directly edit the .zattrs file to specify what the coordinates are:
https://storage.googleapis.com/pangeo-data/MOM6.cesm/.zattrs

from pangeo-example-notebooks.

Actually, there is a problem with the new dataset:

<xarray.Dataset>
Dimensions:       (nv: 2, time: 60, xh: 540, xq: 540, yh: 458, yq: 458, z_i: 35, z_l: 34)
Coordinates:
    geolat        (time, yh, xh) float64 dask.array<shape=(60, 458, 540), chunksize=(1, 458, 540)>
    geolat_u      (time, yh, xq) float64 dask.array<shape=(60, 458, 540), chunksize=(1, 458, 540)>
    geolat_v      (time, yq, xh) float64 dask.array<shape=(60, 458, 540), chunksize=(1, 458, 540)>
    geolon        (time, yh, xh) float64 dask.array<shape=(60, 458, 540), chunksize=(1, 458, 540)>
    geolon_u      (time, yh, xq) float64 dask.array<shape=(60, 458, 540), chunksize=(1, 458, 540)>
    geolon_v      (time, yq, xh) float64 dask.array<shape=(60, 458, 540), chunksize=(1, 458, 540)>
  * nv            (nv) float64 1.0 2.0
  * time          (time) float64 15.5 45.0 74.5 105.0 135.5 166.0 196.5 ...
  * xh            (xh) float64 -286.7 -286.0 -285.3 -284.7 -284.0 -283.3 ...
  * xq            (xq) float64 -286.3 -285.7 -285.0 -284.3 -283.7 -283.0 ...
  * yh            (yh) float64 -79.2 -79.08 -78.95 -78.82 -78.69 -78.56 ...
  * yq            (yq) float64 -79.14 -79.01 -78.89 -78.76 -78.63 -78.49 ...
  * z_i           (z_i) float64 0.0 5.0 15.0 25.0 40.0 62.5 87.5 112.5 137.5 ...
  * z_l           (z_l) float64 2.5 10.0 20.0 32.5 51.25 75.0 100.0 125.0 ...

The geolon coordinates should not have a time dimension!

from pangeo-example-notebooks.

We've recently fixed the values of longitude in the output along the top edge - not sure if that's included here, @gustavo-marques ?

Ideally the dimensions in the dataset would be xq=xh+1, yq=yh+1 but to get this the model needs to be run the model in a different mode. If you zoom in to (100W,90N) I expect there to be a gap? If you plotted ds.SST[0].values[1:,1:] then the gap should disappear? At least, based in pcolormesh behavior.

from pangeo-example-notebooks.

I mistakenly named the new dataset "MOM6.cesm". The time dependence in geolat*/geolon* was introduced when I converted the data to Zarr. I believe my previous fix (manually add the geolat*/geolon*) is a better solution. I will update the dataset in the near future following this fix.

Yes, I can now visualize the data properly (but just snapshots). I am about to send a new PR.
I still have not figure out how to plot the data using Holoviews's interactive tool for exploring changes over time. The issue is that one needs to specify kdims e.g., sst = sst_ds.to(hv.QuadMesh, kdims=["xh", "yh"], dynamic=True). I could not figure out how to pass geolon and geolat instead. Any ideas? It would be really nice to have this feature working properly.

from pangeo-example-notebooks.

@adcroft, the fix you mentioned is not included here. Yes, there is a gap at 100W,90N but it's very small:

If you plotted ds.SST[0].values[1:,1:] then the gap should disappear?

No, it gets worse:

from pangeo-example-notebooks.