ValueError while processing rotated lat/lon .nc files with 365 calander days and 0-360 lon about easymore HOT 10 CLOSED

Thank you very much for posting this issue,

This is an interesting case.

To make the EASYMORE v1.0.0 works with your data, lon values should be shifted by 180 degrees, so perhaps you need to have the lon = lon - 180 without any condition right after correcting for the discontinuity in the grid that you have correctly fixed: ncap2 -O -s 'where(lon>180) lon=lon-360' input.nc output.nc. After this, the remapping should work fine. Note the visualization might not work for this calendar and this will be fixed in the next version 1.1.0.

Now back to why all these happened:

1. EASYMORE assumes the adjacent grids are having continuous lon values. In case there are jumps, such as 0 to 360, or -180 to 180, EASYMORE asks first to either correct the lon or subset the source nc file to make sure the continuity of lon values is assured. Of course, a flag can be set so the user can ask EASYMORE to fix this, however, as more complex cases may occur, it was decided to leave this to the user. I would be interested to know your opinion. A few lines of code and a flag can be added to take care of this.

2. Calendar issues: you have fixed this issue. It will be fixed in the next version release. Note that the visualization may not work.

3. Issue with correction of regions: The error you received seems to be due to the fact that the source shapefile has no intersection with 0 to 180 region while it is intersecting with 180 to 360 region. The assumption of the code was that the intersection with the non-existing region would return an empty dataframe. However, it seems the intersection is not working as expected. This can be fixed with the solution I gave above temporarily (lon= lon -180). In the next version, checks will be added to make sure the shapefiles are falling in the regions and then perform intersection. This was not the case for the GRACE example, in the example on GitHub, as GRACE had both regions (0 to 180 and 180 to 360).

I would appreciate it if you can pass me the original nc file that you were working on initially (before correcting for values larger than 180 by subtracting 360 degrees). I can create a case based on this and it would be a good example to explore and add to the list of examples.

Thank you again and hope the above is helpful.

from easymore.

1. I think a flag that can be set by the user would be very helpful so I would appreciate that being an option.

The Can_pr_crop_time.nc file I provided should work as we just shortened the time step as the OG is for 150 years and did not adjust the lon values, but I can get you the original .nc file on Monday if you still want it.

Thanks for the reply!

from easymore.

The original nc file would be an asset as I can implement the flag and test to see how it works, as you did it rightly just a few time steps would be enough to test. Thank you.

from easymore.

version v1.1.0 is located on the develop branch, I did not yet test it. As soon as I have the source nc file I can give it a try...
the new version can:
1- add or deduce values from the lon values,
2- the issue with empty shapefiles when correcting is hopefully resolved
3- the issue with calendars is resolved (while the visualization may not work and need more further changes)
the new version can be installed following:

rm -rf EASYMORE # remove existing EASYMORE folder
git clone https://github.com/ShervanGharari/EASYMORE.git
git checkout develop
cd EASYMORE
pip uninstall easymore # remove old version
pip install .

make sure version v1.1.0 is installed and also initiated when the code is executed.

in your setup, you should add a flag like

esmr.source_cor_lon_flag       =  True
esmr.source_cor_lon_param      =  ['greater',360, -180] # values greater than 360 are summed by -180 (or deducted by 180)

from easymore.

Here is the .nc file that we used. The name has been changed, but it's the same file. I had to upload it to google drive as the file is very large.

https://drive.google.com/drive/folders/1fMk5LbXOvg7gKEusX_V820yr9vHfb2_l?usp=sharing

from easymore.

Thank you very much. The file seems to be very large! Could you slice only a few days or a month from the file and share it? I can do it on my side, but if you have the daily or monthly files, a smaller size, I would appreciate it if you can share them.

from easymore.

This should work
(https://github.com/ShervanGharari/EASYMORE/files/11773684/Can_pr_crop_time.zip

from easymore.

I have added an example for your case.
The visualization is still not working well for calendar 365 and also a mismatch between the source NetCDF file (that is between 180 to 360 degrees) and the target shapefile, which is between -180 to 0. I have done a temporary fix in a new visualization function. That is why the second figure is colorless. I'll try to fix this later this month.
Regardless, version v1.1.0 should be good at remapping the variables you are after.
Let me know how it goes!
and thank you!

from easymore.

from easymore.

@AlexGiberson the main branch is updated with version 1.1.0 which should handle your case properly.
Your case is provided in examples/10_Calendar_365.ipynb.
1- The new code should work for Calendar 365, nc_remapper.
2- The new code should work for the visualization of Calendar 365 nc_vis.
After a couple of more changes, I'll be releasing the new version on PyPI and then ComputeCanada wheelhouse, it might take a few more weeks. Meanwhile, you can use the main branch with a local installation. Please test and let me know of probable cases of errors or issues. Thank you in advance.

from easymore.