Not possible to comment due to

Reuse the example looking at map_blocks and provide a performance comparison with using ufunc, outline pros/cons of each approach

Perusing the ESDS Projects website today, it would be great to add something about CUPiD under Diagnostics Efforts.

cc @mnlevy1981 @TeaganKing

Would great to have deploy previews in PRs to help prevent things like #247.

Looks like this is an internal alpha feature of the GitHub deploy-page action, but if recall correctly there's a way to do this.

For now, the search bar only shows up after selecting tags. It would be nice to be able to see this search bar on the left by default

We need to redirect the Office hours links to the sundog site:https://sundog.ucar.edu/Interact/Pages/Content/Document.aspx?id=6160 so that registration is blocked by a Duo push.

Who owns the site domain to be able to set this up? @matt-long ? -- Actually I found a way to do this just with HTML in #196

Using Dask on the New Casper PBS Scheduler — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/posts/2021/casper_pbs_dask/

Reading WRF data into Xarray and Visualizing the Output using hvPlot — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/posts/2021/xarray-wrf-example/

We have Google Analytics added to the Project Pythia portal, and I had Google Analytics on the Xdev site. I think we should add Google Analytics to the ESDS site, too. I'm happy to put together a PR on this since I have the GA project links already created.

As mentioned by @dcherian , it would be helpful to have a contributors guide for adding new blog posts!

Just realized we should probably add this to more than links in the blog (e.g. the comms /resources page might be good).

Plotting CESM Data on an Unstructured Grid using Geoviews and Datashader — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/posts/2021/cesm-datashader/

In updating the webpages, it became clear that we should update our FAQ page to be more user-friendly.

In #255 , I included some linked contents to FAQ sections at the top of the FAQ page that hopefully will make it easier to navigate.

Some additional ideas from @kafitzgerald :

• Include getting started level resources and broader scientific python resources
• Update a few resources and/or point to some that may not have existed when these were last updated

Debugging dask workflows: Detrending — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/posts/2022/dask-debug-detrend/

Batch Processing Jupyter Notebooks with Papermill — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/posts/2022/batch-processing-notebooks-with-papermill/

An action item for the GeoCAT-Xdev meeting was to create an active projects page on the ESDS site.

Some projects to include would be:

• GeoCAT
  • comp
  • viz
• Diagnostics efforts?
• ESDS page?

How to Use xarray.map_blocks for Vertical Interpolation of a 3D Field — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/posts/2021/map_blocks_example/index.html

How to Use xarray.map_blocks for Vertical Interpolation of a 3D Field — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/posts/map_blocks_example/

Thanks @TeaganKing for the nudge to review things on here!

I was out yesterday so didn't have a chance to skim things over then, but noticed a few dated links and items missing from the GeoCAT section under the "Projects" page. We can just make these changes separately though. That'll give me a chance to poke the appropriate people for reviews/input as well.

There is a lot of content in the Xdev repository that has been lost due to the automatic forward of the Xdev GitHub page to the ESDS GitHub page. That is problematic, especially since (in our recent Xdev Team Meetings) the Xdev Team has (themselves) claimed that they are not only members of ESDS. In other words, we have all agreed that (sometimes) Xdev and ESDS are different. (Or, another way of saying it is that Xdev isn't an element inside ESDS, but a separate entity that has substantial overlap with ESDS.) Hence, it makes sense that the two pages should be separate.

I think we should separate the two pages. And I think we should move the Xdev blog posts to the ESDS repo. The rest of the content in the Xdev page is still useful on its own (and I need to work on updating it...but I want the GitHub page working.)

I noticed in #35 that you are recommending users set queue='casper'. Casper has 4 different queues depending on what resources you want - I believe casper redirects to htc, which is probably the best choice for a default queue, but can we include information on the available queues (or even just a link to the CISL documentation) so users know what the options are?

I should have caught this in my review, but am still learning how things are set up / configured around here and missed a few things on the SciPy 2023 blogpost that prevented the site from building.

See here: https://github.com/NCAR/esds/actions/runs/6005005970/job/16286821673

We could also do previews in the PRs to help catch these types of thing sooner in the future. I'll log a separate issue for that.

Include blocks of instructions to go with videos

• Meet with Keith Maull from NCAR Library to go over best practices, how we partner with them
• Draft up ideas, general structure
• Submit initial PR
• Receive feedback
• Merge PR

A few comments on the latest blog post

1. It would be useful to include screen shots so users can see what to expect. Definitely useful for walking through making the connection, and possibly also useful for installing the extensions
2. In the "Setup Remote Login", the line "Open a terminal in VS Code after installing the extension, and type in the following :" doesn't contain any commands to type in (I think you removed this step on the advice of @andersy005, so you can remove this sentence as well); it might also be good to add bullet points or a sub-list to separate the different steps
3. I noticed a typo in the "Collaborate!" section (missing an "e" on the word "the"):

Something else that may be helpful is th chat

https://ncar.github.io/esds/posts/2022/xarray-groupby-vs-geocat-climatology/#the-incorrect-way-to-compute-seasonal-averages-from-monthly-data

... Notice that the result has a month dimension. ...

The result appears to have the correct season dimension, no month dimension.

Up-to-date 3-4 step list of "best practices" / things to do to get started specifically on NCAR systems.

Suggestion from @kristenkrumhardt and @mnlevy1981 . Show an example of setting up the remote access, why it is useful to use this workflow, and how you can collaborate in real-time.

In conjunction with the SciPy 2023 recap ESDS Forum next week, @rmshkv and I are planning to put together a blog post.

Missing instructions on how to build the site locally for contributors.

Include how to automate the build process, using Github workflows

@dcherian mentioned including:

• Building the website using github actions.
• The ability to build the website without rerunning the notebook so you don't have to worry about moving data files online

See example workflow here
https://github.com/dcherian/etpac/blob/master/.github/workflows/book.yml

Per this post on Sundog all UCAR sites must have a link to the accessibility statement in the footer by Feb 1.

In looking at this I'm noticing that we don't have really anything in our footer, are there other pieces we should be including? Terms of Use? Etc

About Us — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/about/

It looks like we probably need to make a number of changes associated with the new NSF NCAR branding guidelines.

From the (Re)Introduction to ESDS event, there was a request for a blog post comparing geocat.climatology_average toxarray.resample and xarray.groupby for doing seasonal and yearly climate averages. The GeoCAT implementation handles weighting monthly data by the number of days in each month automatically. The xarray approach requires some extra work to do that weighting. This post will compare the flexibility and ease of both approaches as well as showing what happens when the weighted average is done incorrectly.

Based on conversation with @andersy005 , it would be good to add a work-in-progress notes section to the site

Building an Intake-esm catalog from CESM2 History Files — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/posts/ecgtools-history-files-example/

To test a hypothesis, we want to use the University of Texas coupled high and low-resolution outputs for ET. They have saved the ET as column variables (maybe at grid level). Can anyone help us convert the column variables into lat lon variables? Is there a script or function to process column data? You may find the metadata details below:

netcdf evspsbl_Amon_CESM1-CAM5-SE-HR_hist-1950_r1i1p1f1_gn_195001-201412 {
dimensions:
        nbnd = 2 ;
        grid_corners = 5 ;
        ncol = 777602 ;
        time = UNLIMITED ; // (780 currently)
variables:
        float evspsbl(time, ncol) ;
                evspsbl:_FillValue = 1.e+20f ;
                evspsbl:cell_measures = "area: areacella" ;
                evspsbl:cell_methods = "area: time: mean" ;
                evspsbl:comment = "QFLX" ;
                evspsbl:coordinates = "time lat lon" ;
                evspsbl:description = "at surface; flux of water into the atmosphere due to conversion of both liquid and solid phases to vapor (from underlying surface and vegetation)" ;
                evspsbl:frequency = "mon" ;
                evspsbl:id = "evspsbl" ;
                evspsbl:long_name = "Evaporation Including Sublimation and Transpiration" ;
                evspsbl:mipTable = "Amon" ;
                evspsbl:missing_value = 1.e+20 ;
                evspsbl:out_name = "evspsbl" ;
                evspsbl:prov = "Amon ((isd.003))" ;
                evspsbl:realm = "atmos" ;
                evspsbl:standard_name = "water_evapotranspiration_flux" ;
                evspsbl:time = "time" ;
                evspsbl:time_label = "time-mean" ;
                evspsbl:time_title = "Temporal mean" ;
                evspsbl:title = "Evaporation Including Sublimation and Transpiration" ;
                evspsbl:type = "real" ;
                evspsbl:units = "kg m-2 s-1" ;
                evspsbl:variable_id = "evspsbl" ;
        double lat(ncol) ;
                lat:axis = "Y" ;
                lat:bounds = "lat_bnds" ;
                lat:standard_name = "latitude" ;
                lat:title = "Latitude" ;
                lat:type = "double" ;
                lat:units = "degrees_north" ;
                lat:valid_max = 90. ;
                lat:valid_min = -90. ;
                double lat(ncol) ;
                lat:axis = "Y" ;
                lat:bounds = "lat_bnds" ;
                lat:standard_name = "latitude" ;
                lat:title = "Latitude" ;
                lat:type = "double" ;
                lat:units = "degrees_north" ;
                lat:valid_max = 90. ;
                lat:valid_min = -90. ;
        double lon(ncol) ;
                lon:axis = "X" ;
                lon:bounds = "lon_bnds" ;
                lon:standard_name = "longitude" ;
                lon:title = "Longitude" ;
                lon:type = "double" ;
                lon:units = "degrees_east" ;
                lon:valid_max = 360. ;
                lon:valid_min = 0. ;

I know @jukent mentioned this (maybe others too) and I think it's a great idea.

We could include things like:

• an overview of the event
• resources from tutorials
• recordings

Moving zulip convo here for better tracking...

It'd be nice if we could start some kind of debugging cookbook with known inefficiencies and suggested workarounds

Possible topics:

1. fixing your dask dashboard
2. calculating climatologies and anomalies .... talks about groupby issues
3. reading in netCDF file datasets with open_mfdataset
4. mpi4py vs dask approach to the same problem
5. UCX on cheyenne. when do we expect benefits?
6. writing datasets in parallel:
   1. distributed writes to netCDF
   2. save_mfdataset

Please add commonly encountered problems to this list.

Should these go on the xdev blog; should we start another one?

Python Tutorial Seminar Series - Spring 2021 — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/posts/python-tutorial-seminar-series-spring-2021/

This isn't the best "issue" necessarily, but it'd be great to better highlight ways to get involved with ESDS on a variety of levels.

Having something on the website for this could be really nice as kind of a landing page to point people to.

Issues and more robust contributing guidelines on here could also be a good way to do this.

A few broken links and some dated resources.

• Go through example repository
• Update template repository from @andersy005 to add in documentation build as well
• Draft blog post
• Submit PR
• Merge PR

He was in CISL, not CGD; just wanted an issue ticket to tie to an incoming PR

The majority of questions I get during ESDS office hours are related to using conda and more specifically using conda with NCAR HPC systems. Information and tutorials are out there, but they are hard to find. I think it would be helpful to users if we create a standing page on the website with a summary of the basics and a list of links.

Things to consider including:

• conda cheat sheet This link always points to the latest version of the sheet.
• Using conda envs for Python access An NCAR specific page with instructions on how to clone, make, and use conda envs on the HPC systems
• quick summary of what the NPL conda environment contains
• general trouble shooting tips
  • are you using your own miniconda install or the recommended NCAR provided one through module load conda/latest?
  • conda update conda
  • conda update --name my_env --all
  • check if you have any pinned versions of packages. Remove pins that you no longer need
• links to the NCAR Research Computing Helpdesk

If anyone has suggestions for this page, please comment them! Thanks.

Details: https://sundog.ucar.edu/page/8591

Assemble takeaways from the AMS Python Symposium

It would be neat to add a gallery of Jupyter Books related to different workflows within CGD/more generally NCAR. We could format it as a gallery, but really it would function as a "Library of Books", providing templates at the top of the page for users to easily create their own books.