@paigem, it looks like we're hitting some FileNotFoundErrors in https://pangeo-forge.org/dashboard/recipe-run/338.

Do the reported urls appear to be ones at which real data exists? If not, perhaps this is an issue with the file pattern?

Following on @andersy005's discovery in #8 (comment), I just checked to see if the urls specified by this recipe are publicly available, and found that they are not.

In [1]: # copy and paste the `recipe.py` code in this repo here
In [2]: urls = [url for _, url in pattern.items()]
In [3]: urls[0]
Out[3]: 'https://tds.ucar.edu/thredds/fileServer/datazone/campaign/cesm/collections/ASD/hybrid_v5_rel04_BC5_ne120_t12_pop62/atm-regrid/proc/tseries/daily/FV_768x1152.bilinear.hybrid_v5_rel04_BC5_ne120_t12_pop62.cam.h1.LHFLX.00780101-00781231.nc'
In [4]: !wget 'https://tds.ucar.edu/thredds/fileServer/datazone/campaign/cesm/collections/ASD/hybrid_v5_rel04_BC5_ne120_t12_pop62/a
   ...: tm-regrid/proc/tseries/daily/FV_768x1152.bilinear.hybrid_v5_rel04_BC5_ne120_t12_pop62.cam.h1.LHFLX.00780101-00781231.nc'
   ...: 
--2022-12-22 09:43:24--  https://tds.ucar.edu/thredds/fileServer/datazone/campaign/cesm/collections/ASD/hybrid_v5_rel04_BC5_ne120_t12_pop62/atm-regrid/proc/tseries/daily/FV_768x1152.bilinear.hybrid_v5_rel04_BC5_ne120_t12_pop62.cam.h1.LHFLX.00780101-00781231.nc
Resolving tds.ucar.edu (tds.ucar.edu)... 128.117.181.7
Connecting to tds.ucar.edu (tds.ucar.edu)|128.117.181.7|:443... connected.
HTTP request sent, awaiting response... 401 

Username/Password Authentication Failed.

@paigem, as we discussed offline, I do recall that we wrestled with this issue initially. What I don't recall is if we solved it by finding an unauthenticaed endpoint for this data, or if the solution was that I just manually ran this recipe in a notebook the first time around (using your credentials).

Our discussion in #5 suggests to me that we did actually run this on Pangeo Forge Cloud (not in a notebook), because there would never have been a KilledWorker in a notebook setting (at least as I would have done it). OTOH, I remember there was a long saga around getting pangeo-forge-recipes to support passing credentials as query string parameters, which was motivated by this recipe. Which, on the contrary, makes me think there was never an unauthenticated pathway to this data. 🤷

@paigem, bringing it back to what we can do now, are there currently any alternate endpoints for this data that do not require authentication? If the only option is to get this data via an authenticated endpoint, then we may be temporarily blocked here. Pangeo Forge Cloud does not currently have a mechanism for injecting data provider creds at runtime (other recipes blocked by this).

The tutorial notebooks provided in the docs demonstrate how to run pangeo-forge-recipes yourself in a Jupyter notebook (as opposed to here, on Pangeo Forge Cloud):

https://pangeo-forge.readthedocs.io/en/latest/pangeo_forge_recipes/tutorials/index.html

this could be a possible alternative on the short term, given that you could provide your own credentials in that context.

It looks like the most recent recipe run failed again, this time with a KilledWorker error. This appears to be a different failure than we saw in this previous recipe run, which did not result in an explicit error, but only processed a single chunk in time.

@cisaacstern Any thoughts on how to get passed a KilledWorker error?