Could decadal fixes derive leadtime values on-the-fly? about daops HOT 6 CLOSED

I've started to have a look at this but have come a potential issue:

Deriving lead_times and reftime both depend on having the start_date (which is the same as the sub_experiment_id). If this isn't in the netCDF file, I had been getting it from the ds_id.

According to the initial rules we were given (roocs/34e-mngmt#98) it is possible that startdate/sub_experiment_id won't be in the file, so we can't depend on getting them from there.

This means that either a default time or the ds_id need to be passed to the get_lead_times / get_reftime functions, as we can't necessarily get it from the xarray dataset. At the moment the ds_id isn't passed through to the fix functions (those in daops.data_utils), so isn't available for me to use.

Options I have thought of are:

• we could send the ds_id through to fix functions by default. I think this would help keep the fix templates generic. From a quick look this looks like its possible and relatively straightforward to do.

• we could have a way of passing arguments through somehow with the "derive: daops.data_utils.time_utils.function" string. This would the templates would be dataset specific so maybe isn't worth doing.

If we do the first option and pass the ds_id through to the fix functions and implement this then there is the chance we might come across the same issue in future where the argument we need doesn't exist in the ds_id and would need to passed through by the template.

@agstephens Hopefully that makes sense, what are your thoughts?

from daops.

Hi @ellesmith88, thinking about your first option...

we could send the ds_id through to fix functions by default. I think this would help keep the fix templates generic. From a quick look this looks like its possible and relatively straightforward to do.

I'm going to make some assertions (please tell me if I'm wrong):

1. All fixes in daops and dachar are only meaningful in the context of a ds_id.
2. Therefore, we will always have a ds_id available at the point where we want to look up the fix, in: https://github.com/roocs/daops/blob/master/daops/utils/core.py
3. There are no other places in the stack that would get fundamentally broken by changing the fix signature to: fix(ds_id, ds, **operands)

If that is true, then I like your idea. It may well be useful in other cases of fixes in the future.

from daops.

I think all of your points are correct, I'll implement this and see how it goes

from daops.

Hi @ellesmith88, is this now implemented? Can we close it? Thanks

from daops.

@agstephens yes this is done.

from daops.

Thanks @ellesmith88

from daops.