Comments (4)
I think I might have found out why this is an issue. In daops/utils.core.py
we have:
if apply_fixes:
fix = fixer.Fixer(ds_id)
if fix.pre_processor:
for pre_process in fix.pre_processors:
LOGGER.info(f"Loading data with pre_processor: {pre_process.__name__}")
else:
LOGGER.info(f"Loading data")
ds = xr.open_mfdataset(
file_paths,
preprocess=fix.pre_processor,
use_cftime=True,
combine="by_coords",
)
if fix.post_processors:
for post_process in fix.post_processors:
func, operands = post_process
LOGGER.info(f"Running post-processing function: {func.__name__}")
ds = func(ds, **operands)
else:
ds = open_xr_dataset(file_paths)
I think that both clauses should call open_xr_dataset(...)
but we should allow **kwargs
to be sent to the function, so that you can extend the call but you always get the default arguments set that are common.
NOTE: I might be confused so please double-check I'm not making a daft suggestion here.
from daops.
@agstephens Yes that make sense, I'll change this. Was there a particular issue that it was causing?
from daops.
@ellesmith88 I needed to use decode_timedelta=False
but noticed that it needed changing in roocs utils and daops. So that made me think (a) we need to move the daops call over to using the generic version but (b) for future proofing it would be neat to be able to send extra kwargs in if we had special cases.
Please set decode_timedelta=False
as a default in the roocs utils function.
from daops.
Done
from daops.
Related Issues (20)
- New intake catalog approach could speed up `consolidate` step
- IndexError: list index out of range HOT 3
- Process error: Cannot apply_along_axis when any iteration dimensions are 0 HOT 5
- Can we delete reference to "orchestrate"? HOT 1
- daops: subset-by-point
- daops.consolidate() is inefficient if lots of files - can we provide hints in the file path mapper? HOT 3
- Could decadal fixes derive leadtime values on-the-fly? HOT 6
- daops consolidate - need to check why return types are different
- Fix lookup of catalogs in consolidate so that multiple catalogs could be used (e.g. cmip6 AND cmip5)
- Replicate content between ES indexes HOT 2
- Some CORDEX fixes have been captured by the py-cordex project
- Just send the command-line options directly into the `subset` function HOT 1
- Add unit tests for command-line HOT 2
- Add features: CLI (subset only), Dockerfile and CWL file to the repo HOT 3
- order of elements of area needed for doc strings
- Add unit tests for ResultsDB object HOT 1
- In `check_result(...)` - check that data is not all missing HOT 1
- Enable selection of variables in daops interface
- Release for 0.10.1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from daops.