Comments (3)
saw562@raijin4 ~ $ clef --local cmip5 --model=ACCESS1.0 --experiment_family=RCP --frequency=mon --variable=tas --ensemble=r1i1p1
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/1pctCO2/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/abrupt4xCO2/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/amip/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/historical/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/piControl/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/rcp45/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/rcp85/mon/atmos/Amon/r1i1p1/latest/tas
from clef.
The reason for that is that we don't have experiment_family in the attributes.
To fix that we decided to change local search so it will add a filter(experiment like "...%") when experiment_family is present.
To make this work properly:
- cli experiment_family can accept only one value (fixed in cmip5_args)
- experiment_family in datasets_constraints has to be a list (fixed in cli.py around line 248)
- as we did for var if experiment-family we add an extra filter to local_search in code.py
.filter(C5Dataset.experiment.like(family_dict[family]))
where family_dict is {'RCP' : 'rcp%' , 'decadal': 'decadal%' ...}
it works but now local_search has all this options which modify the search and lots of repetition as a consequence, would be nice to be able to add only the relevant bit of query
from clef.
I've changed the search tidied up the code to keep into consideration that some families have couldn't be captured by one expression to:
.filter(C5Dataset.experiment.like(any_(family_dict[family]))
I've also changed the code so it uses only one main search for both cmip5 and cmip6 and add one line filter for 'variable' and/or 'experiment_family' when defined in CMIP5.
also moved where I adjust the experiment_family from dataset_constraints to terms definition.
I've run tests and also try different queries and all seems to work.
In branch "localfix" which was previously merged with "tests"
from clef.
Related Issues (20)
- version is not getting updated in post_local HOT 1
- Global log file not writeable by all users HOT 2
- capture error if nothing returned by remote query with and option HOT 1
- adding "saving to file" message where appropriate HOT 2
- add function to get variant-label from path where r/I/p1/f are 0 HOT 2
- fix for CMIP6 files with wrong member_id create issue with CMIP5
- check if request submission works correctly HOT 2
- CORDEX tables missing some cordex data HOT 2
- Length of Datasets HOT 1
- access from compute nodes HOT 4
- Update to citation flag: service changed
- Enhancement: Enable alternate help flag HOT 2
- Raise the number of results that can be returned only for the remote option HOT 1
- deprecation warning for pandas
- ESGF introduced limit to remote queries HOT 1
- problem with matching query HOT 1
- fix readthedocs
- deprecation warning pandas HOT 1
- clef.data as package
- clef on NCI missing some local files HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clef.