Giter VIP home page Giter VIP logo

Comments (3)

paolap avatar paolap commented on July 21, 2024

saw562@raijin4 ~ $ clef --local cmip5 --model=ACCESS1.0 --experiment_family=RCP --frequency=mon --variable=tas --ensemble=r1i1p1
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/1pctCO2/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/abrupt4xCO2/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/amip/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/historical/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/piControl/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/rcp45/mon/atmos/Amon/r1i1p1/latest/tas
/g/data1/rr3/publications/CMIP5/output1/CSIRO-BOM/ACCESS1-0/rcp85/mon/atmos/Amon/r1i1p1/latest/tas

from clef.

paolap avatar paolap commented on July 21, 2024

The reason for that is that we don't have experiment_family in the attributes.
To fix that we decided to change local search so it will add a filter(experiment like "...%") when experiment_family is present.

To make this work properly:

  • cli experiment_family can accept only one value (fixed in cmip5_args)
  • experiment_family in datasets_constraints has to be a list (fixed in cli.py around line 248)
  • as we did for var if experiment-family we add an extra filter to local_search in code.py
    .filter(C5Dataset.experiment.like(family_dict[family]))
    where family_dict is {'RCP' : 'rcp%' , 'decadal': 'decadal%' ...}

it works but now local_search has all this options which modify the search and lots of repetition as a consequence, would be nice to be able to add only the relevant bit of query

from clef.

paolap avatar paolap commented on July 21, 2024

I've changed the search tidied up the code to keep into consideration that some families have couldn't be captured by one expression to:
.filter(C5Dataset.experiment.like(any_(family_dict[family]))

I've also changed the code so it uses only one main search for both cmip5 and cmip6 and add one line filter for 'variable' and/or 'experiment_family' when defined in CMIP5.

also moved where I adjust the experiment_family from dataset_constraints to terms definition.
I've run tests and also try different queries and all seems to work.
In branch "localfix" which was previously merged with "tests"

from clef.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.