Add "dimensions" arg to validation CLI about nomenclature HOT 9 CLOSED

See https://github.com/iiasa/ariadne-intern-workflow/blob/31cfbc2f9bf89b498a301b65cce56a585343a9c2/.github/workflows/validation.yml#L28 for a real-world use case.

from nomenclature.

Should be reasonably quick to implement. Who should take this one myself or @LauWien since she did the CLI originally?

from nomenclature.

I created the issue more as a reminder for the medium-term development. Let's leave this until next year...

from nomenclature.

On further thought: currently, nomenclature.testing.assert_valid_structure() only validates the variable and region dimensions.

Going forward, the behavior should be as follows:

• by default, all sub-folders in definitions are validated
• if a value for dimensions is given explicitly, (only) these dimensions are validated

This default is more cautious than only validating default dimensions because additional dimensions may be added over time in a project - and the user adding these dimensions may forget to add these new dimensions to the tests.

from nomenclature.

Hello!
I think my implementation works now.
A few questions still to make sure it's correct:

• Should the non-default dimensions only be the variables on an IamDataFrame (variable, scenario, ...) or could it be a random name? I figured it could be anything because of issue 68 but let me know.
• Now we can pass any number of arguments in the terminal like this "nomenclature validate-project path/myproject variable region scenario". Is it fine like this or do you prefer that the user enters the list in the form "['variable','region',...]"?
• I have a test with an empty dimension, a test working fine with a dimension '"scenario" and one also working fine with a random dimension "foo". Do I need more?
• Are there other steps for this issue?

Thank you!

from nomenclature.

• Should the non-default dimensions only be the variables on an IamDataFrame (variable, scenario, ...) or could it be a random name? I figured it could be anything because of issue 68 but let me know.

As a first PR, I would only add the feature to pass a list of dimensions via the CLI (and not worry about #68 yet).

• Now we can pass any number of arguments in the terminal like this "nomenclature validate-project path/myproject variable region scenario". Is it fine like this or do you prefer that the user enters the list in the form "['variable','region',...]"?

I would expect a CLI like

nomenclature validate-project . --dimensions=["variable", "region"]

so that we can add more options later.

• I have a test with an empty dimension, a test working fine with a dimension '"scenario" and one also working fine with a random dimension "foo". Do I need more?

As mentioned by @phackstock yesterday, it would be good to have one passing and one failing test. To make it easier for the reviewer, it would be useful to clearly name the test-data folders like "cli-dimension-failing" (with a dimension where the validation fails because something is not correct with the specifications) and a "cli-dimension-passing". In both cases, the dimension could be called "foo".

• Are there other steps for this issue?

Better to keep the PR as simple as possible, then discuss and review, merge - and then start another one with a next set of features.

from nomenclature.

OK thank you very much! Now I am stuck... Do either of you know how to pass a list as an option in CLI?

from nomenclature.

Looking at the click documentation quickly, this looks like a promising start https://click.palletsprojects.com/en/8.1.x/options/#multiple-options.
Also when you open a PR you can choose to open it as a draft PR which might be a good idea in this case, then we can discuss more closely to the code. Once it's all ready you set it to "ready for review". https://github.blog/2019-02-14-introducing-draft-pull-requests/

from nomenclature.

Great thank you!!

from nomenclature.