Raise/warn if unexpected regions exist for region-processing about nomenclature HOT 9 CLOSED

Ah yes, agree it should be at the same level as native_regions. I would prefer exclude_regions.

from nomenclature.

That is a difficult design decision, as you said there are cases to be made for both options.
The crux of the decision lies in my opinion in the fact that there are two possible root causes for additional regions:

1. They are somehow model internal debug/test regions which are not intended for upload but exported as results anyway. (If such a use case exists)
2. A region that is intended to be uploaded bu was forgotten in the model mapping.

In the first case we (and the user for that matter) would probably prefer just a warning.
In the second case an error should be raised as that would cause unexpected behavior with regions missing.
Now the problem is that there is no way for us to distinguish between 1. and 2. (at least none that I can think of quickly).

So we're stuck between forcing users to clean up their output before uploading it or potentially missing regions from a scenario explorer.

My personal (although not strong) preference would be to throw an error but I don't know how often cases 1. or 2. occur.

from nomenclature.

Good points. I would say have a common way for variables, scenarios, and regions.
Right now, it seems variables and scenarios that are "unexpected" cause an error. So by that rule error also for regions would make sense.
Overall, and from a user experience, I think on the other hand in all three cases it might be preferable to have warnings instead. These errors are easy to see in the log file, and one can then decide if it would be good to fix the issue or one is OK with what has been accepted by the script.

from nomenclature.

Having a common expextable behaviour for regions, variables and scenarios as Johannes suggest sounds very sensable.
I tend to prefer getting an error message. Then I'm sure that I know that something went different then I might have expected. However, in our scripts that bring the results into IAMC format it is rather easy to exclude e.g. variables that I don't want to upload. But I also see that it might be reasonable to just give a warning, e.g. that I have a country or a region in the results that I do not yet have in the region mapping but might not want to add yet.

from nomenclature.

One more option (slightly more involved though but maybe worth considering), would be to raise an error for unexpected regions unless they are specified as exclude in the model mapping, something like this:

model: model_a
native_regions:
- region_a
- region_b
- exclude:
  - diagnostic_region

The difference would be that the presence of diagnostic_region would initially cause an error which would prompt the user to either remove it from the upload or add it to the exclude section. Once it is in the exclude section it could not prompt anything or simply an INFO level log entry "excluding region diagnostic_region".
If at some point regions are unexpectedly missing from an upload it should be easy to go to the model mapping and look at the exclude section.

If this is not a good way to go I would still prefer raising an error, keeping consistency with the approach for regions, variables and scenarios.
In my opinion it's better to cause a little bit of an overhead of removing results than to have potentially missing results without any errors. I agree with @danielhuppmann that if an upload is successful it is very unlikely that a user will actually go back and search the log file for potential warnings. In the spirit of "better safe than sorry" I would vote in favor of the error.

from nomenclature.

Sounds like a good approach to me, but I would add this as a section at the same level as native_regions and common_regions. Maybe called exclude_regions or ignore_regions...?

from nomenclature.

@danielhuppmann as I'm looking into the implementation I've thought of the following case and was wondering how to handle that.
Assuming we have a model that reports "World" level data and we want to make use of partial region aggregation, we put "World" on the category common_regions (instead of native_regions).
If we now add "World" to exclude_regions what should happen? Should we raise an error since a region cannot be in common_regions and exclude_regions or should this just mean that model native results are ignored and World is created with only results region aggregation?
Another case would be having "World" in both native_regions as well as exclude_regions but there I would say the case is pretty clear cut and we throw an error.

from nomenclature.

I would say that any region name can only one of

• a (target) name of the native-region OR
• a common-region OR
• an exclude-region
  raising an error if a region is in more than one group.

I would keep the current behavior:

• native-regions: keep (with rename)
• common-regions: keep as reported if region exists, otherwise compute aggregate (and compare for info purposes)

from nomenclature.

Perfect, fully agree. Anything else would not be intuitive and lead to unexpected results.
I'll implement it that way.

from nomenclature.