Loading GLUE dataset loads CoLA by default about datasets HOT 3 CLOSED

As a follow-up to this: It looks like the actual GLUE task name is supplied as the name argument. Is there a way to check what names/sub-datasets are available under a grouping like GLUE? That information doesn't seem to be readily available in info from nlp.list_datasets().

Edit: I found the info under Glue.BUILDER_CONFIGS

from datasets.

Yes so the first config is loaded by default when no name is supplied but for GLUE this should probably throw an error indeed.

We can probably just add an __init__ at the top of the class Glue(nlp.GeneratorBasedBuilder) in the glue.py script which does this check:

class Glue(nlp.GeneratorBasedBuilder):
    def __init__(self, *args, **kwargs):
        assert 'name' in kwargs and kwargs[name] is not None, "Glue has to be called with a configuration name"
        super(Glue, self).__init__(*args, **kwargs)

from datasets.

An error is raised if the sub-dataset is not specified :)

ValueError: Config name is missing.
Please pick one among the available configs: ['cola', 'sst2', 'mrpc', 'qqp', 'stsb', 'mnli', 'mnli_mismatched', 'mnli_matched', 'qnli', 'rte', 'wnli', 'ax']
Example of usage:
	`load_dataset('glue', 'cola')`

from datasets.