Issue in running on MNLI dataset about pet HOT 4 CLOSED

Your scripts are very hardcoded for data processing, one needs to change each and every processor for a differently structured data, or force-structure own data to the script

from pet.

Can you please specify the format of MNLI dataset that you used, based on which you wrote your preprocessors in tasks.py

from pet.

Hi there,

Your scripts are very hardcoded for data processing, one needs to change each and every processor for a differently structured data, or force-structure own data to the script

We did not design the task processors as "general purpose" solutions for different kinds of tasks/formats. Instead, they are supposed to provide a simple interface for implementing your own logic for your own data formats (as shown in this example): All you need to do is to define the get_train_examples, get_dev_examples, get_test_examples, get_unlabeled_examples and get_labels methods, where the first four are supposed to return a list of InputExamples (as defined here) and the last one returns a list of string labels.

The MNLI dataset that we used for our task processor is a tab-separated file with the keys (in that order) index, promptID, pairID, genre, sentence1_binary_parse, sentence2_binary_parse, sentence1_parse, sentence2_parse, sentence1, sentence2, label1, ..., labeln, gold_label. However, the only information that we actually make use of in the task processor (see here) are the columns 0 (index), 8 (sentence1), 9 (sentence2) and -1 (gold_label).

For example, the first two lines of the dev (matched) set are:

index	promptID	pairID	genre	sentence1_binary_parse	sentence2_binary_parse	sentence1_parse	sentence2_parse	sentence1	sentence2	label1	label2	label3	label4	label5	gold_label
0	63735	63735n	slate	( ( The ( new rights ) ) ( are ( nice enough ) ) )	( Everyone ( really ( likes ( the ( newest benefits ) ) ) ) )	(ROOT (S (NP (DT The) (JJ new) (NNS rights)) (VP (VBP are) (ADJP (JJ nice) (RB enough)))))	(ROOT (S (NP (NN Everyone)) (VP (ADVP (RB really)) (VBZ likes) (NP (DT the) (JJS newest) (NNS benefits)))))	The new rights are nice enough	Everyone really likes the newest benefits 	neutral	entailment	neutral	neutral	neutral	neutral

You can either modify your own files to match that format or simply implement your own task processor that works with your input format. Let me know if you need any further help for that!

from pet.

Yes, thank you for the detailed answer,
Cheers.

from pet.