Change the data in tests_n500 about checklist HOT 2 CLOSED

The tests_n500 file contains the examples in the test suite presented in the paper. If you want to test a model with different tests, you have to create your own test or suite, and then you can export it as file if you want to. Take a look at this tutorial.

from checklist.

I'm sorry to bother again, but I haven't been able to solve it. It's like it cannot access properly the "new" tests_n500 file and I haven't found where I can specify to use this new evaluation dataset or "where I have to change what". Moreover, I didn't find where I can specify the gold labels for the new customized tests_n500.

Thanks again

IndexError Traceback (most recent call last)
in
1 pred_path = '/Users/Marta/opt/anaconda3/lib/python3.6/site-packages/checklist/release_data/sentiment/predictions/TextBlobPred_AMI2019_en_training.txt'
----> 2 suite.run_from_file(pred_path, overwrite=True)
3 suite.visual_summary_table()

~/opt/anaconda3/lib/python3.6/site-packages/checklist/test_suite.py in run_from_file(self, path, file_format, format_fn, ignore_header, overwrite)
248 format_fn=format_fn,
249 ignore_header=ignore_header)
--> 250 self.run_from_preds_confs(preds, confs, overwrite=overwrite)
251
252 def run(self, predict_and_confidence_fn, verbose=True, **kwargs):

~/opt/anaconda3/lib/python3.6/site-packages/checklist/test_suite.py in run_from_preds_confs(self, preds, confs, overwrite)
220 p = preds[slice(*self.test_ranges[n])]
221 c = confs[slice(*self.test_ranges[n])]
--> 222 t.run_from_preds_confs(p, c, overwrite=overwrite)
223
224 def run_from_file(self, path, file_format=None, format_fn=None, ignore_header=False, overwrite=False):

~/opt/anaconda3/lib/python3.6/site-packages/checklist/abstract_test.py in run_from_preds_confs(self, preds, confs, overwrite)
309 self._check_create_results(overwrite)
310 self.update_results_from_preds(preds, confs)
--> 311 self.update_expect()
312
313 def run_from_file(self, path, file_format=None, format_fn=None, ignore_header=False, overwrite=False):

~/opt/anaconda3/lib/python3.6/site-packages/checklist/abstract_test.py in update_expect(self)
128 def update_expect(self):
129 self._check_results()
--> 130 self.results.expect_results = self.expect(self)
131 self.results.passed = Expect.aggregate(self.results.expect_results, self.agg_fn)
132

/home/marcotcr/work/checklist/checklist/expect.py in expect(self)

/home/marcotcr/work/checklist/checklist/expect.py in (.0)

/home/marcotcr/work/checklist/checklist/expect.py in expect_fn(xs, preds, confs, labels, meta)

IndexError: index 0 is out of bounds for axis 0 with size 0

from checklist.