zeyudai / paragraph-level_implicit_discourse_relation_classification Goto Github PK

Is there need a testEpochs ?
https://github.com/ZeyuDai/paragraph_implicit_discourse_relations/blob/b6a0f00c49a3a1d78c4cdba98e1fcf6ce987eb8c/run_discourse_parsing.py#L393

In Readme you have two sections, one on how to run the code, and the second about data preprocessing. It is not clear to me. Should we preprocess the data first? Do you mention the preprocessing as an optional step? Thank you for the code!!

Hi, I think the process_label function is not correct (https://github.com/ZeyuDai/paragraph-level_implicit_discourse_relation_classification/blob/master/run_discourse_parsing.py#L142). Removing some labels will improve the accuracy and F1.

Here is an example:
pred = np.array([[1,0,0,0], [0,0,0,1]])

target = np.array([[1,0,1,0], [1,0,0,1]])

Then accuracy_score(target, pred) will return 0 and precision_recall_fscore_support(target, pred, average="macro") will return (0.5, 0.375, 0.41666666666666663, None).

After your processing, target will be changed to target = np.array([[1,0,0,0], [1,0,0,0]]). Then accuracy_score(target, pred) will return 1.0 and precision_recall_fscore_support(target, pred, average="macro") will return (0.5, 0.5, 0.5, None).

When I remove this processing function, the accuracy decreases around 3% and F1 decreases more than 1%.

This task is a multilabel classification problem. The standard evaluation tool can be found at https://github.com/attapol/conll15st/blob/master/scorer.py .

Hi,

how is it possible to use your pretrained model on a new data?

Can you upload the link to the pretrained model file?