How many epochs did you train on webnlg dataset? about copymtl HOT 5 CLOSED

That's strange. I try many different settings, but they all work. I update a new version similar to the original one. Please have a try and use the default setting.
'''
epoch 1 loss: 20.777309 F1: 0.050310 P: 0.058574 R: 0.044089
relation F1: 0.294568 P: 0.342954 R: 0.258147
entity F1: 0.201969 P: 0.235144 R: 0.176997

epoch 2 loss: 13.665100 F1: 0.075149 P: 0.089938 R: 0.064537
relation F1: 0.357143 P: 0.427427 R: 0.306709
entity F1: 0.287202 P: 0.343722 R: 0.246645

epoch 3 loss: 12.094102 F1: 0.147541 P: 0.176944 R: 0.126518
relation F1: 0.513413 P: 0.615728 R: 0.440256
entity F1: 0.356930 P: 0.428061 R: 0.306070

epoch 4 loss: 13.580454 F1: 0.198138 P: 0.237500 R: 0.169968
relation F1: 0.539292 P: 0.646429 R: 0.462620
entity F1: 0.410428 P: 0.491964 R: 0.352077

epoch 5 loss: 11.634348 F1: 0.210015 P: 0.252925 R: 0.179553
relation F1: 0.580717 P: 0.699370 R: 0.496486
entity F1: 0.449925 P: 0.541854 R: 0.384665

epoch 6 loss: 10.756370 F1: 0.208622 P: 0.262343 R: 0.173163
relation F1: 0.622787 P: 0.783156 R: 0.516933
entity F1: 0.421093 P: 0.529526 R: 0.349521

epoch 7 loss: 8.445783 F1: 0.230285 P: 0.273768 R: 0.198722
relation F1: 0.630137 P: 0.749120 R: 0.543770
entity F1: 0.465013 P: 0.552817 R: 0.401278

epoch 8 loss: 9.191703 F1: 0.240741 P: 0.303797 R: 0.199361
relation F1: 0.655864 P: 0.827653 R: 0.543131
entity F1: 0.459877 P: 0.580331 R: 0.380831

epoch 9 loss: 6.884187 F1: 0.243792 P: 0.296432 R: 0.207029
relation F1: 0.662152 P: 0.805124 R: 0.562300
entity F1: 0.463506 P: 0.563586 R: 0.393610

epoch 10 loss: 7.504792 F1: 0.244103 P: 0.294756 R: 0.208307
relation F1: 0.676900 P: 0.817360 R: 0.577636
entity F1: 0.497941 P: 0.601266 R: 0.424920

epoch 11 loss: 7.149976 F1: 0.242936 P: 0.285345 R: 0.211502
relation F1: 0.694312 P: 0.815517 R: 0.604473
entity F1: 0.484404 P: 0.568966 R: 0.421725

epoch 12 loss: 6.352170 F1: 0.266135 P: 0.307628 R: 0.234505
relation F1: 0.717912 P: 0.829841 R: 0.632588
entity F1: 0.539521 P: 0.623638 R: 0.475399

epoch 13 loss: 6.124747 F1: 0.267201 P: 0.310491 R: 0.234505
relation F1: 0.717874 P: 0.834179 R: 0.630032
entity F1: 0.537313 P: 0.624365 R: 0.471565

epoch 14 loss: 6.946354 F1: 0.278336 P: 0.317253 R: 0.247923
relation F1: 0.738164 P: 0.841374 R: 0.657508
entity F1: 0.544476 P: 0.620605 R: 0.484984

epoch 15 loss: 5.522422 F1: 0.281497 P: 0.322341 R: 0.249840
relation F1: 0.721382 P: 0.826051 R: 0.640256
entity F1: 0.539237 P: 0.617477 R: 0.478594

epoch 16 loss: 5.934337 F1: 0.267284 P: 0.302176 R: 0.239617
relation F1: 0.730577 P: 0.825947 R: 0.654952
entity F1: 0.530292 P: 0.599517 R: 0.475399

epoch 17 loss: 6.453718 F1: 0.270039 P: 0.305153 R: 0.242173
relation F1: 0.741717 P: 0.838164 R: 0.665176
entity F1: 0.541503 P: 0.611916 R: 0.485623

epoch 18 loss: 4.792238 F1: 0.268985 P: 0.302474 R: 0.242173
relation F1: 0.731725 P: 0.822825 R: 0.658786
entity F1: 0.549326 P: 0.617717 R: 0.494569

epoch 19 loss: 4.421184 F1: 0.276836 P: 0.309392 R: 0.250479
relation F1: 0.739407 P: 0.826361 R: 0.669010
entity F1: 0.570621 P: 0.637727 R: 0.516294

epoch 20 loss: 4.318580 F1: 0.280458 P: 0.306991 R: 0.258147
relation F1: 0.748351 P: 0.819149 R: 0.688818
entity F1: 0.583825 P: 0.639058 R: 0.537380

epoch 21 loss: 4.167426 F1: 0.279696 P: 0.304282 R: 0.258786
relation F1: 0.770028 P: 0.837716 R: 0.712460
entity F1: 0.553177 P: 0.601803 R: 0.511821
'''

from copymtl.

Thank you for your work! I have a try on a new version of the webnlg dataset(after entity masking). I have made some mistake in my dataset. The result is good. If you can add the method of data preprocessing, it will be an excellent generalization framework on NRE task.
'''
epoch 1 loss: 16.559380 F1: 0.142799 P: 0.199543 R: 0.111182
relation F1: 0.415098 P: 0.580046 R: 0.323192
entity F1: 0.420008 P: 0.586907 R: 0.327015

epoch 2 loss: 13.444241 F1: 0.188377 P: 0.278801 R: 0.142243
relation F1: 0.560278 P: 0.829223 R: 0.423065
entity F1: 0.439405 P: 0.650328 R: 0.331794

epoch 3 loss: 10.825327 F1: 0.220714 P: 0.320303 R: 0.168366
relation F1: 0.608060 P: 0.882424 R: 0.463842
entity F1: 0.482982 P: 0.700909 R: 0.368429

epoch 4 loss: 10.490962 F1: 0.241535 P: 0.339389 R: 0.187480
relation F1: 0.642315 P: 0.902537 R: 0.498566
entity F1: 0.502976 P: 0.706747 R: 0.390411

epoch 5 loss: 10.200238 F1: 0.251348 P: 0.347985 R: 0.196719
relation F1: 0.663682 P: 0.918850 R: 0.519433
entity F1: 0.533632 P: 0.738800 R: 0.417649

epoch 6 loss: 9.493075 F1: 0.258635 P: 0.356983 R: 0.202772
relation F1: 0.681227 P: 0.940269 R: 0.534087
entity F1: 0.536164 P: 0.740045 R: 0.420357

epoch 7 loss: 9.527404 F1: 0.260773 P: 0.355825 R: 0.205798
relation F1: 0.691694 P: 0.943817 R: 0.545874
entity F1: 0.530427 P: 0.723768 R: 0.418605

epoch 8 loss: 9.709828 F1: 0.270671 P: 0.356307 R: 0.218222
relation F1: 0.715598 P: 0.942003 R: 0.576935
entity F1: 0.567223 P: 0.746684 R: 0.457311

epoch 9 loss: 10.282122 F1: 0.291675 P: 0.375879 R: 0.238292
relation F1: 0.742835 P: 0.957286 R: 0.606881
entity F1: 0.583934 P: 0.752513 R: 0.477063

epoch 10 loss: 8.802513 F1: 0.288422 P: 0.367151 R: 0.237496
relation F1: 0.756746 P: 0.963310 R: 0.623128
entity F1: 0.575878 P: 0.733071 R: 0.474196

epoch 11 loss: 8.855803 F1: 0.299773 P: 0.369231 R: 0.252310
relation F1: 0.766465 P: 0.944056 R: 0.645110
entity F1: 0.614307 P: 0.756643 R: 0.517044

epoch 12 loss: 9.465460 F1: 0.295006 P: 0.355411 R: 0.252150
relation F1: 0.789042 P: 0.950606 R: 0.674419
entity F1: 0.605293 P: 0.729232 R: 0.517362

epoch 13 loss: 7.226478 F1: 0.302694 P: 0.361326 R: 0.260433
relation F1: 0.799963 P: 0.954917 R: 0.688277
entity F1: 0.632232 P: 0.754696 R: 0.543963

epoch 14 loss: 7.257793 F1: 0.310519 P: 0.366833 R: 0.269194
relation F1: 0.805328 P: 0.951378 R: 0.698152
entity F1: 0.648783 P: 0.766442 R: 0.562440

epoch 15 loss: 6.049053 F1: 0.310207 P: 0.360794 R: 0.272061
relation F1: 0.814384 P: 0.947191 R: 0.714240
entity F1: 0.655467 P: 0.762357 R: 0.574865

epoch 16 loss: 7.307980 F1: 0.316016 P: 0.361032 R: 0.280981
relation F1: 0.831243 P: 0.949652 R: 0.739089
entity F1: 0.661233 P: 0.755424 R: 0.587926

epoch 17 loss: 6.690179 F1: 0.318442 P: 0.362637 R: 0.283848
relation F1: 0.833810 P: 0.949532 R: 0.743230
entity F1: 0.663510 P: 0.755596 R: 0.591430

epoch 18 loss: 6.352150 F1: 0.319246 P: 0.356890 R: 0.288786
relation F1: 0.846628 P: 0.946457 R: 0.765849
entity F1: 0.670893 P: 0.750000 R: 0.606881

epoch 19 loss: 5.875832 F1: 0.321760 P: 0.357839 R: 0.292291
relation F1: 0.849378 P: 0.944618 R: 0.771583
entity F1: 0.670174 P: 0.745320 R: 0.608793

epoch 20 loss: 6.401268 F1: 0.327978 P: 0.362969 R: 0.299140
relation F1: 0.853825 P: 0.944917 R: 0.778751
entity F1: 0.684946 P: 0.758021 R: 0.624721

epoch 21 loss: 4.804408 F1: 0.330060 P: 0.363167 R: 0.302485
relation F1: 0.849396 P: 0.934596 R: 0.778433
entity F1: 0.693317 P: 0.762861 R: 0.635393

epoch 22 loss: 5.498651 F1: 0.331173 P: 0.361988 R: 0.305193
relation F1: 0.855760 P: 0.935386 R: 0.788627
entity F1: 0.692594 P: 0.757038 R: 0.638261

epoch 23 loss: 6.467267 F1: 0.335626 P: 0.364587 R: 0.310927
relation F1: 0.857806 P: 0.931827 R: 0.794680
entity F1: 0.702373 P: 0.762981 R: 0.650685

epoch 24 loss: 4.964077 F1: 0.330294 P: 0.355140 R: 0.308697
relation F1: 0.862378 P: 0.927249 R: 0.805989
entity F1: 0.698935 P: 0.751512 R: 0.653234

epoch 25 loss: 5.744630 F1: 0.338613 P: 0.361548 R: 0.318414
relation F1: 0.866435 P: 0.925122 R: 0.814750
entity F1: 0.716185 P: 0.764695 R: 0.673463
'''

from copymtl.

My preprocessing data_prepare.py is almost the same as the official tensorflow version. And I have not got the method which turns the WebNLG from .xml to .json.

from copymtl.

You can refer to this website webnlg2017.

from copymtl.

Actually, I tried to parse xml by myself. Unfortunately, my preprocessing generated different json from the official one. Finally, I use the tf version directly.

To fairly compare your model with copyre paper, I think:

1. ask the author for the preprocessing.

2. parse xml by yourself and evaluate models in the new WebNLG for NRE

from copymtl.