Evaluation of SyntaxNet on whole, unprocessed documents (no gold SBD etc) about models HOT 2 CLOSED

Thanks for raising this point. It's definitely true that pre-processing is important. For example, it's crucial that the tokenization used during training is as close to the tokenization when evaluating "in the wild" as possible. If you have a different tokenizer that doesn't handle edge cases the same (e.g. splitting on apostrophes, etc), you will introduce systematic errors into the parser, so it's important to take that into account when running Parsey McParseface.

Joint segmentation and parsing models can be implemented in SyntaxNet and we would encourage anyone who is interested to try it out. In particular, using a beam model has the advantage that it can maintain and compare multiple segmentations at once. We can also look into releasing some more detailed benchmarks beyond the ones we have already.

Best,
David Weiss

from models.

Closing this issue for now. Feel free to reopen if you want to add more data points on the effect of pre-processing on syntaxnet models.

from models.