devintdha / spark-nlp Goto Github PK

There are many trainable annotators are missing from this page. We should have full examples for Scala and Python for each trainable annotator, link to Scaladoc and PyDoc just in case, and link to possible notebooks for training if available.

Trainable annotators should be divided into categories:

Datasets:
Classes that help preparing datasets before training like

• CoNLL(),
• POS(),
• CoNLLU(),
• PubTator

Text processing:

• - SentenceDetectorDLApproach
• - Lemmatizer (two approaches: from a file and from a DataFrame which uses CoNLLU() class)
• - PerceptronApproach
• - WordSegmenterApproach
• - DependencyParserApproach
• - TypedDependencyParserApproach

Spell checkers:

• - NorvigSweetingApproach
• - SymmetricDeleteApproach
• - ContextSpellCheckerApproach

Token Classification:

• - NerCrfApproach
• - NerDLApproach

Text Classification:

• - ViveknSentimentApproach
• - ClassifierDLApproach
• - MultiClassifierDLApproach
• - SentimentDLApproach

External trainable models:

• - BertForTokenClassification
• - DistilBertForTokenClassification