Inquiring about details of the Entity Resolution Model about spark-nlp-workshop HOT 2 CLOSED

Hi @agr505 ,

1. SentenceEntityResolverApproach annotator is used for model training, SentenceEntityResolverModel is used for downloading a pretrained resolver model. When you train a model with SentenceEntityResolverApproach and save it, then you can call it by using SentenceEntityResolverModel annotator. All "Model" extensions are used for calling pretrained models in Spark NLP.
2. It is like a tree architecture but not the same, there are some other special algorithms behind it. We don't have a paper for Sentence Entity Resolution yet.
3. These models are trained with the augmented version of the formal datasets, so there may be more than one line that has the same code, but not the same concept name and embeddings, in the model training data. When you want to drop a code from the model, all the lines in the model that has the same code will be dropped. So yes, the embeddings, codes, concept names, etc. will be dropped.
4. You can train your own resolver model and update it from time to time with the new rows, this would be better in your case. But if the pretrained model performances are enough for you, dropping irrelevant codes from the model may help to increase the accuracy since the models return the closest embeddings from the space.
5. The most important stage of getting resolutions of the terms is entity extraction. So having a specific NER model will help you to extract the appropriate entities according to the concept and this will affect the accuracy directly.
6. You can use assertion status models to check the negation status of the entities.

from spark-nlp-workshop.

Also we will have some SDOH models soon @agr505. I hope these answers would help you.

from spark-nlp-workshop.