Add documentation mapping IOB tags with the DEFT paper's Tables 2 and 3 about deft_corpus HOT 7 CLOSED

There are few other tags present in the data which are not BIO tags.

“	data/source_txt/train/t1_biology_1_404.txt	 22015	 22016	 I-Qualifier	 T221	 T220	 Supplements
seahorse”)—a	data/source_txt/train/t1_biology_1_404.txt	 22016	 22028	 Qualifier	 T221	 T220	 Supplements
transfer	data/source_txt/train/t1_biology_1_404.txt	 26543	 26551	 B-Qualifier	 T253	 T251	 Supplements

steady	data/source_txt/train/t1_biology_1_0.txt	 2683	 2689	 I-Alias-Term	 T211	 T210	 AKA
state”)—the	data/source_txt/train/t1_biology_1_0.txt	 2690	 2701	 Alias-Term	 T211	 T210	 AKA
DNA	data/source_txt/train/t1_biology_1_0.txt	 3323	 3326	 B-Alias-Term	 T221	 T220	 AKA

I found the following list of TAGS in data.

• 'O',
• 'I-Definition',
• 'I-Term',
• 'I-Secondary-Definition',
• 'B-Term',
• 'B-Definition',
• 'I-Definiti-frag',
• 'I-Qualifier',
• 'I-Alias-Term',
• 'B-Alias-Term',
• 'B-Secondary-Definition',
• 'I-Referential-Definition',
• 'B-Referential-Definition',
• 'B-Qualifier',
• 'B-Referential-Term',
• 'I-Referential-Term',
• 'B-Definiti-frag',
• 'I-Ordered-Definition',
• 'Definition',
• 'Term',
• 'I-Ordered-Term',
• 'Alias-Term',
• 'B-Te-frag',
• 'B-Ordered-Definition',
• 'B-Ordered-Term',
• 'Secondary-Definition',
• 'I-Te-frag',
• 'Qualifier',
• 'Referential-Definition',
• 'B-Alias-Te-frag'

from deft_corpus.

Thanks @sashaspala for resolving the issue

from deft_corpus.

You're right - we don't really have a discussion about fragments in the paper. They're not super common in the textbooks case, but they do come up occasionally when the definition or term phrase is non-contiguous (often when the term is plopped in the definition phrase). I'll add this to the to-do list.

from deft_corpus.

Thanks, FYI this is the tag list I had generated two months ago, but I am not sure if it is still up-to-date:

from deft_corpus.

Any update on the issue??

from deft_corpus.

Thanks @sashaspala, this fixes the tagging issue raised Mukesh Mehta, but was some documentation added to explain all tags, such as B-Definiti-frag?

from deft_corpus.

Updated list of BIO Tags from Train and Dev set

['B-Referential-Definition',
 'I-Referential-Term',
 'I-Alias-Term',
 'I-Qualifier',
 'B-Ordered-Term',
 'B-Ordered-Definition',
 'B-Referential-Term',
 'O',
 'B-Qualifier',
 'I-Term-frag',
 'I-Definition',
 'I-Definition-frag',
 'I-Referential-Definition',
 'I-Term',
 'B-Secondary-Definition',
 'I-Ordered-Definition',
 'B-Alias-Term',
 'I-Ordered-Term',
 'B-Definition',
 'B-Term-frag',
 'B-Definition-frag',
 'I-Secondary-Definition',
 'B-Term',
 'B-Alias-Term-frag']

from deft_corpus.