mxhofer / named-entity-recognition-bidirectionallstm-cnn-conll Goto Github PK

I tried to install validation library from pip and run the packages.
It seems like compute_f1 is no longer avaliable.
Is scikit learn f1 score a better choice?

hi,

I see that you are padding the inputs to get equal length of 52. But, it seems padding is applied to only character inputs but not to words.

 # 0-pads all words
 def padding(Sentences):
     maxlen = 52
     for sentence in Sentences:
         char = sentence[2]
         for x in char:
             maxlen = max(maxlen, len(x))
     for i, sentence in enumerate(Sentences):
         Sentences[i][2] = pad_sequences(Sentences[i][2], 52, padding='post')
     return Sentences

Sentences contains below:

         dataset.append([wordIndices, caseIndices, charIndices, labelIndices]) 
     return dataset

I see that you have made batches of inputs with words of equal length. Is this the correct approach?
Can you pls let me know.

Hi,

I own the Ontonotes dataset in folders but I am looking to convert them to the same style as the CONLL2003 .txt files that are provided in this repo. Could the authors provide some details about preprocessing Ontonotes dataset so that Ontonotes could train as well?

Thanks!

def padding(Sentences):
maxlen = 52
for sentence in Sentences:
char = sentence[2]
for x in char:
maxlen = max(maxlen,len(x))
for i,sentence in enumerate(Sentences):
Sentences[i][2] = pad_sequences(Sentences[i][2],52,padding='post')
return Sentences

Please, explain what is actually happening here. It will be beneficial to me if you explain with a example.

Hi, I would like to ask another question. As I would like to prepare my own data to test out the NER.
I would like to know whether the tense including POS tag and Chunk tag matters? Can I just have the Word Tag and NER tag is sufficient?

Is there any tool to annotate words and gives this format of train.txt ?

Which TensorFlow and Keras are used? thanks

for testing the new sentences in a string format do you have an inference code?
Thanks