Details of pretrained model about dialogtag HOT 6 OPEN

I tried bert-base-uncased, distilbert-base-uncased and bert-large-uncased. The difference between these models was around 1-1.5 F1 score each with bert-large-uncased performing best. However, I feel it was perfect case of overfitting. bert-base-uncased should be sufficient for this problem. I framed it as a multi-class classification problem by classifying sentences from around 38 intents.

If you are planning to work on it, you can look at existing solutions here (https://nlpprogress.com/english/dialogue.html)

from dialogtag.

from dialogtag.

Further, what kind of model did you use when training? I understand it was a multi-class classification problem, but what was the training process? Thanks!

from dialogtag.

This is similar to a lot of the questions raised in #2

from dialogtag.

Just following up on this. I would like to cite this library in a paper I am publishing. Can you please provide more details, at least with the type of model you used to train the classifier?

from dialogtag.

Hi @angoodkind, Apologies for the delayed response. As mentioned in my comment previously,

I tried bert-base-uncased, distilbert-base-uncased and bert-large-uncased. The difference between these models was around 1-1.5 F1 score each with bert-large-uncased performing best. However, I feel it was perfect case of overfitting. bert-base-uncased should be sufficient for this problem. I framed it as a multi-class classification problem by classifying sentences from around 38 intents.

The model you used depends on how you called the API model = DialogTag('distilbert-base-uncased'), it calls the model with finetuned weights of model name you provided. Since it was a multi-class classification problem, I used CrossEntropyLoss as my loss function for ground truth intent and predicted intent.

from dialogtag.