This repository hosts a Sentiment Analysis on an amazon english reviews dataset using various transformers from Hugging Face.
PyTorch has been used as Gradient Computing Library.

• Bert
• AlBert
• RoBerta
• XlNet
• Electra

For each record:

• Removed html tags
• Case folding

For each record:

• Removed html tags
• Removed non-alpha strings
• Removed single characters
• Case folding

Stopwords have been removed. The nltk english stopwords set has been used.

It was used to reduce the number of tokens since each word is converted to its base form.
The WordNetLemmatizer from nltk has been used.

Preprocessing is done in parallel.

• No. Records: 191486
• Corpus Size: 59692
• Min Sequence Length: 5
• Max Sequence Length: 14004

Sentiment polarity:

• 0: Bad
• 1: Neutral
• 2: Positive

Pre-trained models has been used for extracting features from text (no fine tuning).
A classifier has been trained for obtaining predictions from the extracted features.
Each classifier has been trained for 150 epochs.
70% of the dataset has been used as training set while the other 30% as test set.

The CrossEntropy loss function has been used.
Adam was used as optimezer.

Since the dataset is imbalanced, it was tried to mitigate this aspect by using a weighed loss function. In particular the Inverse Number of Samples (INS) weighting strategy was employed.
INS weights has been used in the CrossEntropy loss function.
Adam was used as optimezer.