The sentiment analysis of Indians after the extension of lockdown announcements to be analyzed with the relevant #tags on twitter and build a predictive analytics model to understand the behavior of people if the lockdown is further extended. Also develop a dashboard with visualization of people reaction to the govt announcements on lockdown extension.
Website Link
: TSA
Video Explaination
: https://drive.google.com/file/d/1n9pUBTNtpbAKnvoJ0HTzufntSq60fyFa/view?usp=sharing
PPT Explaination
: https://drive.google.com/file/d/1ZMPTEt26YY8UiGMDeT5MknRKUEoF11lZ/view?usp=sharing
Twitter hosts abundant user-generated posts, which closely reflect the public’s reactions towards this pandemic with low latency.
Collecting and analyzing different demographic data helps compare and contrast that how different groups are affected in society.
Libraries
Numpy
:http://www.numpy.org/Pandas
:http://pandas.pydata.orgmatplotlib
:http://matplotlib.org/seaborn
:https://seaborn.pydata.orgDatetime
:https://docs.python.org/3/library/datetime.htmlStatistics
:https://docs.python.org/3/library/statistics.htmltextblob
:https://textblob.readthedocs.io/en/dev/
Software Recommended
- Jupyter Notebook
- Anaconda installed with
Python 3.8
- IBM Cloud, used IBM Watson Tone Analyzer service
- Data Gathering
- Data Preprocessing
- Analyzing Polarity and Subjectivity
- Analyzing emotions using IBM Watson Tone Analyzer
- Data Visualization
- Creating Visualization Dashboard
Data Gathering
Tweets are scraped using web-scraping. Scraped data is stored in JSON format and then combining each tweet, converted into .csv file. CSV file then goes for data preprocessing.
Click here for dataset.
Data Preprocessing
- Remove Punctuations
- Tokenizing Words
- Stop words Removal
- Stemming
- Lemmatizing
Click here for Data Preprocessing Folder
Analyzing Polarity and Subjectivity
Each Tweet will be subdivided according to Polarity and Subjectivity:
Polarity: Ranges from -1 to +1, -1 being most negative and +1 being most positive comment.
Subjectivity: Ranges from 0 to 1, w.r.t its analytics.
TextBlob
has predefined functions and NLP models embedded in it, which makes this task efficient and smooth.
Sentiments folder contains all files for this process.
Textblob functions file contains imported textblob library and functions are created.
Polarity Subjectivity Analysis file implement those functions on dataset.
Polarity Subjectivity Dataset is created.
Analyzing tweet using IBM Watson Tone Analyzer
Analyzes the sentiment in the deeper way to classify and reveal the trend more comprehensively.
This technique uses linguistic analysis to detect joy, sadness, anger, fear, analytical, tentative and also confidence tones found in text.
ToneAnalyzer folder contains all files for this process.
IBM Watson Tone Analyzer, code for calling API from IBM Cloud Service.
Dataset Creation, seperate dataset is created for each sentiment.
Data Visualization
Technologies used:
- Matplotlib
- Plotly
- Tableau
- Seaborn
Creating Visualization Dashboard
Stack used:
- HTML
- CSS
- JavaScript
- Tableau
- Bootstrap
You can visit Visualization Dashboard for analyzing tweets.