Designed an application to stream sentiment of tweets by providing the twitter topic (#hashtag) as input by using Kafka and Twitter API. Engineered Spark streaming in Scala to process the sentiments using the Stanford CoreNLP libraries. Showcased the visualization of the sentiments of tweets in near real time using Plotly
Deployment Instructions
- Start Zookeeper server
bin/zookeeper-server-start.sh /Users/avikalchhetri/kafka_2.11-0.8.2.1/config/zookeeper.properties
- Start Kafka server
bin/kafka-server-start.sh /Users/avikalchhetri/kafka_2.11-0.8.2.1/config/server.properties
-
Mention the #hashtag in 'TwitterStream.keyword' in Kafka Producer program
-
Start Kafka producer
./gradlew produce
This will start to read recent tweets, encode them to Avro and send to the Kafka cluster
- Start Kafka consumer
./gradlew consume
- Run python script for visualisation in plotly