Project uses ClusterData form Google 2011 traces specifically from task-event table and consists of Stream producer and Stream Consumer connected with Kafka.
Simple application build with Spring & Kotlin to read data from task-event files and forward it do Kafka broker.
Simple script in Python used to perform analytics on task-event data using Apache Spark Streaming. To run it use the following command:
$SPARK_HOME/bin/spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.12:3.2.0 consumer.py
Directory consisting only of docker-compose file responsible for setting up Kafka and Zookeeper.