This is a repostory that supports the presentation I gave at the DataEngBytes conference in Melbourne. It is basically a rip off from this tutorial created by the wonderful Aiven DevRel team!

This repository is a list of resources that was enumerated in the slides, with a small terraform script for those who don't feel like clicking the buttons.

The conference talk was built using

• Apache Flink documentation
• Aiven for Apache Flink documentation
• Aiven Flink anomaly detection tutorial
• Aiven Terraform provider

• Terraform installed
• Having psql on your path (probably an install of Postgres - this was 15)
• It might be handy to have the Aiven CLI (but not necessary)
• Courage...

• Create yourself an Aiven account. Free trials are available
• Create the authentication token
• Run the terraform script
• Build and run the data generator

> docker build -t fake-data-producer-for-apache-kafka-docker .

> docker run fake-data-producer-for-apache-kafka-docker

The terraform script will build the three Aiven services required for the demo.

This was built and tested with

> terraform version
Terraform v1.5.0
on darwin_arm64

Clone this repository and go to the terraform directory

> cd terraform

Initialise

> terraform init

Plan

> terraform plan

If you like what you see, go ahead and deploy

> terraform deploy

Once deployed, run some data into it by

> cd ../
> ./buildAndRun.sh