A collection of examples for Strimzi/Kafka, Spark, and Volcano

• Install Strimzi

• Deploy cluster

• Produce / consume messages

• Adjust number of brokers

• Benchmark perfomace

• Deploy Kafka UI

• Topic compression

• Backup / restore

• OIDC

• Rack awareness

• Monitoring

• Cruise Control

• Quotas

• Retention policies

• Scale up/down

• Mirror maker

• autoscaling brokers

• Karapace deployment

• Karapace security

• single node single disk deploy to k8s
• bucket notifications into kafka

• Object input
• Object output
• Auto scaling
• ffmpeg process
• Get args from kafka

Smart guy

Smart guys video

S3 == eventual consistency

S3Proxy - access a file system using the S3 API

s3fs

Goofys

Faster but less POSIX-y

rclone

like rsync for cloud, but offers a fuse mount option

based on MinIO - allows s3 access to SMB/NFS/WekaFS data

Paid solution

treats object store like block device. Lacks native object access.

treats like block storage.