Simple example how you can interact with Kafka in python

This example written with poetry so you have to install it first

pip install poetry

git clone https://github.com/Ivan-Feofanov/kafka-python-example.git
cd kafka-python-example
poetry install
poetry shell

Befor you start you have to provide some configuration values as it presented in example .env_example file. You can set them or just rename this file to .env and fill by yourself. Pydantic will parse it and use as application settings.

This example show two ways of Kafka authentification - by certificates and by username and password. You have to choose one of them.

For this example you have to use PostgreSQL database because of using UUID field, but you feel free to fork this repo and change this behaviour.

To start web app you also have to install an ASGI server, such as uvicorn, daphne, or hypercorn.

If you choose uvicorn, you can run development version

uvicorn main:app --reload

It will run web API that will getting messages and passing them in Kafka.

To receive messages and store them in database run receiver

python receiver.py

If api service is running on address 127.0.0.1:8000 you can find usefull API documentation on http://127.0.0.1:8000/docs or http://127.0.0.1:8000/redoc (thanks to FastAPI)

Don't forget to set env variable ENVIRONMENT to test and point your test database in DATABASE_URL.

To run tests use

poetry run python -m pytest

For linter just run

prospector