Request for Assistance with Kafka Client Connections in Kafka Broker Dispatcher about eventing-kafka-broker HOT 3 CLOSED

Hi, thanks for opening an issue, can you please share your setup, how many nodes your Kafka clusters have ? how many Knative brokers with class Kafka as well as how many triggers you have?

from eventing-kafka-broker.

Hi @pierDipi, thanks for your response,

Here's my current setup:

• Kafka Strimzi with 3 nodes: 2 CPU, 4GB memory each
• 30 pods for kafka-broker-dispatcher
• Only 1 broker with 100 partitions and a total of 111 triggers
• config-kafka-broker-webclient.properties uses maxPoolSize=10

After investigation, I noticed that we are using the default setting of Vert.x worker threads with 20 threads. Does this mean that the maximum number of connections will be 20 * 10 * 30, which is around 6000 connections?

Currently, I reduced the number of connections by decreasing the number of pods to 10. Can we control this number not only by decreasing maxPoolSize but also by reducing the number of Vert.x worker threads?

Thanks,

from eventing-kafka-broker.

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

from eventing-kafka-broker.