Multiple Kafka producers generated for each thread in high-concurrency test about xk6-kafka HOT 3 CLOSED

@rrubiostuart

I am afraid that's not possible then. And since VUs don't share context (as in, they are independent of each other, AFAIK), this means that creating a reusable connection won't be possible, and you won't be able to share the connection. You can use iterations, though, as all iterations of a VU will share the same connection.

If you want connection pooling (as far as I understood), it should (and possibly can) be handled in the Go code (in the plugin), but I don't have time to invest on this. However, I'd be very happy to see contributions.

from xk6-kafka.

Hey @rrubiostuart,

Based on the test lifecycle docs, the init code runs once per VU, which is why 5000 connections are created to connect to Kafka. The setup and teardown functions run only once and I recommend creating the connection in the setup function and closing them in the teardown. What error do you see when you say "encountered issues with accepting objects with functions like a Kafka producer"?

from xk6-kafka.

Hi @mostafa, this is the error we faced if we return the kafkaProducer
level=error msg="error marshaling setup() data to JSON: json: unsupported type: func(goja.FunctionCall) goja.Value"

Reading the documentation I see that the setup comes with a restriction "You can pass only data (i.e. JSON) between setup and the other stages. You cannot pass functions."... so how do you propose to create the connection in the setup? Thanks in advance!

from xk6-kafka.