Schema gets updated with every message produce about xk6-kafka HOT 10 CLOSED

@saad1200
I suppose this can be fixed by trying to fetch the schema first before posting one. We can also use something like srclient and have it handle everything, but it's an extra dependency for just two HTTP requests.

Update:
Upon further investigation, I found out that it uses the internal HTTP client of Go, goavro v2, and jsonschema v5 as dependencies, so it's okay for our use-case. I'll try to see how I can integrate it and make a PR when ready.

from xk6-kafka.

@mostafa sorry for the delay. I will block some time next weekend to test the changes.

from xk6-kafka.

@kusumkappdirect
If you are using latest release and still having this issue all you need to do is to setup wiremock on the dev server or locally, then in the k6 setup method get the schema from the schema registry and stub the methods to return that schema using wiremock. Pass wiremock address instead of the schema registry to the k6 extension produce method.
I will share my code with you soon

from xk6-kafka.

Any update on this?

from xk6-kafka.

Hey @saad1200 @ssam5532!

Can any of you provide an example script (and/or configuration) you use, so I can test it? Or at least explain more details.

from xk6-kafka.

@mostafa Any produce code example from the repo can be used to reproduce this. It has been few months now, but I think it is an issue with goavro lib.

we noticed that xk6-kafka is posting a new schema when called with produceWithConfiguration. Looks like it’s because they’re stripping out some of the java specific types and before posting the schema to the registry (which I guess they need to do, to get an ID).

If you look at line 54 and 56, goavro’s codec is turning the schema into and object, then making it into a string again on line 56.

from xk6-kafka.

@ssam5532 our work around is to setup wiremock and use it as schema registry proxy. It is another dependency and far from ideal solution but allowed us to continue using K6.

from xk6-kafka.

@saad1200 @ssam5532 I'd be happy to have your feedback on #52.

from xk6-kafka.

@saad1200 @ssam5532
The PR (#52) is merged now, and you can test it to see if it fixes the issue or not. I haven't released a version yet, so you need to build the latest main according to xk6 build instructions to test it. Let me know if you face any issues along the way!
Also, based on my tests, the schema doesn't get updated.

from xk6-kafka.

@saad1200 , can you please help me here. , my schema also get updated with each sent message on topic , how you retrieve the schema first and then send same ?

from xk6-kafka.