olacabs / fabric Goto Github PK

Hi @shaikidris @shashankg @satks ,
I am consuming messages/event from a Kafka topic. Retryer in PipelineStage class is created with "neverStop" stop strategy. If there is an exception in event processing (using StreamingProcessor) due to some issue in message/event payload, retryer simply retries and keeps processing this particular event/message due to never stop strategy. It gets stuck here and does not process next events. Once max.poll.interval.ms is reached this partition gets assigned to another consumer. Same thing happens with this consumer also and it just continues and after sometime all consumers get stuck. Can we use StopAfterAttemptStrategy here to avoid this or is there any alternative or workaround to solve this problem?
Am I missing anything here ?

I tried to create the processor project but the build failed.

mvn archetype:generate -DarchetypeGroupId=com.olacabs.fabric -DarchetypeArtifactId=fabric-processor-archetype -DarchetypeVersion=0.0.1-SNAPSHOT -DartifactId=<artifact_id_of_your_project> -DgroupId=<group_id_of_your_project> -DinteractiveMode=ture

Hi Idris/Sathish

How your wordCount HashMap in the sample code is going to be behave in clustered environment?

Br
Harvinder