Comments (8)
Can you share the snippet you used to commit offsets?
from kinsky.
Nevermind, I found what the issue is, working on a fix.
from kinsky.
Cool
from kinsky.
If you can, please test latest master, I'm not 100% convinced yet but it's seems to solve the issue.
from kinsky.
I have seen the same thing when doing a resume immediately after a commit. I'm working around it with a good ol' Thread.sleep()
, but that's hacktacular at best.
from kinsky.
@jmglov can you paste the code that triggered the issue?
from kinsky.
Given a control channel control-ch
and a sequence of topic partitions topic-partitions
, this will throw a WakeupException
:
(put! control-ch {:op :pause :topic-partitions topic-partitions})
(do-something-that-takes-a-few-hundred-milliseconds)
(put! control-ch {:op :commit})
(put! control-ch {:op :resume})
This does not throw the exception:
(put! control-ch {:op :pause :topic-partitions topic-partitions})
(do-something-that-takes-a-few-hundred-milliseconds)
(put! control-ch {:op :commit})
(Thread/sleep 1000)
(put! control-ch {:op :resume})
from kinsky.
Hi @jmglov,
I confirm that the following scenario now works:
;; Assuming a topic "test" with 4 partitions
(def t "test")
(def nparts 4)
(def parts (vec (for [i (range nparts)] {:topic t :partition i})))
(def gid (str "testgroup-" (rand-int 1000)))
(Thread/setDefaultUncaughtExceptionHandler
(reify Thread$UncaughtExceptionHandler
(uncaughtException [_ thread e]
(println (pr-str e)))))
(require '[kinsky.client :as client] '[kinsky.async :as async] '[clojure.core.async :as a] '[clojure.pprint :refer [pprint]])
(let [[out ctl] (async/consumer {:bootstrap.servers "localhost:9092" :group.id gid} :string :string)] (def out out) (def ctl ctl))
(a/go (loop [] (pprint (a/<! out)) (recur)))
(a/put! ctl {:op :subscribe :topic t})
(do
(println "pause")
(a/put! ctl {:op :pause :topic-partitions parts})
(println "commit")
(a/put! ctl {:op :commit})
(println "resume")
(a/put! ctl {:op :resume :topic-partitions parts}))
from kinsky.
Related Issues (20)
- Single arity variant of stop! in ConsumerDriver not implemented HOT 1
- Bug in flush! in producer->driver
- Support collection of keyword topics HOT 1
- Verify against Kafka test client? HOT 1
- The async consumer facade looks broken post 0.10.1.0 HOT 2
- How to get an error if Kafka is down? HOT 1
- Kafka AdminClient Wrapper? HOT 2
- Support producer transaction api HOT 1
- Examples does not work HOT 4
- sasl_plaintext auth failed
- Async consumer & manual offset control HOT 5
- Support `.offsetsForTimes` of the consumer API
- Producer errors when using async facade HOT 6
- Subscribing by regex pattern fails with an exception HOT 4
- Are deserialization exception handlers supported HOT 1
- Seek operation in async client HOT 2
- Zombie consumer HOT 2
- Support Kafka Headers HOT 4
- Bump kafka-clients HOT 1
- Why do partition and timestamp default to 0 and 11 in producer record? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kinsky.