Comments (4)
Hello @VishalRam, glad that you like it! :)
Unfortunately we're locked to at least 0.17 because we want to be up to date when 1.0 is released. If it goes about Sidekiq, we could be backward compatible with 3.4.1 (same api, etc), however isn't it just easier to upgraded to 4.0 for you, since it 100% compatibility? You can always use older revision of Karafka (we will be shipping via rubygems soon) - we locked those version few commits ago.
from karafka.
@mensfeld - I have certain dependency on Sidekiq 3.4.1. I use a gem Sidekiq-throttled which works only for Sidekiq <4.0.0 versions
from karafka.
They are looking to update it to work with sidekiq 4. Maybe you should wait a couple of weeks for this or you can help with a PR.
I don't think the fix will be very easy, personally I'm using sidekiq-limit_fetch
, and the change for sidekiq 4.0 was not trivial:
deanpcmad/sidekiq-limit_fetch@206e022
from karafka.
@VishalRam I think I've found a compromise solution for you: you can use Karafka only as a proxy server for your logic making it a separate app that will just have same worker names. Meanwhile you can use your app workers (and everything else) the way you use it (as long as none of your sidekiq extensions changes the perform_async method):
- Create a new Karafka application (based on readme)
- Create controllers for topics on which you want to listen
- Assign custom workers (that way you won;t have to have the #perform method and logic in it)
- Run bundle exec karafka server to start listening
- Create "full" proper workers with the same queues names (or leave default) in your "full" application
- Start workers from your "full" application
from karafka.
Related Issues (20)
- Use forking-introduced config importer instead of direct referencing
- Add nicer API for pre-fork Rails and app eager load HOT 1
- Store last error that occurred details in the consumer for retry action referencing
- Introduce intermediate errors tracker for complex recovery flows
- Allow DLQ to accept a callable instead of number for advanced strategy
- Infinite consecutive error flow with VPs and without DLQ can cause endless offsets accumulation
- [Question] Consumer class concurrency HOT 1
- Support partition as a partition key for ActiveJob
- Provide the preload swarm in config HOT 1
- Check if rdkafka and web-ui threads are marked as fork safe to avoid warnings from Puma on forks
- Consider macOS fallback pid based (instead of pidfd) swarm mode
- Change `node_report_timeout` from 30 to 60 seconds
- Consider better events ticking on long polling HOT 1
- Update karafka.erb template with a placeholder for waterdrop and karafka error instrumentation
- Can't run karafka install (uninitialized constant Rdkafka::Bindings::RD_KAFKA_RESOURCE_ANY) HOT 3
- Introduce a pipe benchmarks
- Static membership fencing error is not propagated explicitely enough
- Static membership routing evaluation happens too early in swarm
- Expose direct partition assignment strategy for window based multi-topic operations core HOT 1
- Support early yield on poll via partition eof
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from karafka.