Create forking worker about backburner HOT 9 CLOSED

I think that's a critical feature of resque which might prohibit me from switching to beanstalkd/backburner. Have you received reports/issues of users having this type of problem?

from backburner.

Yes I have a few people that mentioned to me running into this issue, I have not had this problem yet in my own applications. It really depends on the types of jobs you are running. So far I just create multiple workers (separate processes) and have them process the jobs similar to delayed_job: https://github.com/collectiveidea/delayed_job

I am not convinced that a forked worker is necessary for a large set of cases. That said, I am actually interested in having a threaded and forking worker available so people can choose based on their needs.

from backburner.

@ShadowBelmolve

Check out:

https://github.com/nesquena/backburner/blob/workers/lib/backburner/worker.rb
https://github.com/nesquena/backburner/blob/workers/lib/backburner/workers/simple.rb

What do you think of this? I kind of like using subclassing here to create workers because it requires less code in each individual worker and then usage of workers to be more explicit.

If you like it, can you try porting the forking_thread_worker to this format?

from backburner.

@nesquena yes. My only problem is with tests. I'm not good with this yet so I'll make only the basic ones.

from backburner.

Yeah threaded+forked worker are probably not the easiest to test but let's try and make sure it at least ensures the jobs are all processed perhaps by logging to a file (you can specify the logger backburner uses in configuration) and then checking the file to ensure processing occurred. What do you think? However we can verify basic processing correctness.

Nathan Esquenazi

On Thursday, November 8, 2012 at 1:38 PM, Renan Tomal Fernandes wrote:

@nesquena (https://github.com/nesquena) yes. My only problem is with tests. I'm not good with this yet so I'll make only the basic ones.

—
Reply to this email directly or view it on GitHub (#3 (comment)).

from backburner.

Good reference for a simple forking worker michaeldwan/stalker@3862676 by @michaeldwan

from backburner.

FYI, I've got a start on this here:

https://github.com/danielfarrell/backburner/tree/forking-worker

I need to write a test still before I send a pull request.

from backburner.

@danielfarrell Awesome, thanks Daniel. Let me know when the tests are ready and I will merge and update all the documentation.

from backburner.

Closed with 0.3.1 thanks to @danielfarrell

from backburner.