Refactor the Core_Daemon::show_help() method to work like show_install_instructions().

Create two instance vars on Core_Daemon:

protected $help_usage = array();
protected $help_options = array();

Change the Core_Daemon::getopt() method to write to these arrays. Change the show_help() method to just implode these arrays and print them with a similar format to the existing code.

This will enable application developers to parse their own command line options by overloading getopt (and calling parent::getopt() when they're done), while writing help instructions that get displayed alongside the library-specific instructions.

Is this deamon suitable for irc applications, and if so, where to start would be appreciated

README.md indicates that version 2.1 has been released, but 2.0 is the only stable version listed on Packagist. There are a couple development 2.1 releases, but it's unclear which one is the release (if any). Can you please tag the v2.1 commit?

I'm currently looking to migrate away from PEAR's System_Daemon and this looks awesome. We're using Composer (http://getcomposer.org/) for dependency management. Would you be open to adding a composer.json file (http://getcomposer.org/doc/02-libraries.md) to this project so it can be easily added to projects that would like to use it?

Hi,

please take a look at #76.

Tnx.

Hi
I'm having problem of getting error:
"Dropped Call. Requeuing Call # To {$call->method}"
And "Dropped Call. Requeue threshold reached.

I can't figure out what is the root cause for that behavior happening more over this is happening occasionally in non consistent way.

The function which triggers this is "public function garbage_collector()".
I tried to do restart when it is happening, but to put the problematic call back to daemon queue by the already written line code:
$this->call($call) (and)
$this->daemon->restart(); (my additional line code)

Questions is, if after daemon restart we already have the same daemon queue state as before or it is probably being initialized?

If we don't maintain daemon's queue state between daemon restart how can I return the call to rabbitMQ that will temporary store the call until daemon will be available again to handle the call.

This should happen somehow from this scope of code (Mediator.php)

Hi,

First up thanks for this code. Just playing around with it and it seems like a nice clear bit of code which does exactly what I need!

I've run into a problem because I am storing database config in a config file loaded by Core_Plugins_Ini. I want to create the database connection in my daemon's setup() function. However, due to the order that things happen the setup() in Core_Plugins_Ini hasn't been executed by the time my daemon's setup() function is called.

How would you normally suggest avoiding this problem? One solution would be to init the plugins before the main daemon (around https://github.com/shaneharter/PHP-Daemon/blob/master/Core/Daemon.php#L269 ) but I'm not sure if this would have unwanted side effects?

The other solution (which I'm currently using as a workaround) is to override run() to look like this:

public function run()
{
    $this->init_db();
    parent::run();
}

This seems pretty hacky though. I guess the other solution is to check in execute() whether the connection has been created and create it if now.

How do you recommend dealing with this situation?

Thanks!

We encountered an issue when rotating and cleaning up log files.
It turned out that when a daemon (automatically) restarts, the log file handle is not closed.
This causes the new daemon process to inherit the handle and the file to remain open.
After the first restart the log file will be reopened causing the file to be opened twice.

Luckily on our system the new log file would get a handle value of 0 and be considered to be
STDIN and closed on subsequent restarts. We could not free up disk space by removing the file since it was still open. We had to truncate the file to free up disk space.

This can be easily fixed for the daemon. I will add a pull request for this.

If you have multiple workers their names can't start with the same letter. It's case sensitive. Took me all morning before I tracked down what was causing the error.

This stems from using ftok which takes a single character string on line 538 of Mediator.php. Some error checking should be added to check for this scenario and throw an exception.

$this->guid = ftok($ftok, $this->alias[0]);

$this->worker('RecordWorker', new RecordWorker(), new Core_Worker_Via_SysV(128 * 1024 * 1024));
$this->worker('RecordUpdater', new RecordUpdateWorker(), new Core_Worker_Via_SysV(110 * 1024 * 1024));

[2014-04-24 11:39:48] 11498               Application Startup Complete. Starting Event Loop.
[2014-04-24 11:39:48] 11503 RecordUpdater  Worker Process Started

Warning: msg_send(): msgsnd failed: Identifier removed in /home/br64538/workspace/daemon/lib/vendor/PHP-Daemon/Core/Worker/Via/SysV.php on line 208

Call Stack:
    0.0002     233184   1. {main}() /home/br64538/workspace/daemon/lib/daemon/run.php:0
    0.9073    6718928   2. Core_Daemon->run() /home/br64538/workspace/daemon/lib/daemon/run.php:15
    0.9075    6720472   3. SNUpdateDaemon->execute() /home/br64538/workspace/daemon/lib/vendor/PHP-Daemon/Core/Daemon.php:414
    0.9075    6720704   4. Core_Worker_ObjectMediator->getStalexChanges() /home/br64538/workspace/daemon/lib/daemon/SNUpdateDaemon.class.php:105
    0.9075    6720992   5. Core_Worker_Mediator->__call() /home/br64538/workspace/daemon/lib/daemon/SNUpdateDaemon.class.php:105
    0.9077    6760544   6. Core_Worker_Mediator->call() /home/br64538/workspace/daemon/lib/vendor/PHP-Daemon/Core/Worker/Mediator.php:851
    0.9077    6761112   7. Core_Worker_Via_SysV->put() /home/br64538/workspace/daemon/lib/vendor/PHP-Daemon/Core/Worker/Mediator.php:824
    0.9078    6763192   8. msg_send() /home/br64538/workspace/daemon/lib/vendor/PHP-Daemon/Core/Worker/Via/SysV.php:208

[2014-04-24 11:39:48] 11498 RecordWorker  SysV::put() Failed for call_id 2: Retrying. Error Code: 43
[2014-04-24 11:39:48] 11503 RecordUpdater  getStalexChanges() is Not Callable.

I would like to start work on adding a Core_Worker_Via_Memcache class. It would make it much simpler to implement this new class if there were some unit tests to run to ensure the results from each method call are behaving as intended.

I thought I had reported this issue before but I can't find it now.

To reproduce run in daemon mode and set a low $auto_restart_interval. Every time the daemon restarts it creates a new msg_queue and new shm mallocs for each worker. This will quickly eat up all your memory and the daemon will eventually be unable to restart.

This is possibly related to #57.

When PHP Daemon tears itself down for an automatic reboot it invokes its own destruct method explicitly. The destruct method loops through worker and plugin aliases, invoking each of their teardown methods and removing the objects

        foreach(array_merge($this->workers, $this->plugins) as $object) {
            $this->{$object}->teardown();
            unset($this->{$object});
        }

However, the aliases are maintained in the arrays. I'm noticing that occasionally the PHP interpreter is invoking the destruct method again. In the second iteration, the workers and plugins are already unset so it attempts to call a method on an undefined property. The error that is raised causes PHP Daemon to attempt to restart itself.... Which calls the destruct method again and the whole party continues.

Because the original restart was actually successful in spawning a new instance of the daemon, you end up with one correctly functioning daemon and a few endless looping walking dead daemons.

I'm using version 2.1.1. Also, thanks for the library and the great documentation! Overall, this has been working great for me.

Some system information: this daemon is running on Mac OS X, using PHP 5.5 installed via homebrew. The current stable in homebrew is 5.5.23.

My daemon launches several tasks to download files from a remote host. I have a condition that occurs when, after whatever the first task is that finishes, the main runloop stops executing (or, at the very least, execute() isn't being called). All of the tasks eventually finish and I can still call log() on the parent, and if I look at a process list the daemon is still "running", but not actually doing anything and no new tasks are created.

Downgrading to PHP 5.5.20 fixes it.

I know this is a frustratingly vague bug report, so if there's anything else I can help you with I would be glad to provide assistance.

Hi,
I get this error when I run a example or when I execute my worker.

PHP Unknown: Declaration of Core_Worker_ObjectMediator::check_environment() should be compatible with Core_Worker_Mediator::check_environment(array $errors = Array) in lib/PHP-Daemon/Core/Worker/ObjectMediator.php on line 12 pid 2483

To reapir it I'have changed the file Daemon/Core/Worker/ObjectMediator.php

-- public function check_environment() {
-- $errors = array();

with

++ public function check_environment(Array $errors = array()) {
++ //$errors = array();

inlcude php files one by one?

thanks

Hi,

I'm really sorry to post such a question as an 'issue' but I can't get it working…
I want to have a daemon that cares about my socket server (php) and in the docs you've written about socket server. So I think it should work but I don't know HOW.

If I don't set a call repeat time and leave out my while(true) nothing will matter. If I leave while(true) the server doesn't respond to inputs. And without the daemon it works well with the while(true) and responds to input.

Maybe someone could write the most important things I should care about to run my socket server with your daemon.

Jan

Hello !

The extension PNCTL is installed on my server (apache2, debian) and work on my scripts but when i try to start run.php of examples (PrimeNumbers), i get this error :

Date                  PID   Label         Message
[2013-11-10 14:57:02]  4524               Checking Dependencies... Failed:
[2013-11-10 14:57:02]  4524                 The PCNTL Extension is not installed
[2013-11-10 14:57:02]  4524               Examples\PrimeNumbers\Daemon is Shutting Down...

But effectively, in php -i and phpinfo() the PCNTL extension is enabled.

A little help or indication about this?

Thanks.

I am using command line arguments to control the behavior of the daemon, but they are lost when the daemon restarts. This makes the auto-restart feature unusable for me.

The problem centers around Core_Daemon::command() where the filename of the script is built. Here some default arguments (-d --recoverworkers) are appended, but the original arguments are not mentioned at all.

This is a large feature that I'm currently thinking about. I'm trying to sketch-out what the feature will look like, and how best to integrate it into the existing codebase.

Goals:

1. Integrate an off-the-shelf queuing solution (eg Gearman, Kestrel, Beanstalkd, ActiveMQ, etc) that could be used instead of the existing SysV message queue.
2. Take advantage of the ability an OTS queue provides to distribute workers across multiple machines in a cluster.
3. Implement this in a way that is idiomatic to PHP Simple Daemon and provides for easily moving between queueing providers.

Implementation Thoughts:

1. Create a new directory Core/Worker/Queues
2. Move all of the SysV-specific code out of Core_Worker_Mediator into Core_Worker_Queues_SysV
3. Add an optional argument onto the Core_Daemon::create_worker() method to choose a specific queue.
4. Probably will need to create a unique interface for each queue, or possibly create a queue object in your daemon that you inject into the Worker (as the optional argument mentioned previously) or something else similar to deal with the unique needs of different queues. SysV needs a malloc for example. Gearman would need to know the ip/port of Gearmand. That sorta thing.
5. I sorta like the dependency-injection style. That would work equally as well for workers implemented as closures as it does for IWorker objects. You would instantiate a Queue object before you create the worker, configure it however you want, and then pass it to create_worker()
6. An emphasis will be placed on making it easy to move between different Queue types, but that may not apply to code written on v2.0 workers. It may be possible, but i'm not going to worry about that. You may have a small price if you want to upgrade to 2.1-style queues. If the penalty is too high, I'd consider creating this feature in a 3.0 branch and still supporting bug-fixes on the 2.x branch.

Any comment/input is appreciated. Is this a feature that interests you? Would you be interested in helping on any part of it? In beta testing?

I'd expect this feature to be on a Jan/Feb 2013 timeline.

Please chmod it to 644 and update your master. Thanks!

In row 265 in Daemon.php the following seems to be mistaken: "if (is_numeric($this->loop_interval) == false)".

if loop_interval is NULL which is what is supposed to occur when working in blocking mode then this causes an exception. Removing this row works nicely so can you please check if this is indeed a typo or if I'm missing anything?

Thanks!

Hi,I want to try this php-daemon,but I am having difficulty on how to implement this,I created as socket listener in php and I want to apply this php-daemon to my project.because my listener will exit if there is no client connection and I re-run this by using task-scheduler in every 1 min.but I don't want to exit my script I want to run forever. I hope you can help me

Thank you in advance.

in

public function offsetSet($offset, $value)
    {
        if (is_scalar($offset))
            $this->contents[$offset] = $value;

        throw new Exception('Could not set INI value: $offset must be a scalar');
    }

You need to make that an if else as setting an INI will always crash with the Exception.

So it is :) It must be protected to allow setting any application env variables (shutdown status, for example).

Hi i was wondering if it was possible to allow creating constructors for worker classes that implements Core_IWorker. Much like creating constructors for classes that implements \Core_ITask. It would be quite beneficial to have this feature.

Btw, i love the work you've done on PHP-Daemon. I always dreamt of an efficient creation and management of processes in PHP, and PHP-Daemon made that dream come true. Its a fantastic package overall. Many thanks for the hard work that you've done. Keep it alive and rocking !!

Bug: This foreach loop in Daemon::__destruct() does not behave as expected:

foreach($this->workers + $this->plugins as $object) {
    $this->{$object}->teardown();
    unset($this->{$object});
}

The '+' operator performs an array union using keys, not the values. In this case the array indices may conflict, causing some array values to be excluded. See the note here: http://php.net/manual/en/language.operators.array.php#86379.

Impact: This bug prevents the teardown() function from being called on the complete set of workers and plugins, for example on the Lock_File plugin object. As a result, the lock file may not be unlinked, so restart() fails after the auto_restart_interval with the message "Core_Lock_File::set Failed. Additional Lock Detected."

Solution 1:

foreach($this->workers as $object) {
    $this->{$object}->teardown();
    unset($this->{$object});
}
foreach($this->plugins as $object) {
    $this->{$object}->teardown();
    unset($this->{$object});
}

Solution 2:

foreach(array_unique(array_merge($this->workers,$this->plugins)) as $object){
    $this->{$object}->teardown();
    unset($this->{$object});
}

Using native getopt method from php it doesn`t let you alter the arguments list in order to get the corresponding arguments correctly.

For example, you may access the daemon from a framework, so "php bootstrap.php -d" won`t daemonize the process.

Description

Share memory lock plugin was broken.

Reproduce

To verification, just change the Lock_File in Example/LongPoll to Lock_Shm. You will be able to run more than one instance.

It is possible to launch multiple instances of daemon even when using file lock plugin. For example I run one daemon instance, than after Core_Daemod::loop_interval seconds i can launch more instances of the same daemon. I don't understand the logic of TTL checking, till it equals to Core_Daemod::loop_interval, which is usually a few seconds... I commented out lines 85-87 of Core/Lock/File.php to use this plugin for now

Ok, to start off, I know this isn't an issue or something of that kind but since this package doesn't have a huge following on any forum or whatever, I figured I may aswel try here.

I want to make a daemon script in PHP that basically runs for every user I have in my application. I also want my users to have the ability to stop such a process whenever they want. And also start it back up again.

Is this the package I've been looking for? The average run time of the script is like 2 hours. Sometimes they run for 5 minutes and the user stops it, sometimes they run for a full week.

Thanks for those that considered answering and to those who read this unstructured message. I use everything ranging from Job Queue's to Cron Jobs but I can't seem to find something that completely suits my needs.

In my application I have registered a handler for the ON_SHUTDOWN hook, which cleans up some network connections. During regular shutdown, the hook is executed fine, but when the daemon is automatically restarted, the hook is not called, resulting in lingering connections.

The reason for this is that the hook is dispatched from the destructor of the Daemon class, which in the case of an automatic restart is called when the exit call is made in the restart() method. Unfortunately, for non-intuitive reasons, right before the exit call is made, the array containing all the callbacks is cleared, so once the destructor runs, there aren't any hooks leftover to call.

Mediator::process() can return null and direct call made at line 724

$this->process($call->pid)->kill();

can raise a fatal error

I have 2 Damons, for 2 differnet purposes.
From ObjectMediatior.php I see the ftok is always the same, this means they both seem to want to use the same msg_q

$tmp = sys_get_temp_dir();
$ftok = sprintf($tmp . '/%s_%s', str_replace('/', '_',
$this->daemon->get('filename')), $this->alias);

Am I missing something about configuring my env to allow multple seperate
Daemons run on the same box.

Hi,

congrats on php-daemon! I've just started looking into the framework and find it quite cool and complete! Especially good the cluster-based functionality, very useful. A couple of questions/comments maybe you can help steer me:

• my first daemon needs to poll a particular webservice queue which wasnt developed by us. Nonetheless, the webserver is "blocking", and when it reaches the last even it will "wait" 10 seconds... basically, I need a non-time-looped daemon. Would this be something possible to implement? Basically, whenever it reaches the end, start again... I could just put 1 second interval, but that would flood my e-mail box, and potentially give other complications? (Is there somewhere to disable these catches? would this be the best way?)
• on another note, do you see any implications in porting the memcache portion of your code to use Redis instead? We have changed from memcache to redis for most of our use, and were thikning of disabling the memcache cluster, but we could always keep it for this daemon control (although i'd prefer to just port a lock plugin for redis).

Anyways, great work. I think this will be very useful in several jobs, and when combined with a message queue such as php-resque or whichever one anyone prefers, gives us a complete toolbox for work...

I was thinking it would be nice to have an additional commandline option which would kill (or possibly restart) any processes (started with -d) that are currently running for a given daemon.

My use-case is because I'm still developing the code for my daemons. So occasionally I deploy new code to the server and have to do a kill pid to get the new code read in and executed. It would be nice if there was a way to do this without first looking up the pid etc...

Recently I need to write a program to pull message from SQS constantly. Then I found this library to do it. I extended the abstract class Core_daemon and wrote main function of my program in execute(). When I ran the daemon process, I found the daemon process produced a lots of defunct processes. Is it normal situation? or something I‘ve done wrong?

I've build a deamon with several workers with PHP-Daemon v2.0
At a certain point of time (after approximately 8 hours with 8 workers) the Daemon reaches an "Fatal Error" and tries to restart the worker processes. There's probably something wrong with the memory allocation. Because if I restart the Worker (and automaticly flush the memory) it works like before.

See in the detail here:

[2015-06-10 05:56:04] 13940 mitchell Message Queue Error 13: Permission denied]
[2015-06-10 05:56:04] 13940 mitchell Incrementing Error Count for catchall to 24
[2015-06-10 05:56:04] 13940 mitchell Message Queue Error 13: Permission denied
[2015-06-10 05:56:04] 13940 mitchell Incrementing Error Count for catchall to 25
[2015-06-10 05:56:04] 13940 mitchell Message Queue Error 13: Permission denied
[2015-06-10 05:56:04] 13940 mitchell IPC 'catchall' Error Threshold Reached
[2015-06-10 05:56:04] 13940 mitchell Fatal Error: Worker process will restart
[2015-06-10 05:56:04] 13940 WheresAppDaemon\Daemon is Shutting Down...
[2015-06-10 05:56:07] 13940 Restart Happening Now...

Date PID Label Message
[2015-06-10 05:56:32] 18615 Invalid Alias. Identifiers must be scalar.
[2015-06-10 05:56:32] 18615 WheresAppDaemon\Daemon is Shutting Down...
[2015-06-10 05:56:34] 18615 Restart Happening Now...

Date PID Label Message
[2015-06-10 05:56:36] 18620 Invalid Alias. Identifiers must be scalar.
[2015-06-10 05:56:36] 18620 WheresAppDaemon\Daemon is Shutting Down...

Also when the Daemon tries to restarts itself I got the message Identifiers must be scalar. Maybe this is related to the fact that there's no memory left for the Daemon. I run a Vagrant hashicorp/precise32 Ubuntu machine with 2048MB of memory.

is_idle() is nice, but it would actually be nice to know how many tasks are running (or have a "worker slots available"), in order to more efficiently pop jobs off of a work queue in each execution state of the event loop.

Core_Worker_Mediator::shm_init Failed. Could Not Read Header. If this problem persists, try running the daemon with the --resetworkers option

tried with the --resetworkers option

php run.php --resetworkers

but the error still persists ...

Any idea how it can be resolved?
Its running PHP5.4.8

Thanks

Hi!

The last commit on this project was 8 months ago, there are numerous issues without responses, no apparent development. I am using this tool heavily and am waiting for the changes promised for v2.2.

@shaneharter are you still maintaining this project? We need you :)

Thanks
tobias

Keep getting this one when a worker dies and the whole daemon dies.

Hi,

I am building a solution that crawls some websites, and after a while the following messages are "dumped" in the log.

[2014-04-29 13:32:11] 6013 Workers SysV::put() Failed for call_id 2: Retrying. Error Code: 11
[29-Apr-2014 13:32:11 UTC] PHP Warning: msg_send(): msgsnd failed: Resource temporarily unavailable in /var/www/worker/includes/daemon/Core/Worker/Via/SysV.php on line 208 pid 6013

I am currently backtracing -and debugging the root cause, but also have tried to find an explanation of the Error code provided, so far I have not been able to find any documentation of the error code. Any ideas?

1. Abstract all of the SHM functionality into an SHM storage class.
2. Create additional storage options (Redis, Memcached, etc) and implement the ability for application developers to choose a specific storage class. Create hueristic that will try to select the optional storage engine based on worker throughput and mean arg/return value size.

The idea here is that we've learned from this first release of the Worker API that shared memory has faults and is, you could say, not the most rock solid PHP API available. And if Redis or Memcached are available on the server, one or both could be a better option than SHM.

I am trying to create an application which reads messages off of a distributed queue and processes each of them. As there are very many messages coming in (thousands per second), high throughput and performance are essential for the app.

I looked into using this daemon framework as basis for the app, but found out that despite setting the $loop_interval to 0, the maximum number of iterations per second is 100. While this may be more than enough for other, less intense workloads, it is far too little for my use case.

The code comments suggest that setting the variable to 0 would result in the loop being executed continually without delay (except for the occasional ON_IDLE event), but I found out that even when with the value 0 there is a call to usleep() which wastes a huge amount of time.

The call is in Core_daemon::timer() on line 883 (as of this writing). There the process is set to sleep for 10ms, effectively limiting the maximum number of iterations per second to 100. The comments justify doing this by needing to "give the CPU a break", but 10ms is a looooong time in CPU terms.

Please reduce the length of the break or entirely remove it, so that high-throughput, high-performance apps can also benefit from this otherwise well-written framework. I suggest 100µs as delay in between iterations.

If a worker process dies for some reason then after the timeout interval Daemon::__destruct() calls teardown() on a null object. The result is PHP Fatal error: Call to a member function teardown() on a non-object. This could be handled gracefully by checking that the worker object is non-null before calling teardown().

Bug:

$this->{$object}->teardown();
unset($this->{$object});

Solution:

if(isset($this->{$object})){
    $this->{$object}->teardown();
    unset($this->{$object});
}

Hi,

Since you store the handle to the log file in a static variable the file is only opened once per process. In my case I have a process which has run across a date boundary and I'm using an implementation of log_file() based on date('Ymd').

Since the handle is already there the code never re-calls log_file() and doesn't find out that there should be a new filename so it carries on writing to the old log file.

Possible solutions would be:

• Closing the handle every call to log
• Adding another variable which stores the filename for the current handle and if this is different to that returned by log_file() then close the handle and open a new one
• ...or something else?

I think I prefer the second option and can implement it with a pull request if you like but maybe I'm missing something and there is a better third option?

Thanks,

Kelvin :)

Considering the last paragraph of the wiki page:

If all of that was overly complex I apologize, but you can suffice to say this: Signal handlers can be complex and nuanced. But you should know that any code you create or spawn from within a signal handler will itself be unable to handle signals.

Would a possible solution be to create a callback of some sort and then toss it into an execution queue, to be run outside of the signal handler perhaps? It would introduce a slight delay to execution, but I believe it would allow working around that pitfall.

where i use -d ,it will log every work status message like 'Job x Is Running','Job x Is Running' in lo file,how can i disable it ,let it just log what i want?