barryvdh / laravel-async-queue Goto Github PK
View Code? Open in Web Editor NEWLaravel Async Queue Driver
Laravel Async Queue Driver
hello sorry I speak english a little bit, my problem is :
the database in the remote server have the queue, but don't execute
I don't understand the error :-(
I'm using this driver for my tests only as a way of simulating background processing without actually having to run a queue just for tests. It's much better than the sync driver since it ensures that my test passing isn't strictly dependent on sequential execution, but every item I push in a test spawns a new process:
php artisan queue:async <number> --env=testing --delay=<some delay>
.
It looks like these never exit, even on job completion or exit so I eventually run out of memory and start paging. Even when phpunit complete there are still lots of them left.
I'd love to be able to keep using this driver, but at the moment the only way I can get passable execution time on my test suite is by restarting my development VM after each test run. Any suggestions?
My issue is I've use async queue as an internal queue.
I got my beanstalkd as a default and grab the external queue from it.
In handler, I then re-assign to async queue so it unlock the worker from grabing another queue from beanstalkd.
Worker eventually timed out. even the handler is finished working.
Here's my controller code.
<?php
class QueueController extends BaseController {
public function fire($job, $data)
{
\Queue::connection('async')
->push(function($job) use ($object) {
$object->doSomething(); // This is time consuming process.
$job->delete();
});
$job->delete();
}
Here's my worker command.
php artisan queue:listen --queue=beanstalkd
When i try to install using composer require barryvdh/laravel-async-queue
Am getting these errors
the command
php artisan queue:table
actually creates a table named jobs instead of queue_jobs. Just changing the async connection configuration solves the problem
I'm using async with gmail email smtp. It's not working
Here's my queue.php
'async', 'connections' => array( 'sync' => array( 'driver' => 'sync', ), 'async' => array( 'driver' => 'async', ), 'beanstalkd' => array( 'driver' => 'beanstalkd', 'host' => 'localhost', 'queue' => 'default', 'ttr' => 60, ), 'sqs' => array( 'driver' => 'sqs', 'key' => 'your-public-key', 'secret' => 'your-secret-key', 'queue' => 'your-queue-url', 'region' => 'us-east-1', ), 'iron' => array( 'driver' => 'iron', 'host' => 'mq-aws-us-east-1.iron.io', 'token' => 'your-token', 'project' => 'your-project-id', 'queue' => 'your-queue-name', 'encrypt' => true, ), 'redis' => array( 'driver' => 'redis', 'queue' => 'default', ), 'async' => array( 'driver' => 'async', 'binary' => 'php', 'binary_args' => '', ), ), 'failed' => array( 'database' => 'mysql', 'table' => 'failed_jobs', ), ``` ); Here's the code ``` Mail::queue('emails.welcome.welcome', array('firstName'=>Input::get('firstName')), function($message) { $message->to(Input::get('email'), Input::get('firstName').' '.Input::get('lastName'))->subject('Welcome to eventsRack!'); }); ```composer log
Problem 1
- Conclusion: remove laravel/framework v5.2.3
- Conclusion: don't install laravel/framework v5.2.3
- Installation request for barryvdh/laravel-async-queue 0.4.x -> satisfiable by barryvdh/laravel-async-queue[v0.4.0].
Because in SerializesModels trait is __sleep() magic method..
The payload-field in the table can only store 65536 characters due to TEXT-type. When you pass lots of data to the closure the Queue won't work because the payload gets trimmed after 65536 characters, using LONGTEXT solves this problem.
With best regards,
Andreas Pfurtscheller
The Async queue is working fine in my staging server, and it was working on my development machine yestarday just fine.
The thing is that today I'm getting an error:
ErrorException in AsyncConnector.php line 22: Undefined index: table
I really don't know why this is suddenly happening because I didn't change nothing on the project and I cannot find information for that error.
Thank you.
Hello,
I have installed and configured all follow the instruction, they just finished all of them and worked fine. But when I request the API from postman it just the sync request, I just figure out this problem because I have a tried to put a sleep(60) command in the script, and then request another API calling, it is stuck just because of sleep command so it can not be async.
Furthermore, when I check the Jobs table I saw that none of the queues is inserted into the table.
Anyone have this problem?
Thanks
Queues are not working whenever I try to send emails with it. (e.g. with mail notification or a call to Mail::to
). When I run the command php artisan queue:async <id>
, it gives me this exception
[Symfony\Component\Debug\Exception\FatalThrowableError]
Call to a member function getName() on null
php artisan make:notification TestNotification
.toMail
method should contain the following codepublic function toMail($notifiable)
{
return (new MailMessage)
->line('The introduction to the notification.')
->action('Notification Action', url('/'))
->line('Thank you for using our application!');
}
email
attribute.User::find(1)->notify(new TestNotification);
jobs
table, and try with php artisan queue:async <id>
.Mail::to
.php artisan make:job TestJob
php artisan make:mail TestEmail
handle
method, add the following codepublic function handle()
{
Mail::to('<email address>')->send(new TestEmail);
}
dispatch(new TestJob);
jobs
table, and try with php artisan queue:async <id>
.i'm using "jenssegers/mongodb" package, i've to make changes at the following files to make it work,
src/Models/Job.php
change use Illuminate\Database\Eloquent\Model;
to use Jenssegers\Mongodb\Model;
src/AsyncQueue.php
find the following line,
$string = 'php artisan queue:async %d --env=%s ';
change the %d
to %s
so it'll become,
$string = 'php artisan queue:async %s --env=%s ';
Hi,
The driver runs correctly, but I need to notify the user via alert when queue ends successfully or not.
Thx!
It's Can't work on laravel 5.4.
Have plane to update?
Queue pushes are working without any issue. But, If i try to run queue:work daemon, It is throwing an error
production.ERROR: exception 'InvalidArgumentException' with message 'No connector for []' in QueueManager.php:129
All queues are listed in queue table. But, It is not executing.
I add service provider into config/app.php and update config/queue.php,
but it cannot work.
Hello,
My mail is not sending though I follow async configuration.
But in sync driver it works just fine.
was there any restriction on using async?
First of all i like to thank you for this awesome plugin.
I have created a job class called UploadToDropbox
and the class is as follows
<?php
namespace App\Jobs;
use App\User;
use App\Jobs\Job;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Bus\SelfHandling;
use Illuminate\Contracts\Queue\ShouldQueue;
use League\Flysystem\Dropbox\DropboxAdapter;
use League\Flysystem\Filesystem;
use Dropbox\Client;
use Image;
class UploadToDropbox extends Job implements SelfHandling, ShouldQueue
{
use InteractsWithQueue, SerializesModels;
protected $file;
protected $user;
protected $mainFolderName;
protected $subFolderName;
protected $filename;
/**
* Create a new job instance.
*
* @return void
*/
public function __construct($file, User $user, $mainFolderName, $subFolderName, $filename='file')
{
$this->file = $file;
$this->user = $user;
$this->mainFolderName = $mainFolderName;
$this->subFolderName = $subFolderName;
$this->filename = $filename;
}
/**
* Execute the job.
*
* @return void
*/
public function handle()
{
$this->uploadToDropbox1($this->file, $this->user, $this->mainFolderName, $this->subFolderName, $this->filename);
}
private function uploadToDropbox1($file=null, $user=null, $mainFolderName=null, $subFolderName=null, $filename='file') {
dd('called');
}
}
And i am trying to dispatch the job from controller, and the code is as follows
$job = new UploadToDropbox($filecontents, $user, $mainFolderName, $subFolderName, $filename);
$this->dispatch($job);
But i don't know why everytime i am getting "Class (blank) does not exist". Please have a look at the attached image
Please help me to resolve this. Thank you.
First of all i like to thank you for this awesome plugin.
I have created a job class called UploadToDropbox
and the class is as follows
<?php
namespace App\Jobs;
use App\User;
use App\Jobs\Job;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Bus\SelfHandling;
use Illuminate\Contracts\Queue\ShouldQueue;
use League\Flysystem\Dropbox\DropboxAdapter;
use League\Flysystem\Filesystem;
use Dropbox\Client;
use Image;
class UploadToDropbox extends Job implements SelfHandling, ShouldQueue
{
use InteractsWithQueue, SerializesModels;
protected $file;
protected $user;
protected $mainFolderName;
protected $subFolderName;
protected $filename;
/**
* Create a new job instance.
*
* @return void
*/
public function __construct($file, User $user, $mainFolderName, $subFolderName, $filename='file')
{
$this->file = $file;
$this->user = $user;
$this->mainFolderName = $mainFolderName;
$this->subFolderName = $subFolderName;
$this->filename = $filename;
}
/**
* Execute the job.
*
* @return void
*/
public function handle()
{
$this->uploadToDropbox1($this->file, $this->user, $this->mainFolderName, $this->subFolderName, $this->filename);
}
private function uploadToDropbox1($file=null, $user=null, $mainFolderName=null, $subFolderName=null, $filename='file') {
dd('called');
}
}
And i am trying to dispatch the job from controller, and the code is as follows
$job = new UploadToDropbox($filecontents, $user, $mainFolderName, $subFolderName, $filename);
$this->dispatch($job);
But i don't know why everytime i am getting "Class (blank) does not exist". Please have a look at the attached image
Please help me to resolve this. Thank you.
start /B "php" artisan queue:async 15 --env=production --delay=0 > NUL
Doesn't work properly for windows
Should be:
start /B php artisan queue:async 15 --env=production --delay=0 > NUL
Hello,
I'm receiving 'Fatal Error: Class 'Barryvdh\Queue\AsyncServiceProvider' not found'
Laravel 5.1
My composer file
"require": {
"php": ">=5.5.9",
"laravel/framework": "5.1.*",
"barryvdh/laravel-async-queue": "0.4.x@dev",
"laravel/socialite": "2.0"
},
App.php
Barryvdh\Queue\AsyncServiceProvider::class,
I try to send mail with "Mail::later" and "Queue::Later". It works when I tested in local(with wamp) whereas in my online website, it doesn't work. When I took a look in "laq_async_queue" table", all job have the status 0 ( "start process").
When I add a job in local, they have status 1.
Best regards
PS: sorry for my english
Hi Barry,
I was trying to use your package in one of the application that I am developing whereas it creates the record in JOBS table but it didnt get executed automatically due to which I need to put a route in place but to execute the jobs in queues it does have dependancy whereas it adds few more records to JOBS table.
I have used 0.4.2 tagged package. Appreciate your help if you can share some guidelines on how exactly I can eliminate the route for manual execution and have it managed automatically so that it is ASYNC but processes immediately once a record is added in JOBS table.
Thanks,
Vik
Hi,
I have this class which i included in composer.json:
"files": ["app/lib/coolstuff/A.class.php"]
A.class.php:
<?php
require_once 'functions.php';
class A
{
public function __construct()
{
if (function_exists('foobar')) {
file_put_contents("log.txt", "yes");
}
else{
file_put_contents("log.txt", "no");
}
}
}
And functions.php :
function foobar(){
// do something complicated
}
when using async, function foobar isn't defind but functions.php is included ( using get_included_files ) log.txt has "no" , log.txt path is root path
when using sync driver, foobar is defined and log.txt has "yes" , log.txt path is public_path
Hi, i have a problem with the config.
When i run the command 'php artisan queue:listen' ... I get the following error:
[InvalidArgumentException]
No connector for [async]
I need send a push notification
Thanks!
Just as the title says, but if I run a queue listener (php artisan:queue
) the queue in DB clears itself after running. Is it because I setup the queue wrong? I have the following in my config/queue.php
...
'async' => [
'driver' => 'async',
'table' => 'jobs',
'queue' => 'default',
'expire' => 60,
],
...
Thanks in advance!
I created an async job for a long processing tasks, it gets correctly executed asynchronously when the app is running via apache mod_php, but it gets executed synchronously when the app is running via the PHP CLI server mode.
I'm doing delete & insert to a table report; listening to every change row of another table. It's working fine on first update. On second update, the event is triggered, there is queue in laq_async_queue table with status=2 but it stuck there, not executed, no error log at all. Any idea? Can it be not allowed same payload in queue?
edit: Can not be error in my code since it'll throw error if there's a syntax error.
edit: I executed php artisan queue:async {jobId}
myself with no problem.
The very latest version doesn't work on hhvm 3.4.0 on ubuntu 14.04.
Hi Barry
I'm completely new to laravel and your laravel-async-queue and therefore I don't know if the table created with the queue:table command has changed in 5.3 or if it's because of the new notifications.
The queue was working with mailables but not with notifications. After I added the column 'reserved' it's working perfectly.
Thank you very much for this and the laravel-ide-helper, it's great!
If I follow the instructions in the readme.md
and then try to run migrate:rollback/refresh/reset I get the following error message (formatted and redacted paths):
PHP Fatal error: Class 'CreateQueueTable' not found in $PROJECT_ROOT/vendor/laravel/framework/src/Illuminate/Database/Migrations/Migrator.php on line 297
{
"error": {
"type": "Symfony\\Component\\Debug\\Exception\\FatalErrorException",
"message": "Class 'CreateQueueTable' not found",
"file": "$PROJECT_ROOT/vendor\/laravel\/framework\/src\/Illuminate\/Database\/Migrations\/Migrator.php",
"line": 297
}
}
The usual composer dump-autoload
fix didn't help until I added the following line to autoload.classmap
in my composer.json
:
"vendor/barryvdh/laravel-async-queue/src/migrations"
Is this is the way I am supposed to do it or is there a better alternative. But either way, I think that this should be documented in in the readme.md
file.
Is there any plan to release version for Laravel 5.3?
Hey there,
Is there any plan to support 5.4?
Thanks!
Is laravel 5 support planned?
It is possible to limit the number of php processes that get spawned?
I'm using this package to copy huge amounts of data after an upgrade takes place of my web app, and I don't mind if it takes 1 hour or 24 hours or more. However if there are millions of rows of data that need to get copied, the queue spawns over 100 php processes and CPU usages sits at 100% until most of the queue jobs are down.
Using ->delay(30)
still spawns the php processes, but just delays them running, so the CPU still gets pegged.
Is there a way to keep the number of php process spawned to 5 or any number I choose? i.e. ->limitProc(5)
I've followed every instructions except for service provider, I use this instead:
'Barryvdh\Queue\AsyncServiceProvider'
But I get this exception:
InvalidArgumentException in QueueManager.php line 132:
No connector for [async]
Because I was unable to find anything related to this. How does this package deal with failed jobs? Is there a way to set an amount of tries before the job is delete/ignored/moved to a failed jobs table or anything like that?
Thank you :)
composer install command gives the following error with composer.json line:
"barryvdh/laravel-async-queue": "^0.6.0",
Error:
Installation request for barryvdh/laravel-async-queue ^0.6.0 -> satisfiable by barryvdh/laravel-async-queue[v0.6.0].
- Conclusion: don't install laravel/framework v5.5.2
- Conclusion: don't install laravel/framework v5.5.1
- barryvdh/laravel-async-queue v0.6.0 requires illuminate/support 5.4.x -> satisfiable by illuminate/support[v5.4.0, v5.4.13, v5.4.17, v5.4.19, v5.4.27, v5.4.36, v5.4.9].
Hi, we've been using your code successfully in a testing linux environment but we have to deploy it to a Windows server.
When we run the working code we get the following error.
We thought it might be a permissions issue so gave the IIS user write permission to Windows temp folder, but it didnt help
Any ideas?
We should probably warn people about exactly how this module works. We ran into some problems because the job itself is encoded with Base64 and then fired from the command line with the encoded job as a parameter. There's nothing wrong with this approach per se.
However, if too much data is passed, the command itself will exceed the maximum length of command line commands. On Windows, this might be as little as ~2000 chars. If this happens, everything will fail silently because Windows will simply fail to fire the command.
We have circumvented this by making sure that the closure (job) we fire take up as little code as possible and that data itself is not attached to this but only as few references to the data as possible. This makes sure that the Base64 encoded string is as short as possible.
When the job is fired and the command is run, we then use the references to fetch the necessary data in the class of the job itself, then runs the job. This makes sure that the new process fired by command line is the one that fetches the actual data and thus the data itself is not passed in the Base64 string, only references to the data.
I'm not sure if there's any way to handle this. Perhaps we should throw an exception on line 68 of AsyncQueue, if $command exceeds the maximum length of a command line command of the current OS. I'm not sure if that made any sense at all. :)
I used async queue. And, I want to run my background function only once. How can i do that ?
Here is my code. Is it right or wrong ? What should i fix ? And how can i know which job failed ?
$job = new myjob($var1, $var2);
$this->dispatch($job);
$job->release();
if ($job->attempts() >= 1)
{
$job->delete();
//will do other process when job fail
}
Can you update composer please?
Hello,
I have little problem, i don't understand why get this error in line 49 AsyncJoob.php. Somebody tell me about this error?
(Undefined property: Barryvdh\Queue\Jobs\AsyncJob::$deleted)
On using Queue::Push() jobs are inserted in the database with status 0 and not started.
On using something like : php artisan queue:async 5
It works perfectly.
Tested on Windows/Debian 7/OSX
DB::transaction(function(){
QueueHelper::pushAJobWithAsyncDriver();
sleep(5);
});
Here would raise an exception No query results for model [Barryvdh\\Queue\\Models\\Job].
Reason: The job excuted before transaction commit.
$item = Job::findOrFail($this->argument('job_id'))
//should add a lock
$item = Job::lock()->findOrFail($this->argument('job_id'))
Hi, Barry:
I appreciate this queue driver very much; it purports to be exactly what I need. I'm using it in a controller wherein I check if the exact zip/date combo of a request (movie showtimes) has been previously cached. If it has, I want to return the previously cached result, but still run the requested parameters again in the background to get updates. So, in this case ShowtimeController calls a queued job on itself.
The problem I'm having is it seems like execution of the (calling/original) controller is ending without error or stacktrace when the Queue::push line is executed. Also, the queue table has numerous jobs in it but they don't seem to be executing.
What am I missing?
Thank you,
~Todd
class ShowtimeController extends \BaseController {
function getShowtimes($zipcode,$date) {
$dateZipKey = $zipcode.'-'.$date;
if (Cache::has($dateZipKey)) {
Queue::push('ShowtimeController@showtimesJob', array('zipcode'=>$zipcode, 'date'=>$date));
// ** I can step through with a debugger to this point, then the execution just stops. **
$result = Cache::get($dateZipKey);
return $result;
} else {
$result = $this->loadShowtimes($zipcode,$date);
}
}
/* JOB for 'asynch' execution. */
private function showtimesJob($job, $data)
{
$this->loadShowtimes($data['zipcode'],$data['date']);
$job->delete();
}
/* Synchronous process for fetching and caching */
private function loadShowtimes($mZipcode, $mDate)
{
// get the updated zip/date-based showtimes from external source and Cache::put($dateZipKey)
}
}
InvalidArgumentException
No connector for [async]
Laravel 4.1
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.