shiftonelabs / laravel-sqs-fifo-queue Goto Github PK

When Laravel process queue, not resolve class from SQS message.

The SQS message is:
JSON { "uuid":"1234", "displayName":"offer_status_changed", "job":"offer_status_changed", ... "data":{ "id":71, ... } }

The job name not include class namespace and Laravel throw errors:

• Target class [offer_status_changed] does not exist
• [object] (ReflectionException(code: -1): Class \"offer_status_changed\" does not exist

I understand that job should include namespace with method "job":"App\\Jobs\\offer_status_changed@handle",

It seems that the app isn't binding in at the right time maybe? I added
$app->register(ShiftOneLabs\LaravelSqsFifoQueue\LaravelSqsFifoQueueServiceProvider::class);
to bootstrap/app.php ... any ideas?

I have an event with one of its listeners implementing ShouldQueue with connection set to sqs-fifo like this:

class MyEventListener implements ShouldQueue
{
    public $connection = 'sqs-fifo';

    public function handle($event)
    {
        logger()->debug('queued event listener handled');
    }
}

This works fine. However, I don't seem to be able to specify the Message Group ID. I have tried the following:

1. use SqsFifoQueueable;-trait and call $this->onMessageGroup('custom-group'); in the listener __construct()
2. Specify $messageGroupId property manually: public $messageGroupId = 'custom-group';

Both messages sent to my SQS FIFO queue have default Message Group ID, not custom-group as I had hoped.

Am I missing something? Or is using Queued Event Listeners simply not supported (yet)?

Please provide support for Laravel 7.x.

Hello, notice this issue when updated to latest AWS SDK version

Environment

Laravel 9
PHP 8.2
AWS SDK: 3.273.0

AWS config example

        'sqs-fifo' => [
            'driver' => 'sqs-fifo',
            'key' => env('AWS_ACCESS_KEY'),
            'secret' => env('AWS_SECRET_KEY'),
            'token' => env('AWS_SESSION_TOKEN'),
            'prefix' => env('SQS_PREFIX'),
            'suffix' => null,
            'queue' => 'second_lane.fifo',
            'region' => 'us-west-2',
            'group' => 'default',
            'deduplicator' => 'unique',
            'allow_delay' => null,
        ],

Code

return new SqsFifoQueue(
            new SqsClient(
                Arr::except($config, ['token'])
            ),
            $config['queue'],
            $config['prefix'] ?? '',
            $config['suffix'] ?? '',
            (bool)($config['after_commit'] ?? null),
            $group,
            $deduplicator,
            $allowDelay
        );

Problem

AWS SDK require Token to be provided.
When no Token provided to AwsClient it uses \Aws\Token\TokenProvider::defaultProvider.
It can look for token using some default methods but in some case it could find token somehow as string and fail with error:

Invalid configuration value provided for "token". Expected Aws\Token\TokenInterface|Aws\CacheInterface|array|bool|callable, but got string(488) "...." token: (Aws\Token\TokenInterface|Aws\CacheInterface|array|bool|callable) Specifies the token used to authorize requests. Provide an Aws\Token\TokenInterface object, an associative array of "token", and an optional "expiration" key, false to use a null token, or a callable token provider used to fetch a token or return null. See Aws\Token\TokenProvider for a list of built-in credentials providers. If no token is provided, the SDK will attempt to load one from the environment.

Because there no way for me to find from where it's coming there also no way to set it manually using config and fix the problem using codebase.

Possible Solution

Make token setable form the config:
When token present on top level config use it not only for token field in credentials but also for token array itself

Laravel version: 9.44.0
PHP version: 8.1

Declaration of ShiftOneLabs\LaravelSqsFifoQueue\Support\Str::substr($string, $start, $length = null) must be compatible with Illuminate\Support\Str::substr($string, $start, $length = null, $encoding = 'UTF-8')

It looks like this is a major issue, because it breaks queue workers.

I think extending Illuminate\Support\Str is no more necessary.

Thank you

Declaration of ShiftOneLabs\LaravelSqsFifoQueue\Support\Str::substr($string, $start, $length = null) must be compatible with Illuminate\Support\Str::substr($string, $start, $length = null, $encoding = 'UTF-8') {"userId":1,"exception":"[object] (Symfony\Component\ErrorHandler\Error\FatalError(code: 0): Declaration of ShiftOneLabs\LaravelSqsFifoQueue\Support\Str::substr($string, $start, $length = null) must be compatible with Illuminate\Support\Str::substr($string, $start, $length = null, $encoding = 'UTF-8') at /Users/drew/product/vendor/shiftonelabs/laravel-sqs-fifo-queue/src/Support/Str.php:47)

I was able to fix the error by adding encode as a param to this function in the str.php file

/**
     * Returns the portion of string specified by the start and length parameters.
     *
     * The substr method wasn't added to the Str class until Laravel 5.1.
     * Add the implementation here to support older versions of Laravel.
     *
     * @param  string  $string
     * @param  int  $start
     * @param  int|null  $length
     *
     * @return string
     */
    public static function substr($string, $start, $length = null, $encoding = 'UTF-8')
    {
        return mb_substr($string, $start, $length, $encoding);
    }

That being said, I know it's not a real fix. I'm a fairly new developer, can someone help me figure out what this solution is?

Laravel keeps sending empty messages continuously.
Even bug reported in: laravel-heroku-sqs-worker-sending-empty-messages-continously-error-on-countable

OS: Linux Mint Sylvia
Framework: Laravel 5.6.7

Hello, By any chance, did anyone encounter this error below when trying to send a message to AWS SQS Queue?

Environment

• Laravel 9
• PHP 8

Code

• From Controller:
  • CreateSqsTaskJob::dispatch()->onConnection('sqs-fifo');
• Code for CreateSqsTaskJob class

class CreateSqsTaskJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    // protected $data;

    public function __construct()
    {
        //
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        //
    }
}

• queue.php connection

'sqs-fifo' => [
            'driver' => 'sqs-fifo',
            'key' => env('AWS_ACCESS_KEY_ID'),
            'secret' => env('AWS_SECRET_ACCESS_KEY'),
            'queue' => env('SQS_QUEUE'),
            'region' => env('AWS_DEFAULT_REGION'),
            'group' => 'default',
            'deduplicator' => 'unique',
            'queue_url' => env('SQS_QUEUE_URL'),
        ],

Error

ShiftOneLabs\LaravelSqsFifoQueue\Support\Str::substr()' is not compatible with method 'Illuminate\Support\Str::substr()

I'm not sure if this is a question specific to your Laravel implementation (great work btw) or whether its a SQS FIFO specific thing, but if I create a job with a delay as per the Laravel docs:

ProcessPodcast::dispatch($podcast)
    ->delay(now()->addMinutes(10));

Does that means other jobs (without a delay) sent to the queue will be processed before this one, even though they arrived after?

I guess the question could be rephrased to: can I control the order of FIFO items, by setting the delay?

I want to trigger a pushRaw, for that I'm using the following:
Queue::connection('sqs-fifo')->pushRaw($myData);

however, the following error is reported:

In LaravelSqsFifoQueueServiceProvider.php line 71: Call to undefined method Illuminate \ Support \ Facades \ Queue :: extend ()

I'm use Laravel 5.6

Hi, I'm trying to integrate this packege with laravel 4.2. The problem I'm having is that laravel 4.2 stores the jobs class under the "job" key and from that a new instance is created and so on.

In my case I need different message groups for each job, so I am giving an instance of the job to the queue and setting $messageGroupId for it, but doing so under the "job" the name of the class in not sent.

Expected message sent to sqs fifo
{"job":"App\\Models\\Jobs\\AJob","data":[],"group":"group-1"}
Actual message is sent to sqs fifo
{"job":{"messageGroupId":"events-1","deduplicator":null},"data":[],"group":"group-1"}

Thank you for your great project. Laravel9 was released February 8th, 2022. So, we plan to use Laravel 9 as a web application in our project. And I have a question. Does this oss have plans to test Larave9? I want to know a rough roadmap on it for me and other developers waiting for that.

Also, I will share the information on our local testing result with Laravel9 if I get it.

Hello and thank you for your efforts on this package! It's much appreciated.

I'm working in Laravel 8 on Vapor and am having some trouble getting the queued notifications to use different message group ids. I'm queueing notifications using the following as an example:

(new ReadyNotification($assignment))
    ->onConnection('fifo')
    ->onMessageGroup($assignment->getUuid())

My queue configuration looks like this:

'fifo' => [
    'driver' => 'sqs-fifo',
    'key' => env('SQS_KEY'),
    'secret' => env('SQS_SECRET'),
    'prefix' => env('SQS_PREFIX'),
    'suffix' => env('SQS_SUFFIX', '-develop'),
    'queue' => 'fifo.fifo',
    'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
    'group' => 'fifo',
    'deduplicator' => 'sqs',
    'allow_delay' => env('SQS_ALLOW_DELAY'),
],

As I've inspected jobs on the queue occasionally one will be on the {assignmentUuid} group but they are typically falling onto the default group as specified by the config above.

I've attempted to set the message group in the constructor of the notification but it results in the same inconsistent behavior.

What advice might you have as I continue to debug this?

Again, thank you for your time!

After upgrading from Laravel 8 to Laravel 9 we upgraded to your latest version but now all the places where we used an integer as messageGroupId are failing.

In QueryCompatibleInputMiddleware.php line 165:

  The provided type for `MessageGroupId` value was `integer`. The modeled type is `string`.

Should public function onMessageGroup($messageGroupId) only accept strings (or cast inputs into string)?

Thank you