Undesired parallel execution of Laravel IronMq Queues

Question

I am trying to set an IronMq queue with Laravel, and I have it working already, but the point is that the behavior is not the desired one.

I expect IronMq to wait until a job is complete ($job->delete()) to push a new one, but I found out that it pushes messages before the previous one is finished.

The code is structured as follows:

Route::post('queue/send' ,function() 
{
    ...
    Queue::push(function($job) use ($data)
    {
        ...
        $job->delete();
    }
    return true;
}

Has anyone found out the way to prevent the parallel behavior and make it sequential?

Thank you very much!

score 2 · Answer 1 · answered May 06 '14 at 17:11

2

Push queues will naturally continue to push messages regardless of whether or not the job is done as they are 2 independent systems. Secondly, the purpose of utilizing a push queue is for it to be as realtime as possible.

IronMQ specifically has retries which, as it sounds, will retry pushing the message to the desired endpoint X number of times(which you set) in designated intervals (which you set) - hopefully this will help solve your problems.

If you desire that a job only be processed once the previous has been completed, and a slight delay is acceptable, then I'd recommend polling the queue and retrieving in batches.

answered May 06 '14 at 17:11

Yaron

610
6
13

I guess that polling the queue or using Iron Worker is the way to go, but I can't understand how a queue system does not have the option of having a single process at a time.... The retry is not a solution as the process is well treated, but have some concurrency issues. Thank you for your answer! – Juanjo Lainez Reche May 07 '14 at 08:26
Unlike delayed job or an in-app queuing system, a push queue has no knowledge of when the job is done on the server side. The two are independent systems. – Yaron May 07 '14 at 15:25

score 1 · Answer 2 · answered May 07 '14 at 02:35

1

Also you could try to use IronMQ class instead of laravel Queue class:

$ironmq = new \IronMQ(array(
    'token' => Config::get('queue.connections.iron.token', 'xxx'),
    'project_id' => Config::get('queue.connections.iron.project', 'xxx')
));
$ironmq->postMessage($queue_name, "Hello world");

answered May 07 '14 at 02:35

Dmitriy

399
2
6

I don't see the difference between Laravel and IronMq queues as I set the driver to IronMq in the config/queue.php file. How does it solves the concurrency problem? – Juanjo Lainez Reche May 07 '14 at 08:18

score 0 · Accepted Answer · answered May 20 '14 at 10:37

0

Thank you for your answers, I decided to use Beanstalkd instead of IronMQ.

It's much messier, but it provides with the desired functionality and I am not depending on no one.

answered May 20 '14 at 10:37

Juanjo Lainez Reche

886
9
20

Undesired parallel execution of Laravel IronMq Queues

3 Answers3