CODEWITHSUNDEEP
phplaravelredislaravel-queue
TheWebs
TheWebs

Reputation: 12923

Using a different redis connection to process jobs in Laravel

I have a system where I spin up, every hour, tons of jobs and I would like them to be handled on a different redis connection, separate from the main one jobs are popped on to so they do not interfere with other jobs (ie, delaying them) that get pushed on to the queue.

So I did the following, I created a new redis connection:

    'kingdom_jobs' => [
        'url' => env('REDIS_URL'),
        'host' => env('REDIS_HOST', '127.0.0.1'),
        'password' => env('REDIS_PASSWORD', null),
        'port' => env('REDIS_PORT', 6379),
        'database' => env('REDIS_CACHE_DB', 2),
    ],

Then created a new Queue Connection:

    'kingdom_jobs' => [
        'driver' => 'redis',
        'connection' => 'kingdom_jobs', // Uses the new connection
        'queue' => env('REDIS_QUEUE', 'default'),
        'retry_after' => 90,
        'block_for' => null,
    ],

Then in the loop that spins up all these jobs:

    Kingdom::chunkById(250, function($kingdoms) use ($service) {
        foreach ($kingdoms as $kingdom) {
            UpdateKingdomJob::dispatch($kingdom)->onConnection('kingdom_jobs');
        }
    });

The issue?

Ya they get pushed to this new connection, I see them in horizon - over 500 of them - but they don't do anything. They don't get processed. Is there a way to tell horizon to process this additional connection?

I currently use php artisan horizon in development to process jobs - is there another step?

I assume from the docs, there is some step I am missing, I assume it has to do with php artisan queue:work --tries = 3 from the docs? Trying that did nothing. Do I need another instance of horizon for this new connection or do I configure horizon to know about this connection?

Help?

Upvotes: 1

Views: 2360

Answers (1)

James
James

Reputation: 16339

It is likely that you don't have a Horizon supervisor setup to watch your new queue.

Looking at the Horizon config:

'environments' => [
    'production' => [
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['default'],
            'balance' => 'auto',
            'minProcesses' => 1,
            'maxProcesses' => 10,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
            'tries' => 3,
        ],
    ],
],

It looks like we can just add another supervisor to the production environment (and others if you need it):

'environments' => [
    'production' => [
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['default'],
            'balance' => 'auto',
            'minProcesses' => 1,
            'maxProcesses' => 10,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
            'tries' => 3,
        ],
        'supervisor-2' => [
            'connection' => 'kingdom_jobs',
            'queue' => ['default'],
            'balance' => 'auto',
            'minProcesses' => 1,
            'maxProcesses' => 10,
            'balanceMaxShift' => 1,
            'balanceCooldown' => 3,
            'tries' => 3,
        ],
    ],
],

Upvotes: 1

Related Questions