Jan
Jan

Reputation: 582

Laravel 8: QueueManager.php:156 array offset of type null / Configure redis

I am about to setup Redis and Laravel Horizon. Although I have set up everything correctly (I assume) I get a very weird error. Laravel tells me that it is Trying to access array offset on value of type null at vendor/laravel/framework/src/Illuminate/Queue/QueueManager.php:156

The line 156 is the first return line:

/**
* Resolve a queue connection.
*
* @param  string  $name
* @return \Illuminate\Contracts\Queue\Queue
*/
protected function resolve($name)
{
    $config = $this->getConfig($name);

    return $this->getConnector($config['driver']) // <-- This line
                    ->connect($config)
                    ->setConnectionName($name);
}

So, I assume Laravel cannot access $config['driver']. But when I do a dd of $config I get the following:

array:5 [
  "driver" => "redis"
  "connection" => "default"
  "queue" => "default"
  "retry_after" => 90
  "block_for" => null
]

So, it is impossible that driver is empty because as you can see driver is set to redis. Do you guys have any idea why Laravel cannot access array offset on value of type null?

I have installed phpredis, a Redis database and also Laravel Horizon. When I try to access my /horizon I get the same error as above.

phpredis is installed correctly. I have added extension=redis.so to my php.ini and when I execute php -r "if (new Redis() == true){ echo \"\r\n OK \r\n\"; }" in my command line I also get OK. So, it cannot be phpredis.

My Redis database is also working:

$ redis-cli
127.0.0.1:6379> ping
PONG
127.0.0.1:6379>

And my configs are also correct. This is my .env (just some entries):

APP_ENV=local
APP_DEBUG=true

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=laravel
DB_USERNAME=root
DB_PASSWORD=password

BROADCAST_DRIVER=log
CACHE_DRIVER=file
QUEUE_CONNECTION=redis
SESSION_DRIVER=file
SESSION_LIFETIME=120

REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=6379

Here is my config/queue.php:

<?php

return [

    /*
    |--------------------------------------------------------------------------
    | Default Queue Connection Name
    |--------------------------------------------------------------------------
    |
    | Laravel's queue API supports an assortment of back-ends via a single
    | API, giving you convenient access to each back-end using the same
    | syntax for every one. Here you may define a default connection.
    |
    */

    'default' => env('QUEUE_CONNECTION', 'sync'),

    /*
    |--------------------------------------------------------------------------
    | Queue Connections
    |--------------------------------------------------------------------------
    |
    | Here you may configure the connection information for each server that
    | is used by your application. A default configuration has been added
    | for each back-end shipped with Laravel. You are free to add more.
    |
    | Drivers: "sync", "database", "beanstalkd", "sqs", "redis", "null"
    |
    */

    'connections' => [

        'sync' => [
            'driver' => 'sync',
        ],

        'database' => [
            'driver' => 'database',
            'table' => 'jobs',
            'queue' => 'default',
            'retry_after' => 90,
        ],

        'beanstalkd' => [
            'driver' => 'beanstalkd',
            'host' => 'localhost',
            'queue' => 'default',
            'retry_after' => 90,
            'block_for' => 0,
        ],

        'sqs' => [
            'driver' => 'sqs',
            'key' => env('AWS_ACCESS_KEY_ID'),
            'secret' => env('AWS_SECRET_ACCESS_KEY'),
            'prefix' => env('SQS_PREFIX', 'https://sqs.us-east-1.amazonaws.com/your-account-id'),
            'queue' => env('SQS_QUEUE', 'your-queue-name'),
            'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
        ],

        'redis' => [
            'driver' => 'redis',
            'connection' => 'default',
            'queue' => env('REDIS_QUEUE', 'default'),
            'retry_after' => 90,
            'block_for' => null,
        ],

    ],

    /*
    |--------------------------------------------------------------------------
    | Failed Queue Jobs
    |--------------------------------------------------------------------------
    |
    | These options configure the behavior of failed queue job logging so you
    | can control which database and table are used to store the jobs that
    | have failed. You may change them to any database / table you wish.
    |
    */

    'failed' => [
        'driver' => env('QUEUE_FAILED_DRIVER', 'database'),
        'database' => env('DB_CONNECTION', 'mysql'),
        'table' => 'failed_jobs',
    ],

];

My config/horizon.php

<?php

use Illuminate\Support\Str;

return [

    /*
    |--------------------------------------------------------------------------
    | Horizon Domain
    |--------------------------------------------------------------------------
    |
    | This is the subdomain where Horizon will be accessible from. If this
    | setting is null, Horizon will reside under the same domain as the
    | application. Otherwise, this value will serve as the subdomain.
    |
    */

    'domain' => env('HORIZEN_DOMAIN', null),

    /*
    |--------------------------------------------------------------------------
    | Horizon Path
    |--------------------------------------------------------------------------
    |
    | This is the URI path where Horizon will be accessible from. Feel free
    | to change this path to anything you like. Note that the URI will not
    | affect the paths of its internal API that aren't exposed to users.
    |
    */

    'path' => env('HORIZEN_PATH', 'horizon'),

    /*
    |--------------------------------------------------------------------------
    | Horizon Redis Connection
    |--------------------------------------------------------------------------
    |
    | This is the name of the Redis connection where Horizon will store the
    | meta information required for it to function. It includes the list
    | of supervisors, failed jobs, job metrics, and other information.
    |
    */

    'use' => 'default',

    /*
    |--------------------------------------------------------------------------
    | Horizon Redis Prefix
    |--------------------------------------------------------------------------
    |
    | This prefix will be used when storing all Horizon data in Redis. You
    | may modify the prefix when you are running multiple installations
    | of Horizon on the same server so that they don't have problems.
    |
    */

    'prefix' => env(
        'HORIZON_PREFIX',
        Str::slug(env('APP_NAME', 'laravel'), '_').'_horizon:'
    ),

    /*
    |--------------------------------------------------------------------------
    | Horizon Route Middleware
    |--------------------------------------------------------------------------
    |
    | These middleware will get attached onto each Horizon route, giving you
    | the chance to add your own middleware to this list or change any of
    | the existing middleware. Or, you can simply stick with this list.
    |
    */

    'middleware' => ['web'],

    /*
    |--------------------------------------------------------------------------
    | Queue Wait Time Thresholds
    |--------------------------------------------------------------------------
    |
    | This option allows you to configure when the LongWaitDetected event
    | will be fired. Every connection / queue combination may have its
    | own, unique threshold (in seconds) before this event is fired.
    |
    */

    'waits' => [
        'redis:default' => 60,
    ],

    /*
    |--------------------------------------------------------------------------
    | Job Trimming Times
    |--------------------------------------------------------------------------
    |
    | Here you can configure for how long (in minutes) you desire Horizon to
    | persist the recent and failed jobs. Typically, recent jobs are kept
    | for one hour while all failed jobs are stored for an entire week.
    |
    */

    'trim' => [
        'recent' => 60,
        'pending' => 60,
        'completed' => 60,
        'recent_failed' => 10080,
        'failed' => 10080,
        'monitored' => 10080,
    ],

    /*
    |--------------------------------------------------------------------------
    | Metrics
    |--------------------------------------------------------------------------
    |
    | Here you can configure how many snapshots should be kept to display in
    | the metrics graph. This will get used in combination with Horizon's
    | `horizon:snapshot` schedule to define how long to retain metrics.
    |
    */

    'metrics' => [
        'trim_snapshots' => [
            'job' => 24,
            'queue' => 24,
        ],
    ],

    /*
    |--------------------------------------------------------------------------
    | Fast Termination
    |--------------------------------------------------------------------------
    |
    | When this option is enabled, Horizon's "terminate" command will not
    | wait on all of the workers to terminate unless the --wait option
    | is provided. Fast termination can shorten deployment delay by
    | allowing a new instance of Horizon to start while the last
    | instance will continue to terminate each of its workers.
    |
    */

    'fast_termination' => false,

    /*
    |--------------------------------------------------------------------------
    | Memory Limit (MB)
    |--------------------------------------------------------------------------
    |
    | This value describes the maximum amount of memory the Horizon master
    | supervisor may consume before it is terminated and restarted. For
    | configuring these limits on your workers, see the next section.
    |
    */

    'memory_limit' => 64,

    /*
    |--------------------------------------------------------------------------
    | Queue Worker Configuration
    |--------------------------------------------------------------------------
    |
    | Here you may define the queue worker settings used by your application
    | in all environments. These supervisors and settings handle all your
    | queued jobs and will be provisioned by Horizon during deployment.
    |
    */

    'defaults' => [
        'supervisor-1' => [
            'connection' => 'redis',
            'queue' => ['default'],
            'balance' => 'auto',
            'maxProcesses' => 1,
            'memory' => 128,
            'tries' => 1,
            'nice' => 0,
        ],
    ],

    'environments' => [
        'production' => [
            'supervisor-1' => [
                'connection' => 'redis',
                'queue' => ['default'],
                'balance' => 'simple',
                'processes' => 10,
                'tries' => 2,
                'timeout' => 60 * 60,
            ],
            'mailcoach-general' => [
                'connection' => 'mailcoach-redis',
                'queue' => ['mailcoach', 'mailcoach-feedback', 'send-mail'],
                'balance' => 'auto',
                'processes' => 10,
                'tries' => 2,
                'timeout' => 60 * 60,
            ],
            'mailcoach-heavy' => [
                'connection' => 'mailcoach-redis',
                'queue' => ['send-campaign'],
                'balance' => 'auto',
                'processes' => 3,
                'tries' => 1,
                'timeout' => 60 * 60,
            ],
        ],

        'local' => [
            'supervisor-1' => [
                'connection' => 'redis',
                'queue' => ['default'],
                'balance' => 'simple',
                'processes' => 10,
                'tries' => 2,
                'timeout' => 60 * 60,
            ],
            'mailcoach-general' => [
                'connection' => 'mailcoach-redis',
                'queue' => ['mailcoach', 'mailcoach-feedback', 'send-mail'],
                'balance' => 'auto',
                'processes' => 10,
                'tries' => 2,
                'timeout' => 60 * 60,
            ],
            'mailcoach-heavy' => [
                'connection' => 'mailcoach-redis',
                'queue' => ['send-campaign'],
                'balance' => 'auto',
                'processes' => 3,
                'tries' => 1,
                'timeout' => 60 * 60,
            ],
        ],
    ],
];

Anyone has an idea why I get this error? I am working on php 7.4.16, 10.5.6-MariaDB, redis-cli 6.2.1 and on Laravel Framework 8.32.1. So, everything is on the newest version.

I would appreciate any kind of help! Kind regards

Upvotes: 2

Views: 3289

Answers (1)

Ray
Ray

Reputation: 186

  • Edit: I just thought of something... your Redis setup is functioning correctly right? I mean when you do a $redisclient->setex('test', 123, 'value'); it shows up in your redis-cli right when you do keys test?

Ok let's go through how I would set this up and troubleshoot this. Your error I believe is in the "final stages" of the whole process, let's first check if the queues are running ok etc. So I'm using Docker Containers but that shouldn't matter. I think Horizon is really good, especially compared to using the default "queue:work", due to the dashboard and monitoring you've got access to.

As I said, I think it is a good idea to use the supervisor service to run the queue workers, also as mentioned in the Laravel docs:

To keep the queue:work process running permanently in the background, you should use a process monitor such as Supervisor to ensure that the queue worker does not stop running.

The reason why is really that it auto-restarts your queues etc in case of errors. So below I went through a bit, possibly excessive supervisor stuff...

So the moving parts are:

  • Supervisor > queue worker proces(ses): Horizon
  • Redis
  • Laravel app

As I said I'm running Supervisor in my Docker PHP-FPM container which in it's turn runs cron, Horizon and PHP-FPM but in your case you can just install Supervisor with apt-get like so https://laravel.com/docs/8.x/horizon#installing-supervisor and just let it run horizon. Doing it this way I can also point you out to the log file which will be created which might tell us some more details :) Make sure after installing supervisor and adding the example-hotizon.conf config file to your conf.d directory you do a supervisor reread (rereads the config files) and then a supervisor update process config changes.

====== START SUPERVISOR ======

My Horizon Supervisor config:

[program:example-horizon]
process_name=%(program_name)s
command=php /var/www/example.com/artisan horizon
autostart=true
autorestart=true
user=www-data
redirect_stderr=true
stdout_logfile=/var/www/example.com/storage/logs/horizon.log
# Kill the job after "stopwaitsecs" secs, make sure this is greater than the longest running job
stopwaitsecs=600

Then I think you can just do supervisord -c /etc/supervisor/supervisord.conf With my supervisord.conf being like so, I have taken out my Cron and PHP-FPM configs...:

; http://supervisord.org/configuration.html

[unix_http_server]
file=/var/run/supervisor.sock               ; (the path to the socket file)
chmod=0700                                  ; sockef file mode (default 0700)

[supervisord]
nodaemon=true                               ; This keeps the Docker container alive (in the foreground)
user=root                                   ; default user
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/var/run/supervisord.pid            ; (supervisord pidfile;default supervisord.pid) 
childlogdir=/var/log/supervisor             ; ('AUTO' child log dir, default $TEMP)

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface 

[supervisorctl]
serverurl=unix:///var/run/supervisor.sock   ; use a unix:// URL  for a unix socket 

; The [include] section can just contain the "files" setting.  This
; setting can list multiple files (separated by whitespace or
; newlines).  It can also contain wildcards.  The filenames are
; interpreted as relative to this file.  Included files *cannot*
; include files themselves.
[include]
files = /etc/supervisor/conf.d/*.conf

I assume you have also already published Horizon right? php artisan horizon:publish after you did php artisan horizon:install I believe to use the dashboard functionality.

Let's test what this will output by keeping an eye no:

  • our supervisor console
  • Our log file /var/www/example.com/storage/logs/horizon.log (set in horizon.conf)

So let's start with our supervisor console. run supervisorctl from your command prompt, you'll see your running processes strait away, hoping that your example-horizon process is actually "RUNNING" and not "STOPPED" or simmilar. You can also check the log by, from supervisorctl, running tail <process_name> so tail example-horizon and check that output. supervisor log example In my case of supervisorctl status I have:

# supervisorctl status
cron                             RUNNING   pid 8, uptime 3 days, 22:09:07
example-horizon                  RUNNING   pid 10, uptime 3 days, 22:09:07
php-fpm                          RUNNING   pid 9, uptime 3 days, 22:09:07

====== END SUPERVISOR ======

====== START REDIS ======

Then when the supervisor part is working I assume your Redis config is fine in your .env but what you can do is check your Redis if it has the Horizon queue keys in it: Redis output Mine are called "horizon_LW1_xxx" because I have used "horizon_LW1" as a prefix for horizon in the Horizon config config/horizon.php, mine looks like:

'prefix' => env(
        'HORIZON_PREFIX',
        Str::slug(env('APP_NAME', 'laravel'), '_').'_horizon:'
    ),

In case you don't have this in your Redis we should fix that first...

====== END REDIS ======

====== config/horizon.php ====== Just my local horizon config depending on my env "HOZIRON_QUEUE_LOCAL" which are my different queues: HOZIRON_QUEUE_LOCAL=prio1,prio2,email,default

'local' => [
            'supervisor-it' => [
                'queue' => explode(',', env('HOZIRON_QUEUE_LOCAL', 'default')),
                'maxProcesses' => 23,
                'memory' => 100,
            ],
        ],

Let's just fire a job into the queue, monitoring your supervisor "tail example-horizon" (storage/logs/horizon.log), your Laravel log (storage/logs/laravel.log) and your Redis queue.

Sorry this became a bit of a novel, but basically:

  1. install Supervisor
  2. run Horizon queue via Supervisor
  3. check Redis keys
  4. check logs (supervisor, laravel)

Let me know how the above goes as I believe they are the "roots" of a good Horizon setup. Then we can move on to the next step in case that does not work out successful yet.

Upvotes: 2

Related Questions