gave up: prod_* entered FATAL state, too many start retries too quickly

Question

I have a Supervisor-managed process (prod_00) running on an Ubuntu server. It stopped running on February 19 and wasn't restarted again until February 21, when I manually restarted it.

Issue Description:

From Supervisor logs (journalctl -u supervisor.service), I noticed that prod_00 was crashing repeatedly on Feb 19, and eventually, Supervisor gave up restarting it:

Feb 19 01:18:28,347 INFO exited: prod_00 (exit status 1; not expected)
Feb 19 01:18:29,745 INFO exited: prod_00 (exit status 1; not expected)
Feb 19 01:19:34,486 INFO exited: prod_00 (exit status 1; not expected)
Feb 19 01:19:44,228 INFO gave up: prod_00 entered FATAL state, too many start retries too quickly

There were no further restart attempts after that, until I manually restarted the service on Feb 21, after which it started running fine again.

Feb 21 04:27:02 INFO spawned: 'prod_00' with pid 2226996
Feb 21 04:27:03 INFO success: prod_00 entered RUNNING state

Current Configurations

[program:prod]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/prod/artisan queue:work database --sleep=3 --tries=3
autostart=true
autorestart=true
user=ubuntu
numprocs=8
redirect_stderr=true
stdout_logfile=/var/www/prod/storage/logs/worker.log
stopwaitsecs=3600

Questions

• Why did Supervisor stop trying to restart the process? (I assume it hit the retry limit, but what determines this behavior?)
• How can I configure Supervisor to retry failed processes indefinitely?
• What debugging steps should I follow to determine why the process was failing?

What I’ve Checked So Far

• Supervisor logs show repeated failures (exit status 1).
• Application logs don’t show any useful information, but I’m checking further.
• System resources (CPU, memory, and disk) seem fine.
• Dependencies (e.g., database, Redis) seem to be running.