CODEWITHSUNDEEP
socketsunixnginxgunicorn
Seán Hayes
Seán Hayes

Reputation: 4360

Gunicorn socket disappears

Distributor ID: Ubuntu
Description:    Ubuntu 12.04.4 LTS
Release:    12.04
Codename:   precise

gunicorn (version 19.1.1)
nginx version: nginx/1.1.19

My gunicorn conf:

bind = ["unix:///tmp/someproj1.sock", "unix:///tmp/someproj2.sock"]
pythonpath = "/home/deploy/someproj/someproj"
workers = 5
worker_class = "eventlet"
worker_connections = 25
timeout = 3600
graceful_timeout = 3600

We started getting 502s at around 2PM yesterday in our dev env. This was in the Nginx error log:

connect() to unix:///tmp/someproj1.sock failed (2: No such file or directory) while connecting to upstream"

Both gunicorn sockets were missing from /tmp.

At 11:55AM today I ran ps -eo pid,cmd,etime|grep gunicorn to get the uptime:

4156 gunicorn: master [myproj.    22:53:54
4161 gunicorn: worker [myproj.    22:53:54
4162 gunicorn: worker [myproj.    22:53:54
4163 gunicorn: worker [myproj.    22:53:54
4164 gunicorn: worker [myproj.    22:53:54
4165 gunicorn: worker [myproj.    22:53:53
5207 grep --color=auto gunicorn        00:00

So gunicorn and all its workers have been running uninterrupted since ~1:01PM yesterday. The Nginx access log confirm that requests were successfully being served for about an hour after gunicorn was started. Then it seems for some reason both gunicorn sockets disappeared, and gunicorn continued running without writing any error logs.

Any ideas on what could cause that? Or how to fix it?

Upvotes: 2

Views: 770

Answers (1)

Gx1sptDTDa
Gx1sptDTDa

Reputation: 1588

It turns out this was indeed a bug, where eventlet workers would delete the socket when they themselves would restart.

The fix has already been merged into master branch, but it has unfortunately not been released yet (version 19.3 still has the problem).

Upvotes: 2

Related Questions