Reputation: 151

Chef 502 Bad Gateway

The topic might look repeated but I haven't been able to find correct or consistent answer on how to solve the problem. Chef server was working a while back but now its being giving this Nginx gateway error.

knife commands and chef server UI both are not working.

root@ubuntu:/opt/chef-server/embedded/sbin# knife role list
[fog][WARNING] Unable to load the 'unf' gem. Your AWS strings may not be properly encoded.
ERROR: bad gateway
Response: <html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.2.3</center>
</body>
</html>

hosts file:

root@ubuntu:/opt/chef-server/embedded/sbin# vi /etc/hosts
root@ubuntu:/opt/chef-server/embedded/sbin# cat /etc/hosts
127.0.0.1       localhost
10.44.189.9     ubuntu.xxxx.co.in  ubuntu

Interfaces file:

root@ubuntu:/opt/chef-server/embedded/sbin# cat /etc/network/interfaces
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 10.44.189.9
netmask 255.255.254.0
gateway 10.44.188.1

Output of chef-server-ctl status:

root@ubuntu:/opt/chef-server/embedded/sbin# chef-server-ctl status
run: bookshelf: (pid 1287) 91369s; run: log: (pid 1279) 91369s
run: chef-expander: (pid 1293) 91369s; run: log: (pid 1281) 91369s
run: chef-server-webui: (pid 1283) 91369s; run: log: (pid 1278) 91369s
run: chef-solr: (pid 1294) 91369s; run: log: (pid 1290) 91369s
run: erchef: (pid 6950) 0s; run: log: (pid 1282) 91369s
run: nginx: (pid 27070) 1912s; run: log: (pid 1857) 2542s
run: postgresql: (pid 28153) 34153s; run: log: (pid 1289) 91369s
down: rabbitmq: 1s, normally up, want up; run: log: (pid 1288) 91369s
root@ubuntu:/opt/chef-server/embedded/sbin#`

Output of chef-server-ctl test:

root@ubuntu:/opt/chef-server/embedded/sbin# chef-server-ctl test
Configuring logging...
Creating client pedant_admin_client...
Encountered an error attempting to create client pedant_admin_client
Response Code was: 502
Response Body was: <html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.2.3</center>
</body>
</html>
..
Exception during Pedant credentials setup
RuntimeError
..
Finished in 0.05734 seconds
0 examples, 0 failures

Upvotes: 1

Answers (3)

jorfus

Reputation: 3088

I ran into this too.

tl;dr

sudo chef-server-ctl upgrade

WARNING: My org and users etc. were preserved, but your results may vary.

I suspect what happened was that the chef-server-core package got updated. (a careless apt-get upgrade could do it, I noticed it after a crash and reboot.) I shouldn't use the passive voice, but I certainly don't remember doing that.

My troubleshooting steps for posterity.

sudo chef-server-ctl stop
sudo chef-server-ctl start
sudo chef-server-ctl tail

I see this repeatedly

==> /var/log/opscode/rabbitmq/current <==
2015-08-23_23:51:22 mkdir: cannot create directory '/opt/opscode/embedded/service/rabbitmq/sbin/../var': Permission denied

Which is weird. That ".." in the path is a clue pointing to a config change between software versions. Maybe the chef deb package got upgraded by mistake and configuration needs to be updated to match (recall chef server is like a dozen services working together)