CODEWITHSUNDEEP
puppet
Henrik
Henrik

Reputation: 1

Puppet silently fails to restart service

The Docker daemon sometimes crashes on our puppet-managed servers.

Puppet is expected to bring it back if this should happen, but fails to do so, and I think I know why. From the debug logs I can see puppet is doing this:

rc-service docker status
rc-service docker start

Doing the same manually on a crashed Docker shows:

rc-service docker status; echo $? 
* status: crashed
32

rc-service docker start; echo $?
 * WARNING: docker has already been started
0

Looks like it worked, but the service is still not running:

rc-service docker status; echo $? 
* status: crashed
32

What I'd like Puppet to do:

rc-service docker restart

The Puppet configuration looks like this:

service { ['docker']: 
  ensure => 'running',  
  enable => true,  
  hasrestart => true,
}

How to configure Puppet to use restart in this situation?

Upvotes: 0

Views: 722

Answers (1)

Matthew Schuchard
Matthew Schuchard

Reputation: 28774

You can specify the exact commands for start, stop, status, and restart with the corresponding attributes in the service resource: https://docs.puppet.com/puppet/latest/types/service.html#service-attributes. This is, however, going to become "hacky" since you are using a restart command for a start command, but it should technically not break anything.

service { 'docker':
  ensure => running,
  enable => true,
  start  => 'rc-service docker restart',
}

I recommend verifying your assumption that rc-service docker restart actually successfully restarts the docker service after it crashes before going down this route, otherwise this will not remedy your root cause.

Upvotes: 1

Related Questions