CODEWITHSUNDEEP
chef-infrabackup
user3062065
user3062065

Reputation: 1

chef-server-ctl backup not honoring --timeout

trying to set up automated backups of our chef server but it is timing out during the tar phase. I ran the tar manually and it seems to take over 20 minutes but the default timeout is 10 minutes. I tried using the --timeout and setting it at something high like 6000 but it still times out after 600 seconds.

output:

root@chef-server:/opt/opscode/embedded/lib/ruby/gems/2.1.0/gems/chef_backup-0.0.1# /usr/bin/chef-server-ctl backup --yes --timeout 60000                                                                                                                                   
Locating rsync..
/usr/bin/rsync
Starting Chef Server backup 
Bringing down the Chef Server
ok: down: bookshelf: 1s, normally up
ok: down: nginx: 1s, normally up
ok: down: oc_bifrost: 0s, normally up
ok: down: oc_id: 0s, normally up
ok: down: opscode-chef-mover: 1s, normally up
ok: down: opscode-erchef: 0s, normally up
ok: down: opscode-expander: 0s, normally up
ok: down: opscode-solr4: 0s, normally up
ok: down: rabbitmq: 0s, normally up
ok: down: redis_lb: 0s, normally up
Dumping Postgresql database to /tmp/chef_backup20170613-7603-1hsvyr5/chef_backup-2017-06-13-16-55-10.sql
could not change directory to "/tmp/d20170613-7603-xpkmet"
ok: down: postgresql: 0s, normally up
Writing backup manifest
Creating backup tarball
/opt/opscode/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.2.6/lib/mixlib/shellout/unix.rb:183: warning: conflicting chdir during another chdir block
tar: Removing leading `/' from member names
ERROR: Something wen't terribly wrong, aborting backup
ERROR: Command timed out after 600s:
Command exceeded allowed execution time, process terminated
---- Begin output of tar -czf /tmp/chef_backup20170613-7603-1hsvyr5/chef-backup-2017-06-13-16-55-10.tgz /var/opt/opscode/rabbitmq/db /var/opt/opscode/opscode-solr4/data /var/opt/opscode/redis_lb/data /var/opt/opscode/postgresql/9.2/data /var/opt/opscode/bookshelf/data /v
ar/opt/opscode/upgrades /etc/opscode /etc/opscode-manage /etc/opscode-reporting /etc/opscode-push-jobs-server /etc/opscode-analytics manifest.json chef_backup-2017-06-13-16-55-10.sql ----
STDOUT: 
STDERR: /opt/opscode/embedded/lib/ruby/gems/2.1.0/gems/mixlib-shellout-2.2.6/lib/mixlib/shellout/unix.rb:183: warning: conflicting chdir during another chdir block
tar: Removing leading `/' from member names
---- End output of tar -czf /tmp/chef_backup20170613-7603-1hsvyr5/chef-backup-2017-06-13-16-55-10.tgz /var/opt/opscode/rabbitmq/db /var/opt/opscode/opscode-solr4/data /var/opt/opscode/redis_lb/data /var/opt/opscode/postgresql/9.2/data /var/opt/opscode/bookshelf/data /var
/opt/opscode/upgrades /etc/opscode /etc/opscode-manage /etc/opscode-reporting /etc/opscode-push-jobs-server /etc/opscode-analytics manifest.json chef_backup-2017-06-13-16-55-10.sql ----
Ran tar -czf /tmp/chef_backup20170613-7603-1hsvyr5/chef-backup-2017-06-13-16-55-10.tgz /var/opt/opscode/rabbitmq/db /var/opt/opscode/opscode-solr4/data /var/opt/opscode/redis_lb/data /var/opt/opscode/postgresql/9.2/data /var/opt/opscode/bookshelf/data /var/opt/opscode/up
grades /etc/opscode /etc/opscode-manage /etc/opscode-reporting /etc/opscode-push-jobs-server /etc/opscode-analytics manifest.json chef_backup-2017-06-13-16-55-10.sql returned

any thoughts?

EDIT:

here is the code that I believe is creating the tar:

  def create_tarball
    log 'Creating backup tarball'
    cmd = [
      "tar -czf #{tmp_dir}/#{export_filename}",
      data_map.services.map { |_, v| v['data_dir'] }.compact.join(' '),
      data_map.configs.map { |_, v| v['data_dir'] }.compact.join(' '),
      Dir["#{tmp_dir}/*"].map { |f| File.basename(f) }.join(' ')
    ].join(' ').strip

    res = shell_out(cmd, cwd: tmp_dir)
    res
  end

from the /opt/opscode/embedded/lib/ruby/gems/2.1.0/gems/chef_backup-0.0.1/lib/chef_backup/strategy/backup/tar.rb file

Upvotes: 0

Views: 391

Answers (1)

user3062065
user3062065

Reputation: 1

looks like the shell_out function in the chef_backup/master/lib/chef_backup/helpers.rb file was outdated in 12.6.0 and didn't merge in the --timeout setting. I replaced the file with this: https://raw.githubusercontent.com/chef/chef_backup/master/lib/chef_backup/helpers.rb and was able to override the default timeout and allow the backups to finish.

thanks to the chef slack channel for helping me out with that!

Upvotes: 0

Related Questions