stochastic
stochastic

Reputation: 3413

debugging with ansible: How to get stderr and stdout from failing commands to be printed respecting newlines so as to be human readable?

If I run a simple ansible playbook, I often get difficult-to-read output from failing tasks, like that below. Big problems:

How can I get ansible to print its output in a format that I can read easily, so I can debug?

Here is the yucky output:

$ ansible-playbook playbooks/backUpWebsite.yml

PLAY [localhost] 
***************************************************************

TASK [command] 
*****************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "python -c 'ksjfasdlkjf'", "delta": "0:00:00.037459", "end": "2017-10-03 19:58:50.525257", "failed": true, "rc": 1, "start": "2017-10-03 19:58:50.487798", "stderr": "Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\nNameError: name 'ksjfasdlkjf' is not defined", "stdout": "", "stdout_lines": [], "warnings": []}
        to retry, use: --limit @<snip>playbooks/backUpWebsite.retry

PLAY RECAP 
*********************************************************************
localhost                  : ok=0    changed=0    unreachable=0 failed=1   

Here is the script that generated it:

---
- hosts: localhost
  gather_facts: False
  tasks:
      #wrong on purpose!
    - shell: "python -c 'ksjfasdlkjf'"
      register: unobtainable

Upvotes: 11

Views: 14342

Answers (1)

stochastic
stochastic

Reputation: 3413

To get ansible to respect linebreaks in output so that you can read the stdout of your failing task, change the stdout_callback to minimal.

In your ansible.cfg, change the stdout_callback option:

stdout_callback=minimal

the 'minimal' output is a bit terse, but it prints output respecting linebreaks so that you can see what is happening:

 $ ansible-playbook playbooks/backUpWebsite.yml
 localhost | FAILED | rc=1 >>
 Traceback (most recent call last):
   File "<string>", line 1, in <module>
 NameError: name 'ksjfasdlkjf' is not defined

         to retry, use: --limit @<snip>/playbooks/backUpWebsite.retry

Other things that I hear can work

  • You could write your own callback handler for stdout. People have done this and it seems straightforward to do, but I've never tried it.

Things I tried that don't work:

Three vs. Use "ansible-playbook -vvv". You get a tremendous pile of output, which does include the json nicely linebroken, but the newlines within the stdout output are not printed, so stdout/stderr are still hard to read:

$ ansible-playbook playbooks/backUpWebsite.yml -vvv
Using <snip>/ansible.cfg as config file

PLAYBOOK: backUpWebsite.yml ****************************************************
1 plays in playbooks/backUpWebsite.yml

PLAY [localhost] ***************************************************************

TASK [command] *****************************************************************
task path: <snip>/playbooks/backUpWebsite.yml:6
Using module file <snip>/ansible/lib/ansible/modules/core/commands/command.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: <snip>
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205 `" && echo ansible-tmp-1507083519.4247632-54588405727205="` echo <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpumtlourx TO <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205/command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205/ <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205/command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '<snip>/anaconda3/bin/python <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205/command.py; rm -rf "<snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205/" > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
    "changed": true,
    "cmd": "python -c 'ksjfasdlkjf'",
    "delta": "0:00:00.037657",
    "end": "2017-10-03 20:18:39.843933",
    "failed": true,
    "invocation": {
        "module_args": {
            "_raw_params": "python -c 'ksjfasdlkjf'",
            "_uses_shell": true,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "warn": true
        },
        "module_name": "command"
    },
    "rc": 1,
    "start": "2017-10-03 20:18:39.806276",
    "stderr": "Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\nNameError: name 'ksjfasdlkjf' is not defined",
    "stdout": "",
    "stdout_lines": [],
    "warnings": []
}
        to retry, use: --limit @<snip>/sys/ansible_readynet/playbooks/backUpWebsite.retry

PLAY RECAP *********************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1   

Credit

I posted this issue against ansible on github in hope of persuading people that the ansible defaults are very unhelpful. jhawkesworth helpfully introduced me to the 'minimal' stdout callback.

I have since found this page in the ansible documentation that mentions this plugin in a very non-prominent way.

IMHO, this secret of getting readable error output from ansible is entirely too well hidden from new users, and I have personally witnessed this sour several people I know on the use of ansible in general.

Upvotes: 8

Related Questions