CODEWITHSUNDEEP
python-3.xbashsshansibleremote-server
Filip
Filip

Reputation: 21

ansible.builtin.shell not working with disown and weird output redirection

I'm having a hard time understanding Ansible's overall remote behaviour.

I'm referring to all ansible.builtin.command and ansible.builting.shell modules.

It seems not to collet any output for scripts put in the background and disowned.

It seems to fail properly disown jobs (although nohup seems to work just fine, beside redirections -- so still there is a problem of loosing the whole background job's output).

I want to install a python templated script on multiple remote machines, run it and detach from Ansible's connection so that it runs in the background (as it's a blocking call listening on TCP port). The python script is a couple of imports, print statements and then call to a server.start(), which is a blocking listening on a TCP port.

I wrote this task:

 - name: start services  # not working clearly 
    ansible.builtin.shell:
      executable: /bin/bash
      cmd: | 
        python3 <<-'EOF' &
        {{ lookup("ansible.builtin.template", "../scripts/server_start.tmpl", template_vars={"service": item}) }}
        EOF
        disown -h %1
    loop: "{{hostvars[inventory_hostname].services}}"

which does not work at all, as:

Can anyone give some logical explanation for this? I have tested it locally and it works as expected. I have even tested the script locally by running /bin/bash -c 'python3 <<-'EOF' ...', as that's how it's executed by ansible, I suppose. I have even tested it via SSH and still works. What is it with Ansible that it looses the output and does not correctly disown the job?

As a side note, Ansible captures the output correctly if I omit the blocking server.start() call. I have replaced server.start() call with a long sleep() and it does not work neither.

Upvotes: 1

Views: 465

Answers (2)

Filip
Filip

Reputation: 21

For anyone running into similar troubles with processes spawned remotely using Ansible: https://github.com/ansible/ansible/issues/33410. It seems that there is a re-occuring problem between Ansible's versions which results in killing all child processes upon terminating remote connection. This seems to answer most of my doubts.

I have changed the task to use nohup and python script to use logger with a file handler (as I mentioned in the original post, redirection of script's stdout did not work).

Upvotes: 1

U880D
U880D

Reputation: 12142

Understanding Ansible's overall remote behaviour

Q: "What is it with Ansible that it looses the output and does not correctly disown the job? ... Can anyone give some logical explanation for this?"

It seems that assumptions made are not correct in this case.

I'm referring to all ansible.builtin.command and ansible.builting.shell modules.

These modules are not mentioned as remote or interactive shells like in your SSH session. See How does Ansible work?

It seems not to collcet any output for scripts put in the background and disowned.

Right, that seems to be the expected behavior here. According Introduction to modules and Return Values

Ansible executes each module, usually on the remote managed node, and collects return values.

and

Ansible modules normally return a data structure that can be registered into a variable ...

therefore such behavior is not implemented. And regarding

It seems to fail properly disown jobs (although nohup seems to work just fine, beside redirections -- so still there is a problem of loosing the whole background job's output)

you may see also what is the difference between nohup, disown and &.

As already mentioned within the comments there is only the possibility is to run concurrently: poll = 0 for long running tasks.

If you want to run multiple tasks in a playbook concurrently, use async with poll set to 0. When you set poll: 0, Ansible starts the task and immediately moves on to the next task without waiting for a result.

However, it will not be possible to register both, job_id and rc.

Further Reading

which might help to understand the issue

Upvotes: 0

Related Questions