Ansible wait for ssh is not working

Question

Ansible wait_for ssh stopped working. Ansible playbook is able to launch ec2 instance, the wait_for module is being used to wait for ssh to start. Ansible times out while waiting for ssh.

My playbook

- name: Configuring and lanuching EC2 instance
  hosts: localhost
  connection: local
  vars_files:
    - general_vars

  tasks:
     - name: Launch instance
       ec2:  
        instance_type: "{{ instance_type }}"
        image: "{{ image_ami }}"
        region: "{{ region }}"
        vpc_subnet_id: "{{ subnet_id }}"
        assign_public_ip: yes
        group_id: "{{ security_group }}"
        key_name: "{{ key_pair }}" 
        volumes:
          - device_name: /dev/xvda
            volume_type: gp2
            volume_size: 8

       register: ec2

     - name: Wait for ssh to come up
       wait_for: host="{{ ec2.instances[0].public_dns_name }}" port=22 delay=10  timeout=300
       with_items: "{{ ec2.instances }}"
     - name: Save the Ip Address of the machine
       add_host:
           hostname: "{{ item.public_ip }}"
           groupname: ec2_instance_ips
       with_items: "{{ ec2.instances }}"

Error trace

The full traceback is:
  File "/var/folders/y3/t87nx4q95w9_jmg80csrn6bc0000gn/T/ansible_SpGSRH/ansible_module_wait_for.py", line 540, in main
    s = _create_connection(host, port, min(connect_timeout, alt_connect_timeout))
  File "/var/folders/y3/t87nx4q95w9_jmg80csrn6bc0000gn/T/ansible_SpGSRH/ansible_module_wait_for.py", line 405, in _create_connection
    connect_socket = socket.create_connection((host, port), connect_timeout)
  File "/usr/local/opt/python@2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 575, in create_connection
    raise err

}) => {
    "changed": false,
    "elapsed": 300,
    "invocation": {
        "module_args": {
            "active_connection_states": [
                "ESTABLISHED",
                "FIN_WAIT1",
                "FIN_WAIT2",
                "SYN_RECV",
                "SYN_SENT",
                "TIME_WAIT"
            ],
            "connect_timeout": 5,
            "delay": 10,
            "exclude_hosts": null,
            "host": "",
            "msg": null,
            "path": null,
            "port": 22,
            "search_regex": null,
            "sleep": 1,
            "state": "started",
            "timeout": 300
        }
    },
    "item": {
        "ami_launch_index": "0",
        "architecture": "x86_64",
        "block_device_mapping": {},
        "dns_name": "",
        "ebs_optimized": false,
        "groups": {
            "sg-09664e62": "yyyzzzzbbbbbb"
        },
        "hypervisor": "xen",
        "id": "i-XXxxxxxxxxxxxxxxx",
        "image_id": "XXXXXXXXXXXXX",
        "instance_type": "t2.micro",
        "kernel": null,
        "key_name": "XXXX",
        "launch_time": "XXXXXX",
        "placement": "ccccccccccccccc",
        "private_dns_name": "",
        "private_ip": "XXXXX",
        "public_dns_name": "",
        "public_ip": null,
        "ramdisk": null,
        "region": "XXXX",
        "root_device_name": "/dev/sda1",
        "root_device_type": "ebs",
        "state": "pending",
        "state_code": 0,
        "tags": {},
        "tenancy": "default",
        "virtualization_type": "hvm"
    },
    "msg": "Timeout when waiting for :22"

Ansible does not return anything for public_dns_name and public_ip is "null".

As a result the wait_for ssh fails, even though ec2 is spawned properly and I am able to ssh to machine via appropriate key manually.

What I am doing wrong?

Answer 1

wait_for_connection worked like a charm, it will wait until the host become reachable and sshable

        - name: Wait for the instance to become reachable and SSH becomes accessible
          delegate_to: "{{ item.private_ip_address }}"
          loop: "{{ ec2.instances }}"
          wait_for_connection:
            delay: 5
            timeout: 60

Answer 2

The module wait_for_connection worked for me. I think since Ansible uses ssh to connect to the instances, the module implicitly checks the SSH connection.

- name: "Wait 300 seconds, but only start checking after 60 seconds"
  wait_for_connection:
    delay: 60
    timeout: 300

Answer 3

You could try below playbook which will save your public Ip to Hosts file for future usage.

- name: Create an EC2 instance
  ec2:
    key_name: "{{ project_name }}-{{ env }}-key"
    region: "{{ region }}"
    group_id: "{{ test_firewall.group_id }}"
    instance_type: "{{ instance_type }}"
    image: "{{ ami }}"
    wait: yes
    instance_tags:
        env: "{{ env }}"
    count_tag: env
    exact_count: 1
    vpc_subnet_id: subnet-0e4be06e12efe8eca
    assign_public_ip: yes
  register: ec2

- name: Add the newly created EC2 instance(s) to host group
  lineinfile: dest={{ hostpath }}
              regexp={{ item.public_ip }}
              insertafter="[webserver]"
              line="{{ item.public_ip }} {{hoststring}}"
              state=present
  with_items: ec2.instances

- wait_for: path={{ hostpath }} search_regex={{hoststring}}

- name: Wait for SSH to come up
  local_action: wait_for
                host={{ item.public_ip }}
                port=22
                state=started
  with_items: ec2.instances

- name: Add IP to ec2_hosts group
  add_host: hostname={{ item.public_ip }} groups=ec2_hosts
  with_items: ec2.instances

Thanks

Answer 4

Ansible ec2 module has wait boolean value to wait for an ec2 instance to reach a desirable state. https://docs.ansible.com/ansible/2.6/modules/ec2_module.html

Once added your wait_for ssh would work.

Answer 5

- name: Wait for ssh to come up
  wait_for: host="{{ ec2.instances[0].public_dns_name }}" port=22 delay=10  timeout=300
  with_items: "{{ ec2.instances }}"

You are using the with_items construct, but I don't see any use of the item variable. I don't use AWS, but perhaps you need

 - name: Wait for ssh to come up
   wait_for: host="{{ item.public_dns_name }}" port=22 delay=10  timeout=300
   with_items: "{{ ec2.instances }}"