Ansible dynamic Inventory with bash script

Question

We have recently built an API that allows us to query a list of VM's rather than using an inventory file. I am trying to incorporate this with our ansible set-up. I am very new to using a dynamic inventory and have tried a few different variations with bash since I am not very experienced with python.

This almost works but as long as the playbook does not have become:true. If the playbook requires sudo I get this error

FAILED! => {"changed": false, "module_stderr": "/bin/sh: sudo: command not found", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 127}

SERVER1=($(curl -sS --request GET https://foo/bar| awk 'BEGIN {FS="["} {print $1}'))

cat <<EOF
{
    "test1": {
        "hosts": ["$SERVER1"],
    }
}
EOF

I have tried to find examples of what I am trying to do but have been coming up short.

I have 3 groups I need to run a play on (test1,test2,test3) This is the closest I have gotten it to work. If doing this with python is easier or the preferred method I could work to figure it out I just can't tell at this point if i'm close to a solution or way off. Any help is appreciated.

SERVER1=($(curl -sS --request GET https://foo/bar| awk 'BEGIN {FS="["} {print $1}'))
SERVER2=($(curl -sS --request GET https://foo/bar| awk 'BEGIN {FS="["} {print $2}'))
SERVER3=($(curl -sS --request GET https://foo/bar| awk 'BEGIN {FS="["} {print $3}'))

cat <<EOF
{
    "test1": {
        "hosts": ["$SERVER1"],
    },
    "test2": {
        "hosts": ["$SERVER2"],
    },
    "test3": {
        "hosts": ["$SERVER3"],
    }
}
EOF

Answer 1

Thanks for this!

This was the missing bit of the puzzle.

I changed it so it can interact with AWS directly, using ansible-inventory and jq.

So I'm creating a temporary YAML hosts file using AWS cli + jq and then I'm requesting ansible-inventory to create the _meta data for me.

Here's how I've implemented:

#! /bin/bash

INVENTORY=$(mktemp -t dynhostsXXXX.yaml)

cat << EOF > ${INVENTORY}
all:
  hosts:
    siteframework.service.consul:
  children:
    ssh:
      children:
        ApplicationFoo:
        AutoScalingGroupBar:
        zabbix:
EOF

aws --profile example-prod \
    --region eu-west-1 \
    --output json \
    ec2 describe-instances  |\
    jq -r '.Reservations[].Instances[] | "\(
    if .Tags then .Tags[] |
        select ( .Key == "Environment" ) | .Value else "-" end
    )%\(
        if .Tags then .Tags[] | select ( .Key == "Name" ) |
        .Value else "-" end
    )%\(
        .InstanceId
    )%\(
        if .PrivateIpAddress then .PrivateIpAddress else "-" end
    )%\(
       .State.Name 
    )"' |\
   grep "%running" |\
   sort |\
   awk -F'%' '
       $2FS==x{
           printf "        %s:\n          ansible_host: %s\n", $3, $4
           next
       }
       {
           x=$2FS
           printf "\n    %s:\n      hosts:\n        %s:\n          ansible_host: %s\n", x, $3, $4
       }
       END {
           printf "\n"
       }' |  sed 's/%//g' >> ${INVENTORY}

if [ "$1" == "--list" ]; then
    ansible-inventory -i ${INVENTORY} --list
elif [ "$1" == "--host" ]; then
    echo '{"_meta": {hostvars": {}}}'
else
    echo "{ }"
fi

rm ${INVENTORY}

Note: I've changed some parts to not expose some bits of my infrastructure.

Answer 2

Still quite new at ansible myself but I believe from Ansible Inventory Guide the script must accept a minimum of two arguments --list and --host.

From a paywalled tutorial I'm doing, this is the format suggested for bash. I can't really explain as I just transcribed from the video.

#! /bin/bash

if [ "$1" == "--list" ]; then
cat<<EOF
{
  "bash_hosts": {
    "hosts": [
      "myhost.domain.com",
      "localhost"
    ],
    "vars": {
      "host_test": "test-value"
    }
  },
  "_meta": {
    "hostvars": {
      "myhost.domain.com": {
        "host_specific_test_var": "test-value"
      }
    }
  }
}
EOF
elif [ "$1" == "--host" ]; then
  echo '{"_meta": {hostvars": {}}}'
else
  echo "{ }"
fi