Remove a given hostname+URL from a line containing 3, separated by commas, in any position, using Ansible playbook

Question

Scenario: I have a configuration file for etcd, and one of the nodes in the cluster has failed. I know the name of the failed node, but not its IP address nor the names of the other two hosts in the cluster. I need to write an Ansible play to remove the failed node from a line in the etcd config file, (presumably) using the Ansible builtin replace which (I believe) uses Python as its RE engine.

I have managed to create something that works, with one caveat: If the failed host is the third one listed, the RE leaves a dangling comma at the end of the line. I'm hoping that someone smarter than I am can edit or replace my regex to cover all three positional cases.

The hostname of the failed node is passed into the playbook as a variable, so {{ failed_node }} would be substituted for the actual hostname of the failed node, let's call it app-failedhost-eeeeeeeeee.node.consul in my example.

Given a regex

((?:^ETCD_INITIAL_CLUSTER=)(?:[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})(,?{{ failed_node }}=https:\/\/[0-9]+(?:[.][0-9]+){3}:2380,?)((?:,?[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})

which when being actually run would be (if failed_node=app-failedhost-eeeeeeeeee.node.consul)

((?:^ETCD_INITIAL_CLUSTER=)(?:[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})(,?app-failedhost-eeeeeeeeee.node.consul=https:\/\/[0-9]+(?:[.][0-9]+){3}:2380,?)((?:,?[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})

if run against one of these lines,

ETCD_INITIAL_CLUSTER=app-failedhost-eeeeeeeeee.node.consul=https://192.168.18.39:2380,app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380

ETCD_INITIAL_CLUSTER=app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-failedhost-eeeeeeeeee.node.consul=https://192.168.18.39:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380

ETCD_INITIAL_CLUSTER=app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380,app-failedhost-eeeeeeeeee.node.consul=https://192.168.18.39:2380

(which if you simplify, is ETCD_INITIAL_CLUSTER= followed by three pairs of values, comma-separated, FQDN=https://[IP address]:2380 with the failed node in position 0, 1, or 2) and the replace: is '\1\3', you get

ETCD_INITIAL_CLUSTER=app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380

ETCD_INITIAL_CLUSTER=app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380

ETCD_INITIAL_CLUSTER=app-instance-de24a5c1aefb.node.consul=https://192.168.18.92:2380,app-instance-6cc297ab3cc.node.consul=https://192.168.18.11:2380,

That's correct for the first two cases (failed node in first or second position) but if the failed node is in the third (last) position as in the third example line, then the final comma is left behind.

https://regex101.com/r/f635Wv/1 has the same examples as above.

Playbook, in case the full situation is not clear from the regex above, called node-cleanup.yaml, is called with ansible-playbook node-cleanup.yaml --extra-vars "failed_node=app-failedhost-eeeeeeeeee.node.consul" in the above examples:

---
- name: Clean up failed etcd node
  hosts: etcd
  become: true
  tasks:
  - name: Remove failed host from ETCD_INITIAL_CLUSTER line
    replace:
      path: "/etc/etcd/etcd.conf"
      regexp: '((?:^ETCD_INITIAL_CLUSTER=)(?:[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})(,?{{ failed_node }}=https:\/\/[0-9]+(?:[.][0-9]+){3}:2380,?)((?:,?[a-z0-9-.]{15,}=https:\/\/[0-9]+(?:\.[0-9]+){3}:2380,?){0,2})'
      replace: '\1\3'

but I think that part is fine, I just need some help with that beast of a regex.

---

If the line in the file before is simplified as

ETCD_INITIAL_CLUSTER=host1=IP,host2=IP,host3=IP

and I pass in “host3” for {{ failed_node }}, then I want

ETCD_INITIAL_CLUSTER=host1=IP,host2=IP

to come out, but what I actually get is

ETCD_INITIAL_CLUSTER=host1=IP,host2=IP,

(note the trailing comma)

Answer 1

Given the file

shell> cat test.conf
ETCD_INITIAL_CLUSTER=host1=IP,host2=IP,host3=IP

and the variable

    failed_node: host3

Get the line from the configuration file. There are many options depending on the file is local or remote, e.g.

    - shell: cat test.conf | grep ETCD_INITIAL_CLUSTER
      register: result
      check_mode: false
    - set_fact:
        eic: "{{ result.stdout }}"

gives

  eic: ETCD_INITIAL_CLUSTER=host1=IP,host2=IP,host3=IP

Split the key/value pair and create a new value by rejecting the failed node

    - set_fact:
        _value: "{{ eic|regex_replace('^(.*?)=(.*)$', '\\2') }}"
        _key: "{{ eic|regex_replace('^(.*?)=(.*)$', '\\1') }}"
    - set_fact:
        _new_value: "{{ _hip|reject('search', failed_node) }}"
      vars:
        _hip: "{{ _value.split(',') }}"

gives

  _new_value:
  - host1=IP
  - host2=IP

Now update the key in the configuration file, e.g.

    - replace:
        path: test.conf
        regexp: '{{ _key }}\s*=\s*{{ _value }}'
        replace: '{{ _key }}={{ _new_value|join(",") }}'

running the playbook in the check mode (--check --diff) gives

+++ after: test.conf
@@ -1 +1 @@
-ETCD_INITIAL_CLUSTER=host1=IP,host2=IP,host3=IP
+ETCD_INITIAL_CLUSTER=host1=IP,host2=IP

---

The procedure can be optimized. The tasks below do the same job

    - shell: cat test.conf | grep ETCD_INITIAL_CLUSTER
      register: result
      check_mode: false
    - replace:
        path: test.conf
        regexp: '{{ _key }}\s*=\s*{{ _value }}'
        replace: '{{ _key }}={{ _new_value|join(",") }}'
      vars:
        _key: "{{ result.stdout|regex_replace('^(.*?)=(.*)$', '\\1') }}"
        _value: "{{ result.stdout|regex_replace('^(.*?)=(.*)$', '\\2') }}"
        _new_value: "{{ _value.split(',')|reject('search', failed_node) }}"

---

There are other options on how to get the line from the configuration file. For example, if the file is local, the Ansible way would be lookup plugin, e.g.

    - debug:
        msg: "{{ lookup('ini', 'ETCD_INITIAL_CLUSTER type=properties file=test.conf') }}"

gives the value of ETCD_INITIAL_CLUSTER

  msg: host1=IP,host2=IP,host3=IP

This would further reduce the job to a single task

    - replace:
        path: test.conf
        regexp: '{{ _key }}\s*=\s*{{ _value }}'
        replace: '{{ _key }}={{ _new_value|join(",") }}'
      vars:
        _key: ETCD_INITIAL_CLUSTER
        _value: "{{ lookup('ini', _key ~ ' type=properties file=test.conf') }}"
        _new_value: "{{ _value.split(',')|reject('search', failed_node) }}"