Ansible Multi data center deployment

Question

My dev inventory file is:

[dev]
service1 ansible_host=dev1.com ansible_port=22 ansible_user=ubuntu
service2 ansible_host=dev1.com ansible_port=22 ansible_user=ubuntu
service3 ansible_host=dev1.com ansible_port=22 ansible_user=ubuntu

I execute below roles for service:

roles:
- sync_code
- prep_local
- build
- shutdown_app
- setup_host
- start_app

I have serial: yes so when I deploy any service, each role is executed sequentially.

For prod, there are 2 datacenters, DC1 and DC2; services are deployed in both datacenters, and multiple servers in each datacenter can host that particular service.

When I start deployment of service1 in prod, I want to pick 1 server from DC1 and 1 server from DC2, run all the roles sequentially, and then pick the next 1-1 servers from each datacenter.

What's the best way to organize the inventory file and how should I set the host's value in the playbook to achieve this?

Basically, on two hosts at the same time, run a bunch of code in sequence.

Answer 1

Symmetric groups

Use the module add_host and create dynamic inventory group. For example, given the inventory

shell> cat hosts
[dc1]
node_1
node_2
node_3

[dc2]
node_4
node_5
node_6

The first play will create the inventory group rolling_update. The second play will use it

shell> cat pb.yml
- hosts: all
  tasks:
    - add_host:
        name: "{{ item }}"
        groups: rolling_update
      loop: "{{ groups.dc1|zip(groups.dc2)|flatten }}"
      run_once: true
    
- hosts: rolling_update
  serial: 2
  tasks:
    - debug:
        var: inventory_hostname

gives (abridged)

shell> ansible-playbook pb.yml 

PLAY [all] ***********************************************************************************

TASK [add_host] ******************************************************************************
changed: [node_1] => (item=node_1)
changed: [node_1] => (item=node_4)
changed: [node_1] => (item=node_2)
changed: [node_1] => (item=node_5)
changed: [node_1] => (item=node_3)
changed: [node_1] => (item=node_6)

PLAY [rolling_update] ************************************************************************

TASK [debug] *********************************************************************************
ok: [node_4] => 
  inventory_hostname: node_4
ok: [node_1] => 
  inventory_hostname: node_1

PLAY [rolling_update] ************************************************************************

TASK [debug] *********************************************************************************
ok: [node_2] => 
  inventory_hostname: node_2
ok: [node_5] => 
  inventory_hostname: node_5

PLAY [rolling_update] ************************************************************************

TASK [debug] *********************************************************************************
ok: [node_3] => 
  inventory_hostname: node_3
ok: [node_6] => 
  inventory_hostname: node_6
  ...

See:

• Setting the batch size with serial
• Selecting a strategy
• Ordering execution based on inventory

---

Asymmetric groups

If the groups are not symmetric, for example

shell> cat hosts
[dc1]
node_1
node_2
node_3

[dc2]
node_4
node_5

the filter zip won't exhaust the lists. The result will be (abridged)

PLAY [rolling_update] ************************************************************************

TASK [debug] *********************************************************************************
ok: [node_1] => 
  inventory_hostname: node_1
ok: [node_4] => 
  inventory_hostname: node_4

PLAY [rolling_update] ************************************************************************

TASK [debug] *********************************************************************************
ok: [node_2] => 
  inventory_hostname: node_2
ok: [node_5] => 
  inventory_hostname: node_5

If you want to run the play for all hosts use the filter zip_longest

      loop: "{{ groups.dc1|zip_longest(groups.dc2)|flatten }}"

Then, the result will be (abridged)

PLAY [rolling_update] ************************************************************************

TASK [debug] *********************************************************************************
ok: [node_1] => 
  inventory_hostname: node_1
ok: [node_4] => 
  inventory_hostname: node_4

PLAY [rolling_update] ************************************************************************

TASK [debug] *********************************************************************************
ok: [node_2] => 
  inventory_hostname: node_2
ok: [node_5] => 
  inventory_hostname: node_5

PLAY [rolling_update] ************************************************************************

TASK [debug] *********************************************************************************
ok: [node_3] => 
  inventory_hostname: node_3

---

General solution

Given the hosts

shell> cat hosts
[dc1]
node_1

[dc2]
node_4
node_5
node_6

the result will be (abridged)

PLAY [rolling_update] ************************************************************************

TASK [debug] *********************************************************************************
ok: [node_1] => 
  inventory_hostname: node_1
ok: [node_4] => 
  inventory_hostname: node_4

PLAY [rolling_update] ************************************************************************

TASK [debug] *********************************************************************************
ok: [node_5] => 
  inventory_hostname: node_5
ok: [node_6] => 
  inventory_hostname: node_6

If you always want to run the play for only one host from each group create two dynamic inventory groups

- hosts: all

  vars:

    ru1: "{{ groups.dc1|zip(groups.dc2)|flatten }}"
    idx: "{{ ru1|length - 1 }}"
    ru2: "{{ groups.dc1[idx|int:]|symmetric_difference(groups.dc2[idx|int:]) }}"

  tasks:

    - block:

        - debug:
            msg: |
              ru1: {{ ru1 }}
              ru2: {{ ru2 }}

        - add_host:
            name: "{{ item }}"
            groups: rolling_update1
          loop: "{{ ru1 }}"

        - add_host:
            name: "{{ item }}"
            groups: rolling_update2
          loop: "{{ ru2 }}"

      run_once: true
    
- hosts: rolling_update1
  serial: 2
  tasks:
    - debug:
        var: inventory_hostname
    
- hosts: rolling_update2
  serial: 1
  tasks:
    - debug:
        var: inventory_hostname

Then, the result will be (abridged)

PLAY [rolling_update1] ***********************************************************************

TASK [debug] *********************************************************************************
ok: [node_4] => 
  inventory_hostname: node_4
ok: [node_1] => 
  inventory_hostname: node_1

PLAY [rolling_update2] ***********************************************************************

TASK [debug] *********************************************************************************
ok: [node_5] => 
  inventory_hostname: node_5

PLAY [rolling_update2] ***********************************************************************

TASK [debug] *********************************************************************************
ok: [node_6] => 
  inventory_hostname: node_6

If you run this playbook on symmetric groups you'll see the warning

[WARNING]: Could not match supplied host pattern, ignoring: rolling_update2

[TODO: get rid of the warning]