Ansible copy file to similar path locations

Question

Currently I am trying to replace all Tomcat keystore files in a particular location across multiple nodes. The problem is, the directory structures are similar, but not exactly the same.

For example, our tomcat directory structure looks like this: /home/tomcat121test/jdk-11.0.7+10.

But across the different nodes, the paths are slightly different. The differences are the tomcat folder name and the jdk folder name.

The structure is /home/tomcat<version_no><test_or_prod>/jdk-<jdk_version> All in one word for each folder names.
e.g. /home/tomcat11test/jdk-11.0.7+10

So, the idea is to use cp as shown in the task named Backup the current keystore: cp -p /home/tomcat*/jdk*/keystore /home/tomcat*/jdk*/keystore_old_2021

My play book currently looks like this:

---
- name: Update Tomcat Test Servers Keystore     
  hosts: tomcattest_servers   
  gather_facts: False


  tasks:
  - name: ls the jdk dir
    shell: ls -lah /home/tomcat*/jdk*/bin/
    register: ls_command_output
  - debug:
      var: ls_command_output.stdout_lines

  - name: Backup the current keystore
    shell: >
     cp -p /home/tomcat*/jdk*/keystore /home/tomcat*/jdk*/keystore_old_2021

  - name: Verify copy took place
    shell: ls -lah /home/tomcat*/jdk*/bin
    register: ls_command_output
  - debug:
      var: ls_command_output.stdout_lines

Task names Backup the current keystore is where it seems to be failing.

TASK [Backup the current keystore]
******************************************************************************************************************* fatal: [tomcattest1]: FAILED! => {"changed": true, "cmd": "cp -p
/home/tomcat*/jdk*/keystore /home/tomcat*/jdk*/keystore_old_2021\n",
"delta": "0:00:00.005322", "end": "2022-03-13 18:57:06.091283", "msg":
"non-zero return code", "rc": 1, "start": "2022-03-13
18:57:06.085961", "stderr": "cp: cannot stat
‘/home/tomcat*/jdk*/keystore’: No such file or directory",
"stderr_lines": ["cp: cannot stat ‘/home/tomcat*/jdk*/keystore’: No
such file or directory"], "stdout": "", "stdout_lines": []}

The task names ls the jdk dir works fine and they're both using the shell module, which, in my understanding, is needed if a wildcard needs to be used, instead of the command module.

Answer 1

After reviewing and reading what @β.εηοιτ.βε mentioned about using find, I went back and tried some other things before implementing what was mentioned. ( just needed something quick and fast )

- name: Find the tomcat*/jdk*/bin/keystore, make copy of
  shell: find /home/ -iname keystore -exec cp -p "{}" "{}_old_2021" \;

- name: Check for copied keystore
  shell: ls -lah /home/tomcat*/jdk*/bin/keystore*
  register: ls_command_output
- debug:
    var: ls_command_output.stdout_lines

This did exactly what I needed.

UPDATE:

While the above command worked, when it came time to use the copy module to copy the keystore that was going to be replaced ... the issue came up here:

   - name: Copy New Keystore to Tomcat TEST servers
    copy:
     src: /opt/ansible/playbooks/ssl-renew/keystore
     dest: /home/tomcat*/jdk*/bin/ 

Note the destination path. This did not work. I had to specify the specific path.

So I will be looking more into @β.εηοιτ.βε solution above. I also wanted to say thank you for the detailed and descriptive response to my initial post. I also looked into the fileglob module, but the first note on the ansible documentation page for fileglob, states: "Patterns are only supported on files, not directory/paths."

Answer 2

Here is how I would rephrase, then approach your requirement.

Problem statement:

1. In /home, we have an unknown folder that match a pattern tomcat.* to find.
2. In the folder found here above, we have an unknown folder that match a pattern jdk.* to find.
3. In the folder found here above, I want to ship a new file and backup the state of the existing file prior to copying.

Applying the DRY (Do Not Repeat Yourself) principle:

Clearly the first and second point of our problem statement seem to be the same, so it would be nice if we could have some sort of mechanism that could answer the requirement: "For a given path, return me a unique folder matching a pattern".

Solution:

Ansible have multiple ways of helping you create sets of tasks that you can reuse. Here is a, not exhaustive, list of two of them:

• roles: a quite extensive way to reuse multiple Ansible artefacts, including, but not limited to tasks, variables, handlers, files, etc.
• the include_tasks module that allows you to load an arbitrary YAML containing a list of tasks

Because role is a quite extensive mechanism, it requires the creation of a set of folders that would be unrelated to this solution, so I am going to demonstrate this using the include_tasks module, but depending on your needs and reusability considerations, creating a role might be a better bet.

So, here is what would be the YAML that we would use in the include_tasks:

1. a find task based on a given folder
2. an extraction of the folder matching the given pattern out of the result of the find task, using the selectattr filter and the match test.
3. an assertion that we have a unique folder matching our pattern

This gives us a file, called here find_exactly_one_folder.yml:

- find:
    path: "{{ root_folder }}"
    file_type: directory
  register: find_exactly_one_folder

- set_fact:
    found_folder: >
      {{
        find_exactly_one_folder.files 
        | selectattr('path', 'match', root_folder) 
        | map(attribute='path')
      }}

- assert:
    that: 
      - found_folder | length == 1
    fail_msg: >-
      Did not found exactly one folder, result: `{{ found_folder }}`.
    success_msg: >-
      {{ found_folder.0 | default('') }} found             

Now that we have that "For a given path, return me a unique folder matching a pattern" mechanism, we can have a playbook doing:

1. Find a unique folder matching the pattern tomcat.* from /home
2. Find a unique folder matching the pattern jdk.* from the folder resulting from the previous task
3. Copy the new file in the found folder, using the existing backup mechanism

This result in this set of tasks:

- include_tasks:
    file: find_exactly_one_folder.yml
  vars:
    root_folder: /home
    folder_match: 'tomcat.*'

- include_tasks:
    file: find_exactly_one_folder.yml
  vars:
    root_folder: "{{ found_folder.0 }}"
    folder_match: 'jdk.*'

- copy:
    src: keystore
    dest: "{{ found_folder.0 }}/keystore"
    backup: true

---

Here is for an example playbook, that ends with an extra find and debug task to demonstrate the resulting backup file is being created.

- hosts: node1
  gather_facts: no

  tasks:
    - include_tasks:
        file: find_exactly_one_folder.yml
      vars:
        root_folder: /home
        folder_match: 'tomcat.*'

    - include_tasks:
        file: find_exactly_one_folder.yml
      vars:
        root_folder: "{{ found_folder.0 }}"
        folder_match: 'jdk.*'

    - copy:
        src: keystore
        dest: "{{ found_folder.0 }}/keystore"
        backup: true

    - find:
        path: "{{ found_folder.0 }}"
        pattern: "keystore*"
      register: keystores

    - debug:
        var: keystores.files | map(attribute='path')

This would yield:

PLAY [node1] **************************************************************

TASK [include_tasks] ******************************************************
included: /usr/local/ansible/find_exactly_one_folder.yml for node1

TASK [find] ***************************************************************
ok: [node1]

TASK [set_fact] ***********************************************************
ok: [node1]

TASK [assert] *************************************************************
ok: [node1] => changed=false 
  msg: |-
    /home/tomcat11test found

TASK [include_tasks] ******************************************************
included: /usr/local/ansible/find_exactly_one_folder.yml for node1

TASK [find] ***************************************************************
ok: [node1]

TASK [set_fact] ***********************************************************
ok: [node1]

TASK [assert] *************************************************************
ok: [node1] => changed=false 
  msg: |-
    /home/tomcat11test/jdk-11.0.7+10 found

TASK [copy] **************************************************************
changed: [node1]

TASK [find] **************************************************************
ok: [node1]

TASK [debug] *************************************************************
ok: [node1] => 
  keystores.files | map(attribute='path'):
  - /home/tomcat11test/jdk-11.0.7+10/keystore.690.2022-03-13@22:11:08~
  - /home/tomcat11test/jdk-11.0.7+10/keystore