Ansible - Unspecified Tags Still Executing

Question

Running --list-tags against my playbook, deploy.yml, I get the following:

  play #1 (localhost): Extract Variable Data From Jenkins Call  TAGS: [main,extract]
      TASK TAGS: [extract, main]

  play #2 (localhost): Pull Repo to Localhost   TAGS: [main,pullRepo]
      TASK TAGS: [main, pullRepo]

  play #3 (gms): Deploy GMS     TAGS: [main,gmsDeploy]
      TASK TAGS: [gmsDeploy, main]

  play #4 (localhost): Execute SQL Scripts      TAGS: [main,sql]
      TASK TAGS: [main, sql]

  play #5 (localhost): Configure Hosts for Copying      TAGS: [main,config,tibco]
      TASK TAGS: [config, main, tibco]

  play #6 (copy_group): Copy Files between servers      TAGS: [main,tibco]
      TASK TAGS: [main, tibco]

  play #7 (localhost): Deploy TIBCO     TAGS: [main,tibco]
      TASK TAGS: [main, tibco]

  play #8 (localhost): Validate Deployment with GMS Heartbeat   TAGS: [validate,main]
      TASK TAGS: [main, validate]

  play #9 (tibco): Clean Up     TAGS: [main,cleanup]
      TASK TAGS: [cleanup, main]

However, when I run ansible-playbook deploy.yml --tags "pullRepo" in an attempt to execute only the 2nd play, it still attempts to execute the include tasks on play #4. I think the includes (the act of including, not the tasks being included) are the issue because the tasks before the includes in play #4 don't execute. I was hoping the issue was that the included tasks weren't tagged. Unfortunately, adding the sql tags to the included tasks still shows attempts to execute the included tasks.

I know I can avoid running them with --skip-tags, but I shouldn't have to. Any suggestions why this may be happening?

I would post the playbook, but it's hundreds of lines containing proprietary information and I can't seem to replicate the issue with an MCVE.

NOTE: I'm not using any roles in the playbook, so tags being applied to the tasks due to roles is not a factor. All tags are either on the entire plays or the tasks within the plays (primarily the former).

---

Pretty close to an MCVE:

---

#:PLAY 2 - PULL REPO (LOCALLY)
- name: Pull Repo to Localhost
  hosts: localhost
  any_errors_fatal: true
  tags:
    - pullRepo
    - main
  gather_facts: no
  tasks:

    # Save the repository to the control machine (this machine) for distribution
    - name: Pulling Git Repo for Ansible...
      git: 
        repo: 'https://github.com/ansible/ansible.git'
        dest: '/home/dholt2/ansi'
        accept_hostkey: yes
        force: yes
        recursive: no



# Execute the sql files in the 'Oracle' directory after
## checking if the directory exists
#:PLAY 4 - TEST AND EXECUTE SQL (LOCALLY)
- name: Test & Execute SQL Scripts
  hosts: localhost
  any_errors_fatal: true
  tags:
    - sql
  gather_facts: no
  tasks:

    # Check if the 'Oracle' directory exists. Save the
    ## output to deploy 'Oracle/*' if it does exist.
    - name: Check for presence of the Oracle directory
      stat:
        path: '/home/dholt2/testing/Oracle'
      register: st3


    # Get a list of all sql scripts (using 'ls -v') to run ONLY if the 'Oracle'
    ## directory exists; exclude the rollback script; -v ensures natural ordering of the scripts
    - name: Capture All Scripts To Run
      shell: 'ls -v /home/dholt2/testing/Oracle -I rollback.sql'
      register: f1
      when: st3.stat.isdir is defined

    # Test that the deployment scripts run without error
    - name: Testing SQL Deployment Scripts...
      include: testDeploySql.yml
      any_errors_fatal: true
      loop_control:
        loop_var: deploy_item
      with_items: "{{ f1.stdout_lines }}"
      register: sqlDepl
      when: st3.stat.isdir is defined

The output from executing ansible-playbook deploy2.yml --tags "pullRepo" -vvv:

PLAYBOOK: deploy2.yml *********************************************************************************************************************************************************************************************************************************************************
2 plays in deploy2.yml

PLAY [Pull Repo to Localhost] *************************************************************************************************************************************************************************************************************************************************
META: ran handlers

TASK [Pulling Git Repo for Ansible...] ********************************************************************************************************************************************************************************************************************************************
task path: /home/dholt2/test/deploy2.yml:17
Using module file /usr/lib/python2.6/site-packages/ansible/modules/source_control/git.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: dholt2
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/dholt2/.ansible/tmp/ansible-tmp-1506540242.05-264367109987412 `" && echo ansible-tmp-1506540242.05-264367109987412="` echo /home/dholt2/.ansible/tmp/ansible-tmp-1506540242.05-264367109987412 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpVWukIT TO /home/dholt2/.ansible/tmp/ansible-tmp-1506540242.05-264367109987412/git.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/dholt2/.ansible/tmp/ansible-tmp-1506540242.05-264367109987412/ /home/dholt2/.ansible/tmp/ansible-tmp-1506540242.05-264367109987412/git.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python2.6 /home/dholt2/.ansible/tmp/ansible-tmp-1506540242.05-264367109987412/git.py; rm -rf "/home/dholt2/.ansible/tmp/ansible-tmp-1506540242.05-264367109987412/" > /dev/null 2>&1 && sleep 0'
changed: [localhost] => {
    "after": "5c3bbd4620c4a94ece7741ecfe5514a1bd06422b",
    "before": null,
    "changed": true,
    "invocation": {
        "module_args": {
            "accept_hostkey": true,
            "bare": false,
            "clone": true,
            "depth": null,
            "dest": "/home/dholt2/ansi",
            "executable": null,
            "force": true,
            "key_file": null,
            "recursive": false,
            "reference": null,
            "refspec": null,
            "remote": "origin",
            "repo": "https://github.com/ansible/ansible.git",
            "ssh_opts": null,
            "track_submodules": false,
            "umask": null,
            "update": true,
            "verify_commit": false,
            "version": "HEAD"
        }
    }
}
META: ran handlers
META: ran handlers

PLAY [Test & Execute SQL Scripts] *********************************************************************************************************************************************************************************************************************************************
META: ran handlers

TASK [Testing SQL Deployment Scripts...] **************************************************************************************************************************************************************************************************************************************
task path: /home/dholt2/test/deploy2.yml:54
fatal: [localhost]: FAILED! => {
    "failed": true,
    "msg": "'f1' is undefined"
}

NO MORE HOSTS LEFT ************************************************************************************************************************************************************************************************************************************************************

PLAY RECAP ********************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=1    unreachable=0    failed=1

I created the following directory structure and files for this example: /home/dholt2/testing/Oracle

Inside of the directory, I created the files a.sql, b.sql , c.sql, and rollback.sql

Just as it should, it skips over the first task in play #4 that searches for the Oracle directory, ignores the following task that gets a list of the files... yet it tries to execute the include even though it should only run if the previous task was successful. The result is that the variable the include task is trying to use doesn't exist — so it throws an error — even though it shouldn't even have passed on the when directive since the first task didn't execute to show that the directory exists and the previous task didn't get the list of files.

---

EDIT 1 on 09/28/2017: this task was also in the original playbook, it's the rollback for if the first sql scripts fail:

# Test that the rollback script runs without error
- name: Testing SQL Rollback Script...
  any_errors_fatal: true
  include: testRollbackSql.yml
  register: sqlRoll
  when: st3.stat.isdir is defined

---

EDIT 2 on 10/16/2017: I keep running into a similar problem. I'm often using a with_together where one of the items would always be populated with values, but the other may not (like f1 above). Due to this, the answer below to use the default() filter doesn't work because there's always something for the the include to loop over Fortunately, I found this official issue on Ansible's Github. It discusses why this is happening and how to get around it.

Essentially, you should still have the default() filter, but you need to add static: no to the task as well.

This backs up Konstantins statement about why Ansible may be running the tags even thouth they're not specified.

From the Github issue:

...this is an issue with static vs dynamic includes, inheritance and that when executes inside the with loops.

 

The issue is that with both static includes and when/with interactions Ansible does NOT know if it will skip the task.

Answer 1

I see..

include is not a usual task it's a kind of statement and this statement behave differently when used in static and dynamic manner.
To eliminate this ambiguity, import_tasks and include_tasks were introduced in Ansible 2.4.
Please see recent question on serverfault: https://serverfault.com/questions/875247/whats-the-difference-between-include-tasks-and-import-tasks/875292#875292

When you specify a tag, Ansible still goes through all tasks under the hood and silently skips those without corresponding tag. But when it hit dynamic include (like in your case), it must process this include statement, because included file may contain tasks marked with that tag.

But you have some undefined variable within you include statement, so it fails. This is expected behaviour.