How to forward ssh key in ansible to checkout git repository on target machine from origin?

Question

I'm trying to configure ansible to checkout a git repository from bitbucket and put it on the target machine. The control machine (my PC) has the bitbucket private ssh key. The public key is uploaded to bitbucket and ssh access with it is tested and working.

Here's the ansible yml task code:

- name: Checkout application
  become: no
  git: repo=git@bitbucket.org:bitbucketusername/deployment.git
       dest=/tmp/myapp
       accept_hostkey=True
       key_file=/home/me/.ssh/bitbucket_ssh_key

Here's the error:

Warning: Identity file /home/me/.ssh/bitbucket_ssh_key not accessible: 
No such file or directory.
Permission denied (publickey)

So I take it that key forwarding is not working? This is odd, because in my ~/.ssh/config I have forwarding enabled for the target machine 111.222.333.444:

Host 111.222.333.444
  ForwardAgent yes

So what's going wrong and how do I get my repo from bitbucket onto my target machine? I'm using ansible 2.1.1.0.

Edit: here's the part where it complains when the -vvvv flag is on:

TASK [Checkout application] *********************************************
task path: /home/me/path/to/the/ansible/playbook.yml:49
<111.222.333.444> ESTABLISH SSH CONNECTION FOR USER: deploy
<111.222.333.444> SSH: EXEC ssh -C -vvv -o ForwardAgent=yes -o StrictHostKeyChecking=no -o 'IdentityFile="/home/me/.ssh/the_ssh_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 111.222.333.444 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1477655760.17-42684399995480 `" && echo ansible-tmp-1477655760.17-42684399995480="` echo $HOME/.ansible/tmp/ansible-tmp-1477655760.17-42684399995480 `" ) && sleep 0'"'"''
<111.222.333.444> PUT /tmp/tmp0NYGtg TO /home/deploy/.ansible/tmp/ansible-tmp-1477655760.17-42684399995480/git
<111.222.333.444> SSH: EXEC sftp -b - -C -vvv -o ForwardAgent=yes -o StrictHostKeyChecking=no -o 'IdentityFile="/home/me/.ssh/the_ssh_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 '[111.222.333.444]'
<111.222.333.444> ESTABLISH SSH CONNECTION FOR USER: deploy
<111.222.333.444> SSH: EXEC ssh -C -vvv -o ForwardAgent=yes -o StrictHostKeyChecking=no -o 'IdentityFile="/home/me/.ssh/the_ssh_key"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=deploy -o ConnectTimeout=10 -tt 111.222.333.444 '/bin/sh -c '"'"'LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 /usr/bin/python /home/deploy/.ansible/tmp/ansible-tmp-1477655760.17-42684399995480/git; rm -rf "/home/deploy/.ansible/tmp/ansible-tmp-1477655760.17-42684399995480/" > /dev/null 2>&1 && sleep 0'"'"''
fatal: [app1]: FAILED! => {"changed": false, "cmd": "/usr/bin/git ls-remote '' -h refs/heads/HEAD", "failed": true, "invocation": {"module_args": {"accept_hostkey": true, "bare": false, "clone": true, "depth": null, "dest": "/tmp/myapp", "executable": null, "force": false, "key_file": "/home/me/.ssh/bitbucket_ssh_key", "recursive": true, "reference": null, "refspec": null, "remote": "origin", "repo": "git@bitbucket.org:memeares/deployment.git", "ssh_opts": null, "track_submodules": false, "update": true, "verify_commit": false, "version": "HEAD"}, "module_name": "git"}, "msg": 
"Warning: Identity file /home/me/.ssh/bitbucket_ssh_key not accessible: No such file or directory.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.", "rc": 128, "stderr": "Warning: Identity file /home/me/.ssh/bitbucket_ssh_key not accessible: No such file or directory.
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
", "stdout": "", "stdout_lines": []}

And I've also got forwarding configured in the ansible.cfg file (as visible in the above output):

[ssh_connection]
# Enable SSH Agent Forwarding so that the private key used to be able to
# checkout from git does not have to be on the server
ssh_args=-o ForwardAgent=yes

Answer 1

The final error that fixed it was adding the key to the ssh-agent by ssh-add ~/.ssh/the_ssh_key.

I think in my case it was a combination of errors though that got me to this point, since I had previously added the key using ssh-add but had destroyed the instance after trying to debug other errors.

For reference, the debugging checklist.

And stuff that's not mentioned there:

• If the key isn't id_rsa, then manually specify it using the -i the_ssh_key
• Ensure the server actually has a copy of the public key the_ssh_key.pub file in the relevant user's ~/.ssh directory
• Ensure /etc/ssh/sshd_config has AllowAgentForwarding yes
• ssh-agent -L tells you whether the ssh-agent "knows about" your ssh key in question. (I don't know why this is necessary when you specify the key file with ssh -i the_ssh_key)
• Include the -o AllowAgentForwarding=yes when sshing from the control machine.