Gitlab CI/Docker: ssh-add keeps asking for passphrase

Question

What I'm currently trying to do, is triggering an script on a remote machine from the Gitlab CI/CD Docker container. The job is configured as follows:

stages:
  - deploy

image: maven:3.3.9

server-deploy:
  stage: deploy
  allow_failure: false
  script:
    ## Install ssh agent
    - apt update && apt install openssh-client -y
    - eval $(ssh-agent -s)
    ## Create SSH key file
    - "echo \"-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW
QyNTUxOQAAACByjJBGT21Arna/pirWVXQqGAr/aszqQ5HzvrA2MzVDZAAAAJiGKEEKhihB
CgAAAAtzc2gtZWQyNTUxOQAAACByjJBGT21Arna/pirWVXQqGAr/aszqQ5HzvrA2MzVDZA
AAAEAKbObQgJGXbrKQt4wdCy3YQfpVBqkT5RNEt2IYU5pv3HKMkEZPbUCudr+mKtZVdCoY
Cv9qzOpDkfO+sDYzNUNkAAAAFHN2ZW5AREVTS1RPUC0xTjVKUjRSAQ==
-----END OPENSSH PRIVATE KEY-----\" > deploy-key"
    ## Fix permissions on key file and .ssh folder
    - chmod 700 deploy-key; mkdir -p ~/.ssh; chmod 700 ~/.ssh
    ## Import SSH key
    - ssh-add -k deploy-key
    ## Make sure that ssh will trust the new host, instead of asking
    - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
    ## Run script on the remote server
    - ssh -t user@255.255.255.255 "./deploy-master"

(The SSH key is just a temporary one, specifically generated for the SO question) Now the job fails when it arrives at the "ssh-add -k deploy-key" command, asking for a passphrase, as such:

$ ssh-add -k deploy-key
Enter passphrase for deploy-key: ERROR: Job failed: exit code 1

The SSH key obviously has no passphrase attached to it, I can verify this by running the exact same commands on my own Linux machine, where they just work as they should.

So my question is: how can I prevent ssh-add from asking for a passphrase? And I'm also quite curious why this is only occurring on the Gitlab CI Docker container and not on my own PC.

Thanks in advance!

Answer 1

ssh-add with an encrypted ssh-key in ssh-agent

This solution has an ed25519 encrypted ssh-key in the variable SSH_PRIVATE_KEY, and the passphrase to decrypt it in the variable SSH_PASSPHRASE.

image: ubuntu:trusty

before_script:
  ##
  ## Install ssh-agent if not already installed, it is required by Docker.
  ## (change apt-get to yum if you use an RPM-based image)
  ##
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client git -y )'

  ##
  ## Run ssh-agent (inside the build environment)
  ##
  - eval $(ssh-agent -s)

  ##
  ## Create the SSH directory and give it the right permissions
  ##
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh

  ## Create a shell script that will echo the environment variable SSH_PASSPHRASE
  - echo 'echo $SSH_PASSPHRASE' > ~/.ssh/tmp && chmod 700 ~/.ssh/tmp

  ##
  ## Why would you encrypt your private keys? Can I echo the value to stdout?
  - echo $SSH_PRIVATE_KEY

  ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
  ## We're using tr to fix line endings which makes ed25519 keys work
  ## without extra base64 encoding.
  ## https://gitlab.com/gitlab-examples/ssh-private-key/issues/1#note_48526556
  ##
  ## If ssh-add needs a passphrase, it will read the passphrase from the current
  ## terminal if it was run from a terminal.  If ssh-add does not have a terminal
  ## associated with it but DISPLAY and SSH_ASKPASS are set, it will execute the
  ## program specified by SSH_ASKPASS and open an X11 window to read the
  ## passphrase.  This is particularly useful when calling ssh-add from a
  ## .xsession or related script. Setting DISPLAY=None drops the use of X11.
  - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | DISPLAY=None SSH_ASKPASS=~/.ssh/tmp ssh-add -

  ##
  ## Use ssh-keyscan to scan the keys of your private server. Replace gitlab.com
  ## with your own domain name. You can copy and repeat that command if you have
  ## more than one server to connect to.
  ##
  - ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
  - chmod 644 ~/.ssh/known_hosts

  ##
  ## Alternatively, assuming you created the SSH_SERVER_HOSTKEYS variable
  ## previously, uncomment the following two lines instead.
  ##
  #- echo "$SSH_SERVER_HOSTKEYS" > ~/.ssh/known_hosts'
  #- chmod 644 ~/.ssh/known_hosts

Answer 2

Using a block in the yaml would probably work.

stages:
  - deploy

image: maven:3.3.9

server-deploy:
  stage: deploy
  allow_failure: false
  script:
    ## Install ssh agent
    - apt update && apt install openssh-client -y
    - eval $(ssh-agent -s)
    ## Create SSH key file
    - |
      echo '-----BEGIN OPENSSH PRIVATE KEY-----
      b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW
      QyNTUxOQAAACByjJBGT21Arna/pirWVXQqGAr/aszqQ5HzvrA2MzVDZAAAAJiGKEEKhihB
      CgAAAAtzc2gtZWQyNTUxOQAAACByjJBGT21Arna/pirWVXQqGAr/aszqQ5HzvrA2MzVDZA
      AAAEAKbObQgJGXbrKQt4wdCy3YQfpVBqkT5RNEt2IYU5pv3HKMkEZPbUCudr+mKtZVdCoY
      Cv9qzOpDkfO+sDYzNUNkAAAAFHN2ZW5AREVTS1RPUC0xTjVKUjRSAQ==
      -----END OPENSSH PRIVATE KEY-----' > deploy-key
    ## Fix permissions on key file and .ssh folder
    - chmod 700 deploy-key; mkdir -p ~/.ssh; chmod 700 ~/.ssh
    ## Import SSH key
    - ssh-add -k deploy-key
    ## Make sure that ssh will trust the new host, instead of asking
    - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
    ## Run script on the remote server
    - ssh -t user@255.255.255.255 "./deploy-master"

Answer 3

Okay, I got it working. It turns out that ssh-add is very picky about the format of the file and especially the newlines. The newlines in the .gitlab-ci.yml are not transferred directly to the command and so the key ended up being one big line.

Here is how I solved it:

- echo -----BEGIN OPENSSH PRIVATE KEY----- >> deploy-key
- echo b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW >> deploy-key
- echo QyNTUxOQAAACByjJBGT21Arna/pirWVXQqGAr/aszqQ5HzvrA2MzVDZAAAAJiGKEEKhihB >> deploy-key
- echo CgAAAAtzc2gtZWQyNTUxOQAAACByjJBGT21Arna/pirWVXQqGAr/aszqQ5HzvrA2MzVDZA >> deploy-key
- echo AAAEAKbObQgJGXbrKQt4wdCy3YQfpVBqkT5RNEt2IYU5pv3HKMkEZPbUCudr+mKtZVdCoY >> deploy-key
- echo Cv9qzOpDkfO+sDYzNUNkAAAAFHN2ZW5AREVTS1RPUC0xTjVKUjRSAQ== >> deploy-key
- echo -----END OPENSSH PRIVATE KEY----- >> deploy-key

This way the newlines in the file automatically get created, and now ssh-add pick up the format.