Docker build dockerfile with SSH keys used during build process (for private Git repos)

Question

Docker-Compose v3

I'm struggling to install simple GitHub repositories that are made private by my company. We're working on moving our projects over to Docker. So, we are currently using Docker and Docker-Compose in order to build and run our containers.

But 90% of our projects use private repos, custom Gems, and we'll eventually be using private containers.

I can't seem to install my repos despite having moved my SSH keys onto the container via Habitus, manually copying them over during the build process, and attempting to mount volumes. But nothing I do seems to work.

eg. DockerFile

FROM ubuntu 

# Installing tons of libraries. Not all of these might be needed
RUN apt-get -y update && apt-get -y upgrade && apt-get -y install nodejs npm ruby curl openssh-server git php
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

# pulling id_rsa from local habitus network (this SUCCEEDS)
ARG host
RUN wget -O ~/.ssh/id_rsa http://$host:8080/v1/secrets/file/id_rsa && chmod 600 ~/.ssh/id_rsa

# checking the ssh files are installed (this FAILS)
RUN ssh -vvv -T git@git.my.company.com 

RUN gem install bundler
RUN bundle install

Command via Habitus

sudo habitus --build host=192.168.99.100 --secrets=true 

Output

OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g  1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
Pseudo-terminal will not be allocated because stdin is not a terminal.
debug1: Connecting to 192.168.99.100 [192.168.99.100] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.2 pat OpenSSH* compat 0x04000000
debug1: Authenticating to git.my.company.com:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:OkseSPnItLVT0phkACs7TwGA1CZb9nMBSwp5UxdkIf4
debug1: Host 'git.my.company.com' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /root/.ssh/id_rsa
debug1: Server accepts key: pkalg rsa-sha2-512 blen 535
debug1: read_passphrase: can't open /dev/tty: No such device or address
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: Trying private key: /root/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey).

Now, I've tried many other methods. With every single method, I've had the same issue. I'll sometimes put my build to sleep for a few minutes while I SSH into the server and check my ~/.ssh/ folder. Everytime, all of the files match the one's on my local which does successfully bundle install the Gems. So, why is it telling me "can't open /dev/tty: ..."

It's looking for a passphrase. Is this because the docker container runs as "root"? Can I bypass this?

How do you guys add your SSH keys to the DockerFile during the build process so you can run commands like "Bundle install" or "npm install"? I've been scouring the internet for days trying all the solutions I find, but I always end up with the same response from the SSH attempts.

Answer 1

Try OnVault, Secret store to be used on Docker image building. It is capable of binding ssh keys at runtime, exactly at the required build step and removes it immediately after completion (before committing the layer to the image). This leaves no trace of ssh keys in docker image, which enhances security.

Some Advantages:

• Needs for addition and deletion of ssh keys are eliminated
• Reduces complexities with dockerfile
• As it binds the entire .ssh Folder, configs for multiple GitHub accounts in your local, works during the Build steps as well.
• It is a binary file and can be used with keyword ONVAULT. eg: RUN ONVAULT bundle install

Note: Its works for me and it is awsome..!!, give it a try ;-)

Answer 2

Just as a heads up to anyone else that continually runs into Mac OS X causing this error on their SSH attempts:

debug1: read_passphrase: can't open /dev/tty: No such device or address
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: Trying private key: /root/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey).

You have to include a special ENV variable before you make any SSH calls. Otherwise it'll always look for the password. I believe this is some odd default of running SSH commands as root.

ENV DEBIAN_FRONTEND noninteractive

At the end of your commands, you must run:

RUN unset DEBIAN_FRONTEND 

Here is my sample docker file:

FROM ubuntu

RUN apt-get -y update && apt-get -y upgrade && apt-get -y install nodejs-legacy npm ruby curl openssh-server git php
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

#### THE SETTING OF DEBIAN noninteractive ####
ENV DEBIAN_FRONTEND noninteractive

RUN gem install bundler
RUN mkdir /website
WORKDIR /website

# I made this temporary SSH key folder in the relative path of my Dockerfile because that is a requirement of DockerFile. However, I plan on using Habitus to safely and conveniently wget my keys from a local server

RUN mkdir /root/.ssh 
COPY dockerkeys /root/.ssh

RUN ssh -v -T git@git.my.company.com

COPY . .

RUN bundle install
RUN composer install
RUN npm install

#### UNSETTING DEBIAN ENV ####
RUN unset DEBIAN_FRONTEND 

RUN bundle install
RUN composer install
RUN npm install

CMD [ "npm", "start" ]

I haven't tested it with other images, but the principle remains the same. Looks like you need to explicitly set a noninteractive shell. Otherwise, it will prompt you for the password for your SSH key everytime.

Answer 3

Add identity to the ssh agent on every line you want to invoke ssh.

RUN eval $(ssh-agent);ssh-add;ssh -vvv -T git@git.my.company.com 

Answer 4

For npm private modules, you can check the official documentation. Docker and private modules

As for the ruby gems, you can find a similar solution here.