Docker Load key "/root/.ssh/id_rsa": invalid format

Question

I am trying to clone a repo that has submodules in it. The main repo is cloning fine but when I do git submodule update --init --recursive in the dockerfile the submodules throws and error.

fatal: clone of 'git@github.com:jkeys089/lua-resty-hmac.git' into submodule path '/tmp/third-party/lua-resty-hmac' failed
Failed to clone 'third-party/lua-resty-hmac'. Retry scheduled
Cloning into '/tmp/third-party/lua-resty-jwt'...
load pubkey "/root/.ssh/id_rsa": invalid format
Warning: Permanently added the RSA host key for IP address '140.82.118.3' to the list of known hosts.
Load key "/root/.ssh/id_rsa": invalid format
git@github.com: Permission denied (publickey).

In the image I have this

# authorise ssh host
RUN mkdir /root/.ssh/ \
    && chmod 700 /root/.ssh \
    && ssh-keyscan github.com > /root/.ssh/known_hosts

# add key and set permission
RUN echo "${SSH_PRIVATE_KEY}" >> /root/.ssh/id_rsa \
    && echo "$ssh_pub_key" > /root/.ssh/id_rsa.pub \
    && chmod 600 /root/.ssh/id_rsa.pub \
    && chmod 600 /root/.ssh/id_rsa

I have no control of the submodules. I am not sure if I can change from git@github.comto https to get submodules.

I even tried using the GITHUB_TOKEN route

# start up git and clone
RUN git config --global url."https://${GITHUB_TOKEN}@github.com/".insteadOf "https://github.com/" \
    && git clone https://github.com/GluuFederation/gluu-gateway.git /tmp \
    && cd /tmp/ \
    && git submodule update --init --recursive

And below is the part of the build command. build --build-arg GITHUB_TOKEN=${GITHUB_TOKEN} --build-arg SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)" --build-arg ssh_pub_key="$(cat ~/.ssh/id_rsa.pub)"

Please help out on this. It's very frustrating. :(

Answer 1

In most cases, you should not add any ssh key in your container due to security risks since any person can get the ssh key if they have access to your image.

Similarly, using ARG is not recommended by Docker either, as you can retrieve the secrets using docker history <image>. The secret is embedded in the layer on the build. You can find more details here https://docs.docker.com/engine/reference/builder/#arg

The recommended approach is to use CI/CD to clone the GitHub repo to the local folder, then use COPY operation to copy the code into your image. But if you absolutely want to clone your GitHub repo, there are two approaches:

1. Mount secret into a volume during build https://render.com/docs/docker-secrets. Please note that the secret mount has root permission, therefore you will get a permission error if you read the secrets as non-root users.
2. Using multi-stage build. You can have gitsync image in the first stage, clone the repo, then copy the content of the repo to your image in the second stage. https://docs.docker.com/build/building/multi-stage/. This way you can load secret into the first stage, clone the repo, then copy the artifact to the later stage. The final stage image will not have any trace of your secrets if you do it correctly.

An example of cloning a private library and install it in the final image:

# Clone python library from a private repo, then copy to the python image and install as library
# docker build -t <img_tag> --build-arg BOT_GITHUB_USER=$BOT_GITHUB_USER --build-arg BOT_GITHUB_PRIVATE_TOKEN=$BOT_GITHUB_PRIVATE_TOKEN  .
# multi stage build, clone lib repo then copy to main image
FROM k8s.gcr.io/git-sync/git-sync:v3.6.1 as gitsync
ARG BOT_GITHUB_USER
ARG BOT_GITHUB_PRIVATE_TOKEN

ENV GIT_SYNC_USERNAME=$BOT_GITHUB_USER
ENV GIT_SYNC_PASSWORD=$BOT_GITHUB_PRIVATE_TOKEN
ENV GIT_SYNC_BRANCH=dev
RUN /git-sync --repo=https://github.com/yourrepo/mylib --one-time


FROM python

RUN mkdir mylib

COPY --from=gitsync /tmp/git/mylib mylib/

RUN pip install -e mylib/

Answer 2

In my case it was because of missing trailing empty line.

Answer 3

Make sure there is a \n after the last line. Took me too long to figure this out.

Answer 4

This answer is for Windows users (haven't tried this on linux).

I searched many answers and articles but still was getting invalid format error while building my docker image.

The actual reason is that when we pass the content of the private key file as an argument, they are passed in a single line. The escape characters are converted to space which is basically invalid format for the key. To avoid this error, there are two ways of passing private key to the docker image:

1. Use COPY command in the docker file to copy the private key file and use it in the docker image. This is not considered as a good option as it may expose your private key. Example: COPY id_rsa /root/.ssh/id_rsa
2. This is a kind of hack which I used and it worked. Open the private key file in a text editor and add \n at the end of every line in the private key and join every line to create the whole key in one line. For example, your generated key looks like this:

-----BEGIN OPENSSH PRIVATE KEY----- b3BlbnNzaC1redjEAAAAABG5vvmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW QyNTUxOQAAACdmF7/Vo4m2FWPf+8uZRRF88dnsyj+z+lCWNWBrT8gAAAJh1tssodbbL -----END OPENSSH PRIVATE KEY-----

make it look like this:

-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1redjEAAAAABG5vvmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW\nQyNTUxOQAAACdmF7/Vo4m2FWPf+8uZRRF88dnsyj+z+lCWNWBrT8gAAAJh1tssodbbL\n-----END OPENSSH PRIVATE KEY-----\n

Save it in the key file and pass the modified file in the arguments while building docker image.

Answer 5

For reference only.

I created a private key file key.id_rsa manaully and pasted content to it. But when I used it to clone git repository:

$ git clone my_repo
Cloning into 'my_repo'...
Load key "/path/to/key.id_rsa": invalid format

The key's content is definitely correct. So then I tried compare my key with the other generated key by ssh-keygen, it's truely invalid format.

Just no end of new line in my key.

After added new line to the end of key, all worked delightful~ What a suprise!

[Update]: Remember use \n instead of \r\n even on windows.

Answer 6

Do not use echo "${SSH_PRIVATE_KEY}" >> /root/.ssh/id_rsa to pass the private key (same for the public key). I had a similar error Load key "/root/.ssh/id_rsa": invalid format when I tried

RUN echo "$ssh_prv_key" > /root/.ssh/id_rsa &&     chmod 600 /root/.ssh/id_rsa

This led to errors like identity file /root/.ssh/id_rsa type -1 invalid format and read_passphrase: can't open /dev/tty.

The right way would be to use

COPY id_rsa /root/.ssh/id_rsa
RUN chmod 600 /root/.ssh/id_rsa

The solution explained: my private key was wrongly formatted - instead of many lines, it was passed as a one-liner, and you might have any other format issue like a forgotten "-" at the start or end, or something wrong at the end of the lines, like a missing newline format or an additional letter at the end of a line.

See Dockerfile: clone repo with passwordless private key. Errors: “authentication agent” or “read_passphrase: can't open /dev/tty” for more details, with the main idea from Add private key to ssh-agent in docker file, which again had the idea from Gitlab CI/Docker: ssh-add keeps asking for passphrase.

Answer 7

Another possible gotcha is if you're using a Makefile to run the docker build command. In that case the command in the Makefile would look something like:

docker-build:
    docker build --build-arg SSH_PRIVATE_KEY="$(shell cat ~/.ssh/id_rsa)"

Make unfortunately replaces newlines with spaces (make shell)

This means that the ssh key which is written into the container has a different format, yielding the error above.

I was unable to find a way to retain the newlines in the Makefile command, so I resorted to a workaround of copying the .ssh directory into the docker build context, copying the files through the Dockerfile, then removing them afterwards.

Answer 8

If the key is "invalid format", try and regenerate it with the old PEM format.

ssh-keygen -m PEM -t rsa -P "" 

Make sure to add the public key to your GitHub account for proper authentication.

The OP Shammir adds in the comments:

I think the issue is that nothing is being copied from host machine to docker image during build.

In "docker build --build-arg SSH_PRIVATE_KEY="$(cat ~/.ssh/id_rsa)" returning empty", Shammir uses dockito/vault to manage the private key, but also configure it to "AddKeysToAgent": that is not needed if the private key is not passphrase protected (as in my command above)