CODEWITHSUNDEEP
dockersshcontinuous-integrationcontinuous-deploymentgithub-actions
jns_ai_unr
jns_ai_unr

Reputation: 903

SSH "Host key verification failed" in GitHub Actions - but key exists in known_hosts

I have the weirdest error in GitHub Actions that I have been trying to resolve for multiple hours now and I am all out of ideas.

I currently use a very simple GitHub Action. The end goal is to run specific bash commands via ssh in other workflows.

Dockerfile:

FROM ubuntu:latest

COPY entrypoint.sh /entrypoint.sh

RUN apt update && apt install openssh-client -y
RUN chmod +x entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

entrypoint.sh:

#!/bin/sh

mkdir -p ~/.ssh/
echo "$1" > ~/.ssh/private.key
chmod 600 ~/.ssh/private.key
echo "$2" > ~/.ssh/known_hosts

echo "ssh-keygen"
ssh-keygen -y -e -f ~/.ssh/private.key
echo "ssh-keyscan"
ssh-keyscan <IP>

ssh -i ~/.ssh/private.key -tt <USER>@<IP> "echo test > testfile1"
echo "known hosts"
cat ~/.ssh/known_hosts
wc -m ~/.ssh/known_hosts

action.yml

name: "SSH Runner"
description: "Runs bash commands in remote server via SSH"
inputs:
  ssh_key:
    description: 'SSH Key'
  known_hosts:
    description: 'Known Hosts'
runs:
  using: 'docker'
  image: 'Dockerfile'
  args:
    - ${{ inputs.ssh_key }}
    - ${{ inputs.known_hosts }}

current workflow file in the same repo:

on: [push]

jobs:
  try-ssh-commands:
    runs-on: ubuntu-latest
    name: SSH MY_TEST
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: test_ssh
        uses: ./
        with:
          ssh_key: ${{secrets.SSH_PRIVATE_KEY}}
          known_hosts: ${{secrets.SSH_KNOWN_HOSTS}}

In the github action online console I get the following output:

ssh-keygen
---- BEGIN SSH2 PUBLIC KEY ----
Comment: "2048-bit RSA, converted by root@844d5e361d21 from OpenSSH"
AAAAB3NzaC1yc2EAAAADAQABAAABAQDaj/9Guq4M9V/jEdMWFrnUOzArj2AhneV3I97R6y
<...>
9f/7rCMTJwae65z5fTvfecjIaUEzpE3aen7fR5Umk4MS925/1amm0GKKSa2OOEQnWg2Enp
Od9V75pph54v0+cYfJcbab
---- END SSH2 PUBLIC KEY ----
ssh-keyscan
# <IP>:22 SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
# <IP>:22 SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
# <IP>:22 SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
# <IP>:22 SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
# <IP>:22 SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
<IP> ssh-ed25519 AAAAC3NzaC1lZD<...>9r5SNohBUitk
<IP> ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRNWiDWO65SKQnYZafcnkVhWKyxxi5r+/uUS2zgYdXvuZ9UIREw5sumR95kbNY1V90<...>
qWXryZYaMqMiWlTi6ffIC5ZoPcgGHjwJRXVmz+jdOmdx8eg2llYatRQbH7vGDYr4zSztXGM77G4o4pJsaMA/
***
Host key verification failed.
known hosts
***
175 /github/home/.ssh/known_hosts

As far as I understand *** is used to replace GitHub secrets which in my case is the key of the known host. Getting *** as a result for the ssh-keyscan and the cat known_host should mean, that the known_hosts file is correct and a connection should be possible. Because in both cases the console output is successfully censored by GitHub. And since the file contains 175 characters I can assume it contains the actual key. But as one can see the script fails with Host key verification failed.

When I do the same steps manually in another workflow with the exact same input data I succeed. Same goes for ssh from my local computer with the same private_key and known_host files.

This for example works with the exact same secrets

- name: Create SSH key
        run: |
          mkdir -p ~/.ssh/
          echo "$SSH_PRIVATE_KEY" > ../private.key
          sudo chmod 600 ../private.key
          echo "$SSH_KNOWN_HOSTS_PROD" > ~/.ssh/known_hosts
        shell: bash
        env:
          SSH_PRIVATE_KEY: ${{secrets.SSH_PRIVATE_KEY}}
          SSH_KNOWN_HOSTS: ${{secrets.SSH_KNOWN_HOSTS}}
- name: SSH into DO and run
        run: >
          ssh -i ../private.key -tt ${SSH_USERNAME}@${SERVER_IP}
          "
            < commands >
          "

Using the -o "StrictHostKeyChecking no" flag on the ssh command in the entrypoint.sh also works. But I would like to avoid this for security reasons.

I have been trying to solve this issue for hours, but I seem to miss a critical detail. Has someone encountered a similar issue or knows what I am doing wrong?

Upvotes: 16

Views: 21445

Answers (3)

Jannik
Jannik

Reputation: 466

For the GitHub runner in 2023, the .ssh dir doesn't exist, so you have to create it and known_hosts first:

mkdir -p ~/.ssh/ && touch ~/.ssh/known_hosts

Then add the public SSH host key:

ssh-keyscan [hostname] >> ~/.ssh/known_hosts

And the private key:

eval $(ssh-agent)
ssh-add - <<< "${{ secrets.SSH_PRIVATE_KEY }}"

Upvotes: 12

jns_ai_unr
jns_ai_unr

Reputation: 903

So after hours of searching I found out what the issue was. When force accepting all host keys with the -o "StrictHostKeyChecking no" option no ~/.ssh/known_hosts file is created. Meaning that the openssh-client I installed in the container does not seem to read from that file. So telling the ssh command where to look for the file solved the issue:

ssh -i ~/.ssh/private.key -o UserKnownHostsFile=/github/home/.ssh/known_hosts -tt <USER>@<IP> "echo test > testfile1"

Apparently one can also change the location of the known_hosts file within the ssh_config permanently (see here).

Hope this helps someone at some point.

Upvotes: 12

VonC
VonC

Reputation: 1323803

First, add a chmod 600 ~/.ssh/known_hosts as well in your entrypoint.

For testing, I would check if options around ssh-keyscan make any difference:

ssh-keyscan -H <IP>
# or
ssh-keyscan -t rsa -H <IP>

Check that your key is generated using the default rsa public-key cryptosystems.
The HostKeyAlgorithms used might be set differently, in which case:

ssh-keyscan -H -t ecdsa-sha2-nistp256 <IP>

Upvotes: 1

Related Questions