Mohammad Reza Mousavi
Mohammad Reza Mousavi

Reputation: 1444

How can I fix "kex_exchange_identification: read: Connection reset by peer"?

I want to copy data with scp in a GitLab pipeline using PRIVATE_KEY.

The error is:

kex_exchange_identification: read: Connection reset by peer
Connection reset by x.x.x.x port 22
lost connection

Pipeline log:

$ mkdir -p ~/.ssh

$ echo "$SSH_PRIVATE_KEY" | tr -d '\r' > ~/.ssh/id_rsa

$ chmod 600 ~/.ssh/id_rsa

$ eval "$(ssh-agent -s)"
Agent pid 22

$ ssh-add ~/.ssh/id_rsa
Identity added: /root/.ssh/id_rsa (/root/.ssh/id_rsa)

$ ssh-keyscan -H $IP >> ~/.ssh/known_hosts
# x.x.x.x:22 SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.10
# x.x.x.x:22 SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.10

$ scp -rv api.yml root@$IP:/home/services/test/

Executing: program /usr/bin/ssh host x.x.x.x, user root, command scp -v -r -t /home/services/test/

OpenSSH_8.6p1, OpenSSL 1.1.1l  24 Aug 2021
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to x.x.x.x [x.x.x.x] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
kex_exchange_identification: read: Connection reset by peer
Connection reset by x.x.x.x port 22
lost connection

Upvotes: 126

Views: 549047

Answers (23)

Joe Bloggs
Joe Bloggs

Reputation: 1

I had the same problem, and the cause (and fix), was simple.

I had an "ssh" connection (to some server), open in one terminal window (which I'd forgotton about).

While trying to "scp" to the same server in another terminal window (which generated the error).

Closing the "ssh" connection fixed the problem.


Presumably, you can do ONLY ONE "ssh"/"scp" type thing, to any given server, at any given time. Which seems quite reasonable to me.

Upvotes: -2

KSroido
KSroido

Reputation: 75

for whom receive this when using vs code check proxy settings in your Windows setting -> proxy vs code defaultly using that as itz proxy settings which might lead to protential problems.

Upvotes: 0

Naoki Ogawa
Naoki Ogawa

Reputation: 31

I suggest to check the routing table for one possibility. In my case on Ubuntu 20.04 (Focal Fossa), I added a local network routing entry to recover when I got the same error message when connecting to the server using SSH. It had disappeared unexpectedly, leaving only the default route.

route -n

Output:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 enp1s0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 enp1s0  # <= disappeared

It seemed as if ack was being filtered by an incomplete routing table although the first syn passed.

Upvotes: 3

PHZ.fi-Pharazon
PHZ.fi-Pharazon

Reputation: 2093

I was connecting from Ubuntu 22.04 to old CentOS7 host. However, if I used an old server with Ubuntu 18.04 to connect to the same CentOS -server, it worked fine. This is probably related to SSH client and server versions. You might want to try to debug it by:

ssh -vvvv user@server

Upvotes: 1

dlyng
dlyng

Reputation: 31

This issue will also appear after updating to openssh9.81p1 (on Arch Linux systems at least, perhaps others). In that case, you MUST restart sshd in the same session where the upgrade took place or you will be locked out. On vanilla installs with systemd, you can run sudo systemctl restart sshd.

See: https://gitlab.archlinux.org/archlinux/packaging/packages/openssh/-/issues/5.

Upvotes: 1

lugr
lugr

Reputation: 29

I had this problem when connecting with ssh via a forwarded port:

kex_exchange_identification: read: Connection reset

My setup: openssh server running on a RedHat 8.6 machine, connecting from a Windows 10 client. I use a Putty Terminal to forward port 22 from the server to some free local port (say 5001) on my windows client. The direct ssh connection works. But then I got the error when trying to connect with ssh to the forwarded localhost:5001

On the server, I saw in the logs (/var/log/secure):

refused local port forward: originator 0.0.0.0 port 0, target localhost port 22

FIX:

I was able to fix it by setting the following property in /etc/ssh/sshdconfig:

AllowTcpForwarding yes

Followed by a test and then restart of sshd using:

sudo sshd -t # test

sudo systemctl restart sshd

Upvotes: 2

ruoyi qiao
ruoyi qiao

Reputation: 1

Maybe useful for those using proxy like myself:

I encountered the same issue when uploading files in the VSCode terminal. However, executing the same command in the terminal of the local machine fixed this issue.

According to Google, when directly accessing HTTP/HTTPS services using tools like VSCode for port forwarding within the development machine, proxy issues may arise.

Upvotes: 0

Ali Samie
Ali Samie

Reputation: 470

I had to change my DNS, I used shecan.ir If you are from Iran, you most probably have the same issue, particularly when using gitlab.com

Upvotes: 2

John White
John White

Reputation: 977

In my case it was with a Synology server, the private IP accidentally went into the DSM's block list. Removing it and putting the whole subnet on the allow list fixed the issue.

Upvotes: 1

Marquess
Marquess

Reputation: 31

On CentOS 7.9.2009 create empty dir

mkdir /var/empty/sshd
chmod 0711 /var/empty/sshd

This helped to me

Upvotes: 1

Shiva Gyawali
Shiva Gyawali

Reputation: 61

I was having the same issue and resolved by:

  1. Check if sshd is running on both machine.

  2. Check if there exists any firewall rules which is not letting you succeed.

    iptables -L

  3. check packets received while you are doing ssh using tcpdump on server:

    tcpdump -i any dst destination_IP and src Source_IP and dst port 22

output will show only 3 packets are being shared and the server closed connection.

  1. check if hosts.allow is allowing only some host to make sshd connection and denying all other.

    cat /etc/hosts.allow

output may include: sshd: IP-range subnet_range

Also, check if hosts.deny is denying some ip range to make sshd connection:

cat /etc/hosts.deny

Output may include: sshd: ALL

If point 4 is your cause, then add your machine IP (from where you are trying to make ssh connection) as a hosts allowed on server machine. Follow these steps, then, you are good to go.

Upvotes: 5

SystemDee
SystemDee

Reputation: 61

I had the same error. I have an Ubuntu 20.04 host and two RHEL 8 VMs running on VMware. I log in into the two VMs from my Ubuntu terminal. I use Ethernet and Wi-Fi connections. Every time I tried to log into a VM after rebooting it, I would get the error:

kex_exchange_identification: read: Connection reset by peer
Connection reset by x.x.x.x port 22

Restarting the sshd service would not solve the problem. Sometimes, the problem would be resolved if I physically disconnected and reconnected the Ethernet cable.

Finally I turned off my Wi-Fi connection with:

nmcli conn down <name_of_Wi-Fi_connection>

Or turning it off from settings and this gave me a permanent solution.

Both my Ethernet and Wi-Fi connections (static connections) had the same IP address, so I think the VMs were rejecting two "suspicious" similar connections.

Upvotes: 0

zzy
zzy

Reputation: 39

Maybe you should change the resolution of host

Press win+R, enter C:\Windows\System32\drivers\etc to open the location of the host file: enter image description here

add [IP] : Domain name for example 1.1.1.1 google

Upvotes: -1

ijazahmed Bagawan
ijazahmed Bagawan

Reputation: 101

I was facing the same issue whenever I was trying to connect to my Amazon Linux 2 instance through SSM or terminal:

Why did it show me the error: The /etc/hosts.deny file specifies which IP addresses are not permitted to connect to the host. Where as I had added "ALL: ALL" to /etc/hosts.deny and it will deny all IP address to not connect to the instance (server).

Solution: Simply you can stop the instance and edit the USER DATA and run the command

sed -i 's/ALL: ALL/#ALL: ALL/g' /etc/hosts.deny

This will comment hosts.deny. Once this is done, start the instance (server).

Upvotes: 2

aborde
aborde

Reputation: 1

I had this error today when I was trying to use my Dell laptop running Ubuntu 20.04.5 LTS (Focal Fossa) and trying to SSH into a Raspberry Pi. When I was on my home Wi-Fi network and tried to SSH into the pi (also on my home Wi-Fi network) I got the error:

ssh [email protected]

Output:

kex_exchange_identification: read: Connection reset by peer'

However, when I switched my Ubuntu Laptop over to a mobile hotspot, the error disappeared, and I was able to SSH without issue. Will update this post as soon as I figure out how to resolve the root cause.


Issue resolved (but full reason unclear). I followed the instructions to change my DNS server here to 8.8.8.8 and 8.8.4.4.

After about 5 minutes had elapsed, I was able to use SSH from my command line terminal just fine.

Upvotes: 0

Frotz
Frotz

Reputation: 592

For those who came across this page after upgrading a FreeBSD machine to 13.1 and then trying to ssh into it, see Bug 263489. sshd does not work after reboot to 13.1-RC4.

After the upgrade, the previous sshd daemon (OpenSSH < 8.2) is still running with new configurations (OpenSSH >= 8.2). The solution is to stop and then restart the sshd daemon. The FreeBSD 13.1 release notes now mention this and after 13.1, the freebsd-update script will automatically restart the daemon.

Upvotes: 1

zkl
zkl

Reputation: 141

TL;DR:
find the server-side process listen-to-ssh port and kill it, and then restart the ssh service. It should solve this problem.

On the client side:

ssh account@ip -pPORT

kex_exchange_identification: read: Connection reset by peer

I tried it on the server side:

service ssh status

[ ok ] sshd is running.

service ssh restart

[ ok ] Restarting OpenBSD Secure Shell server: sshd.

but the client side ssh command still fail with the same kex_exchange_identification error.

Then I stop the ssh service on the server side (as root):

service ssh stop

[ ok ] Stopping OpenBSD Secure Shell server: sshd.

And the following client side ssh command still fails with the same kex_exchange_identification error. It's strange; if no process listen the port, it should be the error Connection refused.

It could be the process on the server side listening to the SSH port is dead, and even a restart / stop service do not work. So to find the process, and killing it may solve the problem.

The PORT here is the SSH port defined in 'server /etc/ssh/sshd_config', and the default is 22. As root:

netstat -ap | grep PORT

tcp 0 0 0.0.0.0:PORT 0.0.0.0:* LISTEN 8359/sshd
tcp6 0 0 [::]:PORT [::]:* LISTEN 8359/sshd

kill 8359
netstat -ap | grep PORT

no result

service ssh start

[ ok ] Starting OpenBSD Secure Shell server: sshd.

netstat -ap | grep PORT

tcp 0 0 0.0.0.0:PORT 0.0.0.0:* LISTEN 31418/sshd: /usr/sb
tcp6 0 0 [::]:PORT [::]:* LISTEN 31418/sshd: /usr/sb

The following client-side ssh command succeed.

Upvotes: 14

rockettc
rockettc

Reputation: 1321

Similar to naoki-ogawa, I had a problem with my routing table. In my case, I had an extra route for my local network.

As root:

route

Output:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         RT-AX92U-3E20   0.0.0.0         UG    100    0        0 eno1
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 virbr1
192.168.50.0    RT-AX92U-3E20   255.255.255.0   UG    10     0        0 eno1
192.168.50.0    0.0.0.0         255.255.255.0   U     100    0        0 eno1

I simply removed the gateway on the local network (192.168.50.0):

route del 192.168.50.0/24 via 192.168.50.1

The problem was resolved.

Upvotes: 1

Jian Zhang
Jian Zhang

Reputation: 87

I met this issue after I changed my Apple ID password, so I updated my Apple ID and restarted my Mac. It works now.

git pull origin master

Output:

kex_exchange_identification: read: Connection reset by peer
Connection reset by 20.205.243.166 port 22
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Upvotes: 5

shreyasi samanta
shreyasi samanta

Reputation: 23

Try to check if OpenSSH server is up and running on the server side.

Try checking the sshd configuration. It worked this way for me.

Upvotes: -2

Kenster
Kenster

Reputation: 25439

kex_exchange_identification: read: Connection reset by peer

When an SSH client connects to an SSH server, the server starts by sending a version string to the client. The error that you're getting means that the TCP connection from the client to the server was "abnormally closed" while the client was waiting for this data from the server, in other words immediately after the TCP connection was opened.

As a practical matter, it's likely to mean one of two things:

  1. The SSH server process malfunctioned (crashed), or perhaps it detected some serious issue causing it to exit immediately.
  2. Some firewall is interfering with connections to the ssh server.

It looks like the ssh-keyscan program was able to connect to the server and get a version string without an error. So the SSH server process is apparently able to talk to a client without crashing.

You should talk the administrators of this x.x.x.x host and the network that it's attached to, to see if they can identify the problem from their end. It's possible that something—a firewall, or the ssh server process itself—is seeing the multiple connections, first from the ssh-keyscan process, then by the scp program, as an intrusion attempt. And it's blocking the second connection attempt.

Upvotes: 94

Harsh Wardhan Gupta
Harsh Wardhan Gupta

Reputation: 393

The same issue with me:

I have fixed the issue by doing the below steps.

  1. edit file etc/hosts.allow. Command to do so sudo nano /etc/hosts.allow.
  2. At the end, update the value of ALL keys to ALL like ALL : ALL. Save the file and try again.

Basically, ALL might be set to something else therefore while establishing ssh connection to the host, it is expecting that the request should come from the IP address starting from 10...* if ALL set to ALL : 10.. Therefore by replacing 10. with ALL, you are allowing connection from everywhere.

Upvotes: -3

Mehmet Avni CELIK
Mehmet Avni CELIK

Reputation: 582

I had the same problem. I rebooted the server, then it was all good.

Upvotes: 40

Related Questions