Why is `ssh` not working after adding public key to authorized hosts?

Question

I have a cluster set up with a master node and one slave node. I'd like to ssh to and from the master and slave nodes.

Now, I am able to ssh from both my local machine, and from my master node into the slave directly, using my private key: ssh -i ~/.ssh/id_rsa ec2-user@<IP>. I get asked for a password, and I enter it and can access the slave node from both my local machine, and my dev master.

Ideally, I want to remove my private id_rsa key from the ~/.ssh directory on dev master and only use my public key id_rsa.pub to ssh back and forth between master and slave nodes. (this is for security reasons, and plus it shouldn't be that difficult, right?).

So, I added my public key id_rsa.pub to authorized_keys on the slave node. Now, when I try to ssh from the master node to the slave it asks me for a password for the id_rsa.pub:

ssh -i ~/.ssh/id_rsa.pub ec2-user@<IP>
Enter passphrase for key '~/.ssh/id_rsa.pub'

And, I enter the same password I do for id_rsa but that password fails. :/ (I also tried just pressing enter (no password)) doesn't work either.

I have two questions to this:

• 1) Why isn't the password for id_rsa.pub the same as id_rsa?
• 2) Why can't I ssh into the slave node using my public key id_rsa.pub when it is added to the authorized_keys file?

When I try to ssh from my master to my slave, with the public key added to ~/.ssh/authorized_keys here is the verbose output:

debug2: ssh_connect: needpriv0
debug1: Connecting to <ip> [<ip>] port 22.
debug1: Connection established.
debug1: identity file /home/ec2-user/.ssh/identity type -1
debug3: Not a RSA1 key file /home/ec2-user/.ssh/id_rsa.
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug3: key_read: missing keytype
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug3: key_read: missing whitespace
debug2: key_type_from_name: unknown key type '-----END'
debug3: key_read: missing keytype
debug1: identity file /home/ec2-user/.ssh/id_rsa type 1
debug1: identity file /home/ec2-user/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug2: fd 3 setting O_NONBLOCK
debug1: SSH2_MSG_KEXINIT sent
debug3: Wrote 792 bytes for a total of 813
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: first_kex_follows 0 
debug2: kex_parse_kexinit: reserved 0 
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: first_kex_follows 0 
debug2: kex_parse_kexinit: reserved 0 
debug2: mac_setup: found hmac-md5
debug1: kex: server->client aes128-ctr hmac-md5 none
debug2: mac_setup: found hmac-md5
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug3: Wrote 24 bytes for a total of 837
debug2: dh_gen_key: priv key bits set: 138/256
debug2: bits set: 508/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug3: Wrote 144 bytes for a total of 981
debug3: check_host_in_hostfile: filename /home/ec2-user/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 1
debug1: Host '<ip>' is known and matches the RSA host key.
debug1: Found key in /home/ec2-user/.ssh/known_hosts:1
debug2: bits set: 521/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: Wrote 16 bytes for a total of 997
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug3: Wrote 48 bytes for a total of 1045
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /home/ec2-user/.ssh/identity ((nil))
debug2: key: /home/ec2-user/.ssh/id_rsa (0x7f2d4970bf40)
debug2: key: /home/ec2-user/.ssh/id_dsa ((nil))
debug3: Wrote 64 bytes for a total of 1109
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-keyex
debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-keyex
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug2: we did not send a packet, disable method
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug3: Trying to reverse map address <ip>.
debug1: Unspecified GSS failure.  Minor code may provide more information
Server krbtgt/US-WEST-2.COMPUTE.INTERNAL@host.COM not found in Kerberos database
debug2: we sent a gssapi-with-mic packet, wait for reply
debug3: Wrote 96 bytes for a total of 1205
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /home/ec2-user/.ssh/identity
debug3: no such identity: /home/ec2-user/.ssh/identity
debug1: Offering public key: /home/ec2-user/.ssh/id_rsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug3: Wrote 368 bytes for a total of 1573
debug1: Server accepts key: pkalg ssh-rsa blen 277
debug2: input_userauth_pk_ok: SHA1 fp df:e5:5b:18:67:05:8b:4d:06:6c:f9:99:c3:b5:2b:9f:4e:f2:52:8a
debug3: sign_and_send_pubkey
debug1: read PEM private key done: type RSA
debug3: Wrote 640 bytes for a total of 2213
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Trying private key: /home/ec2-user/.ssh/id_dsa
debug3: no such identity: /home/ec2-user/.ssh/id_dsa
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

I don't know why it won't let me ssh into the slave node, when my public key has been added as an authorized_keys.

Or, if there is an easier way to ssh between master and slave nodes, I'd be open to that as well. Thanks.

Answer 1

ssh -i ~/.ssh/id_rsa.pub ec2-user@<IP>

You should use private key here. Not the public one. SSH is wrongly asking passphrase for quite everything in few recent version, but it should be fixed now.

2) Why can't I ssh into the slave node using my public key id_rsa.pub when it is added to the authorized_keys file?

The keys are probably set up wrong. Run the client and server in debug mode (LogLevel DEBUG3 in ssh_confing and sshd_config) and have a look on the logs. If you will not be able to find out from that, update the question with the logs.

debug1: Server accepts key: pkalg ssh-rsa blen 277
debug2: input_userauth_pk_ok: SHA1 fp df:e5:5b:18:67:05:8b:4d:06:6c:f9:99:c3:b5:2b:9f:4e:f2:52:8a
debug3: sign_and_send_pubkey
debug1: read PEM private key done: type RSA
debug3: Wrote 640 bytes for a total of 2213
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic

The logs above says that the key was accepted, but it was not enough to grant you access to that server. More information will be visible in the server log.

Answer 2

Public and Private keys are lock key pairs. Your public key is not a key but a lock. You can give away that lock to anyone but never your private key. Your server has that lock. As stated in the comment below, you cannot open a lock with a lock. Using ssh -i privatekey ip@aws will work. Not ssh -i publickey ip@aws.