Reputation: 1666

Unable to fix SSH Permission denied (publickey) error

I keep getting this error whenever I try to access a server via ssh using VSCode's Remote Explorer or command line terminal.

I already checked this question, but was unable to solve the issue.

I have:

made sure that the .ssh folder and files belong to the same user using chmod and whoami
made sure that the ssh folder is 700 and key file is 600 by using chmod and stat -f "%Lp" filename
made sure the config and key files are under Users/[username]/.ssh
tried setting IdentitiesOnly and PubKeyAuthentication to "yes"
made sure the private keys file is local and the public keysfile is on the server

Is there anything else I could be missing? Any chances I have to change something in the server side?

I am using a Mac laptop, and it is the first time setting up any remote access on this specific laptop.
I am trying to access a GCP VM instance (that also contains the ssh key).

this is the output of ssh -i ~/.ssh/private_keys_file -p port username@hostname -vv

OpenSSH_9.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/name/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug2: resolve_canonicalize: hostname hostname is address
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to hostname [hostname] port portnumber.
debug1: Connection established.
debug1: identity file /Users/name/.ssh/private_keys_file type 0
debug1: identity file /Users/name/.ssh/private_keys_file-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.9p1 Ubuntu-3ubuntu0.6
debug1: compat_banner: match: OpenSSH_8.9p1 Ubuntu-3ubuntu0.6 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to hostname:portnumber as 'username'
debug1: load_hostkeys: fopen /Users/name/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: [email protected],curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,[email protected]
debug2: host key algorithms: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected],zlib
debug2: compression stoc: none,[email protected],zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,[email protected],diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,[email protected]
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected]
debug2: compression stoc: none,[email protected]
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:some_different_key?
debug1: load_hostkeys: fopen /Users/name/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '[hostname]:portnumber' is known and matches the ED25519 host key.
debug1: Found key in /Users/name/.ssh/known_hosts:3
debug1: ssh_packet_send2_wrapped: resetting send seqnr 3
debug2: ssh_set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: ssh_packet_read_poll2: resetting read seqnr 3
debug1: SSH2_MSG_NEWKEYS received
debug2: ssh_set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_ext_info_client_parse: server-sig-algs=<ssh-ed25519,[email protected],ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected]>
debug1: kex_ext_info_check_ver: [email protected]=<0>
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: ssh_fetch_identitylist: agent contains no identities
debug1: Will attempt key: /Users/name/.ssh/private_keys_file RSA SHA256:CJ9uoUi5OhdyGtue7jMFySsiVUXp0y6dlJ4YFycS17s explicit
debug2: pubkey_prepare: done
debug1: Offering public key: /Users/name/.ssh/private_keys_file RSA SHA256:CJ9uoUi5OhdyGtue7jMFySsiVUXp0y6dlJ4YFycS17s explicit
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
username@hostname: Permission denied (publickey).

This is the output when I try to use the remote explorer:

Host my_project_name
    HostName hostname
    User username
    Port portnumber
    IdentityFile ~/.ssh/my_private_key_file
    IdentitiesOnly yes
    PubKeyAuthentication yes

[18:49:53.928] Log Level: 2
[18:49:53.932] SSH Resolver called for "ssh-remote+my-project-name", attempt 1
[18:49:53.934] "remote.SSH.useLocalServer": true
[18:49:53.934] "remote.SSH.useExecServer": true
[18:49:53.934] "remote.SSH.path": undefined
[18:49:53.934] "remote.SSH.configFile": undefined
[18:49:53.934] "remote.SSH.useFlock": true
[18:49:53.934] "remote.SSH.lockfilesInTmp": false
[18:49:53.934] "remote.SSH.localServerDownload": auto
[18:49:53.934] "remote.SSH.remoteServerListenOnSocket": false
[18:49:53.934] "remote.SSH.showLoginTerminal": false
[18:49:53.934] "remote.SSH.defaultExtensions": []
[18:49:53.934] "remote.SSH.loglevel": 2
[18:49:53.935] "remote.SSH.enableDynamicForwarding": true
[18:49:53.935] "remote.SSH.enableRemoteCommand": false
[18:49:53.935] "remote.SSH.serverPickPortsFromRange": {}
[18:49:53.935] "remote.SSH.serverInstallPath": {}
[18:49:53.938] VS Code version: 1.90.2
[18:49:53.938] Remote-SSH version: [email protected]
[18:49:53.938] darwin arm64
[18:49:53.939] SSH Resolver called for host: my-project-name
[18:49:53.939] Setting up SSH remote "my-project-name"
[18:49:53.941] Acquiring local install lock: /var/folders/00/74r40yj568lgg9nq5q3g_3sh0000gp/T/vscode-remote-ssh-4932b980-install.lock
[18:49:53.941] Looking for existing server data file at /Users/name/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-4932b980-5437499feb04f7a586f677b155b039bc2b3669eb-0.112.0-es/data.json
[18:49:53.941] Using commit id "5437499feb04f7a586f677b155b039bc2b3669eb" and quality "stable" for server
[18:49:53.943] Install and start server if needed
[18:49:53.947] PATH: /Users/name/.nvm/versions/node/v20.11.1/bin:/Users/name/.npm-packages/bin:/Users/name/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[18:49:53.947] Checking ssh with "ssh -V"
[18:49:53.970] > OpenSSH_9.6p1, LibreSSL 3.3.6

[18:49:53.971] askpass server listening on /var/folders/00/74r40yj568lgg9nq5q3g_3sh0000gp/T/vscode-ssh-askpass-685034522e84a74c430df959bf8a94ccc8cc51b0.sock
[18:49:53.972] Spawning local server with {"serverId":1,"ipcHandlePath":"/var/folders/00/74r40yj568lgg9nq5q3g_3sh0000gp/T/vscode-ssh-askpass-4ddffc100b702667efcf8eceaedbc57fed8a3b5a.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","52070","-o","ConnectTimeout=15","my-project-name"],"serverDataFolderName":".vscode-server","dataFilePath":"/Users/name/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-4932b980-5437499feb04f7a586f677b155b039bc2b3669eb-0.112.0-es/data.json"}
[18:49:53.972] Local server env: {"SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.5FLI42FeEC/Listeners","SHELL":"/bin/zsh","DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin)","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"","VSCODE_SSH_ASKPASS_MAIN":"/Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/00/74r40yj568lgg9nq5q3g_3sh0000gp/T/vscode-ssh-askpass-685034522e84a74c430df959bf8a94ccc8cc51b0.sock"}
[18:49:53.972] Spawned 73935
[18:49:54.064] > local-server-1> Running ssh connection command: ssh -v -T -D 52070 -o ConnectTimeout=15 my-project-name
[18:49:54.066] > local-server-1> Spawned ssh, pid=73944
[18:49:54.077] stderr> OpenSSH_9.6p1, LibreSSL 3.3.6
[18:49:54.168] stderr> debug1: Server host key: ssh-ed25519 SHA256:some_different_key?
[18:49:54.298] stderr> username@hostname: Permission denied (publickey).
[18:49:54.299] > local-server-1> ssh child died, shutting down
[18:49:54.306] Local server exit: 0
[18:49:54.306] Received install output: local-server-1> Running ssh connection command: ssh -v -T -D 52070 -o ConnectTimeout=15 my-project-name
local-server-1> Spawned ssh, pid=73944
OpenSSH_9.6p1, LibreSSL 3.3.6
debug1: Server host key: ssh-ed25519 SHA256:some_different_ssh_key?
username@hostname: Permission denied (publickey).
local-server-1> ssh child died, shutting down

[18:49:54.309] Resolver error: Error: Permission denied (publickey).
    at g.Create (/Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:499918)
    at /Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:496392
    at t.handleInstallOutput (/Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:497162)
    at e (/Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:558422)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:580377
    at async t.withShowDetailsEvent (/Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:584036)
    at async /Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:555127
    at async T (/Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:553178)
    at async t.resolveWithLocalServer (/Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:554667)
    at async k (/Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:577548)
    at async t.resolve (/Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:581407)
    at async /Users/name/.vscode/extensions/ms-vscode-remote.remote-ssh-0.112.0/out/extension.js:2:848023
[18:49:54.313] ------

this is the output of namei -mo $(realpath ~/.ssh/authorized_keys ) on the server side:

 drwxr-xr-x root    root    /
 drwxr-xr-x root    root    home
 drwxr-x--- username username username
 drwx------ username username .ssh
 -rw------- username username authorized_keys

Running journalctl -u ssh -ef on the server printed No journal files were found..

This was the output for running tail -f /var/log/auth.log:

Jul  2 09:37:00 cs-1091963627082-default sshd[346]: pam_unix(sshd:session): session opened for user username(uid=1000) by (uid=0)
Jul  2 09:37:00 cs-1091963627082-default sshd[346]: pam_systemd(sshd:session): Failed to connect to system bus: No such file or directory
Jul  2 09:37:00 cs-1091963627082-default sshd[367]: reprocess config line 45: Deprecated option RSAAuthentication
Jul  2 09:37:00 cs-1091963627082-default sshd[367]: reprocess config line 52: Deprecated option RhostsRSAAuthentication
Jul  2 09:37:01 cs-1091963627082-default sudo:  username : TTY=pts/0 ; PWD=/home/username ; USER=root ; COMMAND=/usr/bin/touch /var/run/google/devshell/38357
Jul  2 09:37:01 cs-1091963627082-default sshd[367]: Accepted publickey for username from 127.0.0.1 port 44678 ssh2: ECDSA SHA256:Og9M7KmC/Jk0ccfAdxaOwWgUNMkGuZxvvkCQXMCQsto
Jul  2 09:37:01 cs-1091963627082-default sudo: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1000)
Jul  2 09:37:01 cs-1091963627082-default sshd[367]: pam_unix(sshd:session): session opened for user username(uid=1000) by (uid=0)
Jul  2 09:37:01 cs-1091963627082-default sshd[367]: pam_systemd(sshd:session): Failed to connect to system bus: No such file or directory
Jul  2 09:37:01 cs-1091963627082-default sudo: pam_unix(sudo:session): session closed for user rootJul  2 09:41:45 cs-1091963627082-default sshd[744]: rexec line 30: Deprecated option UsePrivilegeSeparation
Jul  2 09:41:45 cs-1091963627082-default sshd[744]: rexec line 33: Deprecated option KeyRegenerationInterval
Jul  2 09:41:45 cs-1091963627082-default sshd[744]: rexec line 34: Deprecated option ServerKeyBits
Jul  2 09:41:45 cs-1091963627082-default sshd[744]: rexec line 45: Deprecated option RSAAuthentication
Jul  2 09:41:45 cs-1091963627082-default sshd[744]: rexec line 52: Deprecated option RhostsRSAAuthentication
Jul  2 09:41:45 cs-1091963627082-default sshd[744]: error: kex_exchange_identification: banner line contains invalid characters
Jul  2 09:41:45 cs-1091963627082-default sshd[744]: banner exchange: Connection from 127.0.0.1 port 60734: invalid format

Not sure if this is me connecting to the shell from the GCS website.

The authorized_keys file should be containing the public keys:

$ cat ~/.ssh/authorized_keys
ssh-rsa AAAA......=

After following the instructions on step 4 I got the following:

ECDSA key fingerprint is SHA256:some_fingerprint.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
username@localhost: Permission denied (publickey).

I guess this makes sure that it is a server side issue?

Upvotes: 1

Answers (2)

nu_nad

Reputation: 9

3rd party platforms such as virtual private server or cloud providers include their own settings in the Linux images provided which override custom SSH changes.

To enable SSH password authentication these changes also need to be made:

in /etc/ssh/ssd_config edit:

#Include /etc/ssh/sdd_confid./*conf

in /etc/cloud/cloud.cfg edit:

disable_root: false
ssh_pwauth: true

in /etc/cloud/cloud.cfg.d/00_defaults edit:

ssh_pwauth: true

Upvotes: -1

jeb

Reputation: 82390

You should check the output of ssh -vv.
Is your private key used and offered?
Do you connect to the correct server on the correct port?
You could check (on the server) if all directories have the proper rights/owner.
Because ssh doesn't use authorized_keys, if any of the parent directories have permissions that are too open.

namei -mo $(realpath ~/.ssh/authorized_keys )
You could look into the journal (on the server) while connecting from the client

sudo journalctl --unit ssh --pager-end --follow
As a simple test you could create a new key on the server and try a local connect

ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519 -N ""
cat ~/.ssh/id_ed25519 >> ~/.ssh/authorized_keys
ssh localhost

If that works, copy the id_ed25519 file to your client and try it from there

Upvotes: 1

Unable to fix SSH Permission denied (publickey) error

Answers (2)

Related Questions