CODEWITHSUNDEEP
amazon-web-servicesapache-sparksshamazon-ec2
bsky
bsky

Reputation: 20222

SSH connection error - Permission denied (publickey)

I'm trying to run a Spark cluster on AWS using https://github.com/amplab/spark-ec2.

I've generated a key and and login credentials, and I'm using this command:

./spark-ec2 --key-pair=octavianKey4 --identity-file=credentials3.csv --region=eu-west-1 --zone=eu-west-1c launch my-instance-name

However, I keep getting this:

Warning: SSH connection error. (This could be temporary.)
Host: mec2-myHostNumber.eu-west-1.compute.amazonaws.com
SSH return code: 255
SSH output: Warning: Permanently added 'ec2-myHostNumber.eu-west-1.compute.amazonaws.com,myHostNumber' (ECDSA) to the list of known hosts.
Permission denied (publickey).

If I quit the console and then try to start the cluster again, I get this:

Setting up security groups...

Searching for existing cluster my-instance-name in region eu-west-1...
Found 1 master, 1 slave.
ERROR: There are already instances running in group my-instance-name-master or my-instance-name-slaves

Upvotes: 0

Views: 904

Answers (2)

helloV
helloV

Reputation: 52375

Can you add --resume to your spark-ec2 command and try? Your slave may not have the key. --resume will make sure it is transferred to the slave.

Running Spark on EC2

If one of your launches fails due to e.g. not having the right permissions on your private key file, you can run launch with the --resume option to restart the setup process on an existing cluster.

Upvotes: 1

acsrujan
acsrujan

Reputation: 1049

The command is incorrect. Key pair name should be the one you mention in AWS. Identity file is .pem file associated. You can't ssh into a machine with AWS credentials (your csv file is credentials).

./spark-ec2 --key-pair=octavianKey4 --identity-file=octavianKey4.pem --region=eu-west-1 --zone=eu-west-1c launch my-instance-name

Upvotes: 2

Related Questions