CODEWITHSUNDEEP
hadoopamazon-web-servicessshamazon-ec2elastic-map-reduce
cybertextron
cybertextron

Reputation: 10961

cannot ssh into Elastic MapReduce

I'm using elastic-mapreduce to spun new clusters from the command line. After reading this tutorial, I have:

elastic-mapreduce --create --alive \
--instance-type m1.xlarge\
--num-instances 5 \
--supported-product mapr \
--name m7 \
--args "--edition,m7"

a new cluster with 6 nodes(1 master + 5 slaves) will be created. So I try to ssh into the master:

elastic-mapreduce --jobflow j-3FLVMX9CYE5L6 --ssh

and I get Permission denied (publickey) The permissions.json file is in the elastic_mapreduce home, and also the path to the my_key.pem file. Also, by doing:

ssh -i my_key.pem hadoop@masternode

I also get Permission denied (publickey). my_key.pem has been given 400 permissions. I honestly don't know what can I try next ... any clues?

When I go to the EC2 console, and I click in Connect, I get the following error message:

Instance is not associated with a key pair.
This instance is not associated with a Key Pair. Without a Key Pair you will need to log into this instance using a valid username and password combination.

Upvotes: 2

Views: 2164

Answers (2)

David Beveridge
David Beveridge

Reputation: 560

To augment Rico's comment, above; I would add that you can pull down the key on the command line when you --create the cluster: for your example:

elastic-mapreduce --create --alive \
--instance-type m1.xlarge\
--num-instances 5 \
--supported-product mapr \
--name m7 \
--args "--edition,m7"
--key-pair hadoopKey

Were "hadoopKey" is the name given to a keypair created via the AWS console.

When you SSH in, you are providing this key as "identity"--this means that the host server must have the private key (PEM file in this case). This action parameter (or Rico's example above) will push that key file onto the (and ONLY the) Master EC2 server.

N.B., if you want to SSH from the Master to one of the Worker boxen, you need to push the same PEM file to the master and then proceed as usual from there.

E.g., 

you@devbox:$ scp -i hadoopKey.pem hadoopKey.pem hadoop@ec2-the-master-server:
you@devbox:$ssh -i hadoopKey.pem hadoop@ec2-the-master-server:
<figure out the IP of a worker from w/in that Master
hadoop@ec2-the-master-server:$ ssh -i ~/hadoopKey.pem [email protected]

Hope that helps.

Upvotes: 0

Rico
Rico

Reputation: 61521

When you create your EMR cluster make sure you enable a key pair. On EMR console you can see it like below:

EMR

Upvotes: 3

Related Questions