CODEWITHSUNDEEP
amazon-web-servicessshlambdaamazon-data-pipeline
bjamin
bjamin

Reputation: 143

SSH to ec2 instance and execute

I have a datapipeline application that I need to respond to. When it finishes, I ssh to an ec-2 instance and execute a script. What is the best way to do ssh to that box after datapipeline finishes? Should I use a lambda function and have it listen for datapline completion and ssh to that box and execute the script? I can't take the script off of that box.

Upvotes: 0

Views: 605

Answers (2)

bjamin
bjamin

Reputation: 143

Here's what I ended up doing. I wrote a lambda application that reads the SNS message from datapipeline, pulled a PEM from an encrypted S3 bucket, and sshed into the box, and executed the shell script.

Upvotes: 0

user1452132
user1452132

Reputation: 1768

There are couple of solutions here that I can think of.

  • At the end of completion of Datapipeline, write a zero byte file on S3 to signal end of it. That S3 put can signal a Lambda function as you noted. But I do not know how to rexec a shell script.

  • Another approach is to install taskrunner on the ec2-box and make that participate in your Data pipeline. That shell script could be the 'activity' in the pipeline. Here is a how that can be done http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-how-task-runner-user-managed.html This approach does not need to go outside your current Datapipeline control flow and if the shellscript fails for any reason, the pipeline would fail providing you with integrated view.

Upvotes: 1

Related Questions