CODEWITHSUNDEEP
amazon-web-services
user1652054
user1652054

Reputation: 465

Stuck in WAITING_FOR_RUNNER while using an AMI for EC2 Resource

I was able to successfully run Data pipeline for predefined templates. I wanted to use a customized AMI for my EC2 Resource as I require some libraries and packages to be installed.

I also have to run a python script as a part of the process.

So, I have created a Base Image of EC2 Resource with all packages required and the code that has to be run.

As part of my activity, I trigger a shell command activity, where I execute the python script as the command that has to be run.

The EC2 resource comes up successfully based on the Customized AMI that I have specified. I am able to login to that machine using the key pair that I specified but just that the Activity gets stuck in "WAITING FOR RUNNER" state.

I am not sure as to how to solve this problem. Please let me know if there are better ways to fix the same. Am I missing some basic step while trying to use a EC2 resource from an AMI.

Upvotes: 5

Views: 3476

Answers (3)

bruckerrlb
bruckerrlb

Reputation: 305

I realize this is old but if you are using a custom AMI with runson specified, you should make sure your custom AMI has all dependencies installed: https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-custom-ami.html

Upvotes: 0

jqualls
jqualls

Reputation: 1503

Use Amazon Linux when creating your custom AMI and it will resolve this issue. Their OS comes preinstalled with tools that are used by data pipelines to communicate with the instance.

Upvotes: 2

user3019227
user3019227

Reputation: 1

Are you running in a VPC or EC2Classic? I had the same problem when running in a VPC. When I checked run.out on the EC2 instance, I saw an error message:

Error in custom provider, java.lang.RuntimeException: java.net.UnknownHostException: . . . "

The TaskRunner was not able to resolve its own hostname, and was failing to start.

I solved this by setting the "DNS hostnames" setting to yes on my VPC in the VPC console. By default on new accounts it is set to no. This resolved the issue.

Upvotes: 0

Related Questions