CODEWITHSUNDEEP
amazon-s3amazon-iamaws-glue
JoeJ
JoeJ

Reputation: 172

Glue Job failing due to inability to download script from S3

I am working on creating an AWS Glue ETL process to pull CSV data from S3 into an AWS Aurora DB.

I have successfully created and run the crawlers for both the source file and the destination DB table. I have also created the ETL Job, performed the mappings and saved the auto-generated script. All (S3, DB, Glue) are located in the same region.

When running the ETL job, it errors out with the following messages

fatal error: Connect timeout on endpoint URL: <REDACTED>
Error downloading script: fatal error: Connect timeout on endpoint URL: <REDACTED>

In the Cloudwatch Log, I have 2 relevant messages:

Specifying us-west-2 while copying script. 
S3 copy with region specified failed. Falling back to not specifying region.

I have tried the following:

Any other thoughts or solutions are appreciated!

Upvotes: 1

Views: 5430

Answers (1)

Prabhakar Reddy
Prabhakar Reddy

Reputation: 5144

I think the VPC which you have used to launch Glue job is missing S3 endpoint. As the traffic will not leave AWS network there will not be internet access inside your Glue job's VPC. So if you want to connect to S3 then you need to add it to your VPC.

Refer to this to know more and this to make sure that you properly configured security groups

Upvotes: 3

Related Questions