AWS Glue returning error while running job

Question

I want to generate parquet file format from the data stored on aws redshift for which I have created connection and crawler to fetch the data information. All the database catalogs have been fetched successfully but while running job to fetch the table data, generate parquet file and store it to aws s3, AWS Glue gives me following error :

Reverse dns resolution of ip "someIp address" failed

However I can successfully generate parquet file with source data store and target data store being AWS S3. Error is generating only while I use my redshift cluster as a source data store and S3 as a target data store.

I know that error has nothing to do with Route53 service for which it is showing me an error. I have also checked my security group. It also allows all ingress traffic in VPC. I have also checked that S3 bucket and Redshift are in same region. Subnet is also correct.

Answer 1

Try setting SG egress rule of glue connection for DNS UDP to all. Protocol UDP, port 53.

Answer 2

you should enable DNS hostname and resolution.

go to your VPC > select your VPC > click on actions > click on Edit DNS resolution > click on Yes > click on save

similar to above, enable DNS hostname for your VPC

click on actions > click on Edit DNS hostname > click on Yes > click on save

Answer 3

You need to create reverse DNS record for all IPs in the redshift subnet.