CODEWITHSUNDEEP
amazon-web-servicesaws-glueamazon-vpc
Naguib Ihab
Naguib Ihab

Reputation: 4496

At least one security group must open all ingress ports. AWS Glue connecting to RDS

I am still starting out with AWS Glue and I am trying to connect it to my publicly accessible MySql database hosted on RDS Aurora to get its data.

So I start by creating a crawler and in the data store I create a new connection as in the screenshot below: enter image description here

I go through the rest and eventually try to run the crawler but I get the following error: At least one security group must open all ingress ports.To limit traffic, the source security group in your inbound rule can be restricted to the same security group

I am not sure what I need to change in the security group attached to the RDS but here's what I have right now for the inbound rules:

enter image description here

You'll notice that I have a self-referencing rule in there that's pointing to the same security group.

The outbound rules are going to all traffic.

Any idea what I might be doing wrong?

Upvotes: 26

Views: 50198

Answers (4)

Vzzarr
Vzzarr

Reputation: 5620

I found @David I. Rock solution to be working but has the inconvenience to stop connections via SQL Clients.

On top of that I also added the inbound rule:

  • Type: MYSQL / Aurora
  • Protocol: TCP (automatically generated)
  • Port Range: 3306 (automatically generated)
  • Source: My IP (or adapt to your requirements)

Upvotes: 0

David I. Rock
David I. Rock

Reputation: 125

You need to set a new rule in the security group that is attached to your DB instances where you define:

  • Type: All TCP
  • Protocol: TCP
  • Range: 0 - 65535
  • Source: Custom sg-(the id of this/self security group)
  • Description: whatever you want

Upvotes: 9

Christopher Armstrong
Christopher Armstrong

Reputation: 2307

The inbound rule (Glue Connection security group) is set to allow TCP Port 0 to allow traffic. Instead, it should allow ALL traffic. Edit your rules, and where there's a dropdown that says "Custom TCP Rule", and change it to "All TCP".

The documentation explains how to setup the security group

Upvotes: 27

jlwhite
jlwhite

Reputation: 51

To solve the second error mentioned above in the comments (VPC S3 endpoint validation failed for SubnetId: subnet-1944ab40. VPC: vpc-c8605bad. Reason: Could not find S3 endpoint or NAT gateway for subnetId: subnet-1944ab40 in Vpc vpc-c8605bad) you have to create an Amazon VPC Endpoints for Amazon S3. https://docs.aws.amazon.com/glue/latest/dg/vpc-endpoints-s3.html

Upvotes: 5

Related Questions