CODEWITHSUNDEEP
amazon-web-servicesamazon-vpcamazon-sagemaker
mon
mon

Reputation: 22388

SageMaker Studio is unable to reach SageMaker endpoint

Question

Please help understand the cause and how to fix.

Problem

Starting the kernel in the SageMaker Studio fails as below.

Failed to start kernel

SageMaker Studio is unable to reach SageMaker endpoint. Please ensure your VPC has connectivity to SageMaker via Internet or VPC Endpoint. If you are using VPC Endpoints, please ensure Security Groups allows traffic between Studio and VPC endpoints. Learn more about SageMaker Studio VpcOnly mode - https://docs.aws.amazon.com/sagemaker/latest/dg/studio-notebooks-and-internet-access.html

enter image description here

Deployment

SageMaker Studio VPC

SageMaker Studio has been deployed to the VPC subnet subnet-a8d0e6c1.

enter image description here

VPC Endpoints

As per VPC only communication with the internet, VPC endponits have been created.

  • SageMaker API : com.amazonaws.us-east-1.sagemaker.api
  • SageMaker runtime: com.amazonaws.us-east-1.sagemaker.runtime. This is required to run Studio notebooks and to train and host models.
  • Amazon S3: com.amazonaws.us-east-1.s3.

enter image description here

Internet access

Internet is accessible via NAT/IGW from the SageMaker Studio Terminal. Git clone etc has been working.

enter image description here

Upvotes: 1

Views: 2548

Answers (1)

mon
mon

Reputation: 22388

The VPC Endpoint SG needs to accept all TCP traffic from the SageMaker Studio ENI to the VPC endpoints. Seemingly being able to access the Internet may not be enough.

enter image description here

sg-56cb133e is attached to the VPC endpoints and the SageMaker Studio ENI (not ideal though). By allowing all TCP traffic among those associated with the SG fixed the issue.

enter image description here

enter image description here

Upvotes: 2

Related Questions