ValueError: Must setup local AWS configuration with a region supported by SageMaker

Question

I am new to Sagemaker and trying to use Sagemaker with python SDK with sample minist code provided by aws, and called it sm_mnist.py:

import boto3
import sagemaker
import tensorflow as tf
import argparse
import os
import numpy as np
import json

from sagemaker import get_execution_role

def model(x_train, y_train, x_test, y_test):
    model = tf.keras.models.Sequential([
        tf.keras.layers.Flatten(),
        tf.keras.layers.Dense(1024, activation=tf.nn.relu),
        tf.keras.layers.Dropout(0.4),
        tf.keras.layers.Dense(10, activation=tf.nn.softmax)
    ])

    model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',
                  metrics=['accuracy'])
    model.fit(x_train, y_train)
    model.evaluate(x_test, y_test)

    return model


def _load_training_data(base_dir):
    x_train = np.load(os.path.join(base_dir, 'train_data.npy'))
    y_train = np.load(os.path.join(base_dir, 'train_labels.npy'))
    return x_train, y_train


def _load_testing_data(base_dir):
    x_test = np.load(os.path.join(base_dir, 'eval_data.npy'))
    y_test = np.load(os.path.join(base_dir, 'eval_labels.npy'))
    return x_test, y_test


def _parse_args():
    parser = argparse.ArgumentParser()

    # Data, model, and output directories
    # model_dir is always passed in from SageMaker. By default this is a S3 path under the default bucket.
    parser.add_argument('--model_dir', type=str)
    parser.add_argument('--sm_model_dir', type=str, default=os.environ.get('SM_MODEL_DIR'))
    parser.add_argument('--train', type=str, default=os.environ.get('SM_CHANNEL_TRAINING'))
    #parser.add_argument('--hosts', type=list, default=json.loads(os.environ.get('SM_HOSTS')))
    #parser.add_argument('--currenthost', type=str, default=os.environ.get('SM_CURRENT_HOST'))

    return parser.parse_known_args()


if __name__ == "__main__":
    args, unknown = _parse_args()

    train_data, train_labels = _load_training_data(args.train)
    eval_data, eval_labels = _load_testing_data(args.train)

    mnist_classifier = model(train_data, train_labels, eval_data, eval_labels)

    if args.current_host == args.hosts[0]:
        # save model to an S3 directory with version number '00000001'
        mnist_classifier.save(os.path.join(args.sm_model_dir, '000000001'), 'my_model.h5')

I created the Tensorflow estimator train.py:

from sagemaker.tensorflow import TensorFlow
role = 'AmazonSageMaker-ExecutionRole-20200928T205562'
mnist_estimator = TensorFlow(entry_point='train.py',
                              role=role,
                              train_instance_count=2,
                              train_instance_type= 'ml.p3.2xlarge', #'local',
                              framework_version= '1.15.2',#,'2.1.0'
                              py_version='py3',
                             script_mode=True)
training_data_uri = 's3://my-dataset-us-east-1/mnist'
mnist_estimator.fit(training_data_uri)

and here is my dockerfile:

FROM tensorflow/tensorflow:1.15.2-gpu

# Install sagemaker-training toolkit to enable SageMaker Python SDK

RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y git
RUN pip3 install --upgrade pip && \
        pip3 install sagemaker-training

# Copies the training code inside the container
COPY train.py opt/ml/code/train.py
COPY sm_mnist.py opt/ml/code/mnist.py
COPY requirements.txt .
RUN pip3 install -r requirements.txt
# Defines train.py as script entrypoint
ENV SAGEMAKER_PROGRAM train.py
ENTRYPOINT ["python","opt/ml/code/train.py"]

I can create the image using:

docker build -t mnist_test:latest .
docker tag mnist_test:latest xxxx.dkr.ecr.us-east-1.amazonaws.com/mnist_test:latest
docker run --rm mnist_test --model_dir s3://my-dataset/models

I am getting this error which I could not solve it:

Traceback (most recent call last):
  File "opt/ml/code/train.py", line 27, in <module>
    sess = sagemaker.Session()
  File "/usr/local/lib/python3.6/dist-packages/sagemaker/session.py", line 115, in __init__
    sagemaker_runtime_client=sagemaker_runtime_client,
  File "/usr/local/lib/python3.6/dist-packages/sagemaker/session.py", line 129, in _initialize
    "Must setup local AWS configuration with a region supported by SageMaker."
ValueError: Must setup local AWS configuration with a region supported by SageMaker.

I do not know where my mistake is?

Answer 1

I got this same error and this worked for me, (taken from one of their examples)

role = sagemaker.get_execution_role()
region = boto3.Session().region_name

Answer 2

The get_execution_role takes in a sagemaker_session argument that defaults to None. I was able to get around this error by passing in a pre-built Sagemaker session, like this:

from sagemaker import get_execution_role
sagemaker_session = sagemaker.Session(boto3.session.Session(region_name=AWS_REGION))

sagemaker_session.boto_region_name # make sure this is set and not None

# pass in the sagemaker session as an argument
sagemaker_execution_role = get_execution_role(sagemaker_session=sagemaker_session)
# Output: 'arn:aws:iam::AWS_ACCOUNT_ID:role/EXECUTION_ROLE_NAME'

In the AWS documentation, there is a note that

The execution role is intended to be available only when running a notebook within SageMaker. If you run get_execution_role in a notebook not on SageMaker, expect a "region" error.

Since the goal is to get the execution role ARN, you could also use the approach recommended in the documentation:

try:
    role = sagemaker.get_execution_role()
except ValueError:
    iam = boto3.client('iam')
    role = iam.get_role(RoleName='AmazonSageMaker-ExecutionRole-20201200T100000')['Role']['Arn']

Answer 3

The error message indicates that you do not have an AWS region configured in your environment. There are a few ways to do this, including:

AWS CLI (docs):

$ aws configure  # follow the prompts
[...]
Default region name [None]: your-region-name

Environment variable (docs):

export $AWS_DEFAULT_REGION=your-region-name