CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-s3boto3amazon-emr
Aviv Oron
Aviv Oron

Reputation: 565

ValueError: Invalid endpoint: https://s3..amazonaws.com

When EMR machine is trying to run a step that includes boto3 initialisation it sometimes get the following error: ValueError: Invalid endpoint: https://s3..amazonaws.com When I'm trying to set up a new machine it can suddenly work. Attached the full error:

self.client = boto3.client("s3")
  File "/usr/local/lib/python3.6/site-packages/boto3/__init__.py", line 83, in client
    return _get_default_session().client(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/boto3/session.py", line 263, in client
    aws_session_token=aws_session_token, config=config)
  File "/usr/local/lib/python3.6/site-packages/botocore/session.py", line 861, in create_client
    client_config=config, api_version=api_version)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 76, in create_client
    verify, credentials, scoped_config, client_config, endpoint_bridge)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 285, in _get_client_args
    verify, credentials, scoped_config, client_config, endpoint_bridge)
  File "/usr/local/lib/python3.6/site-packages/botocore/args.py", line 79, in get_client_args
    timeout=(new_config.connect_timeout, new_config.read_timeout))
  File "/usr/local/lib/python3.6/site-packages/botocore/endpoint.py", line 297, in create_endpoint
    raise ValueError("Invalid endpoint: %s" % endpoint_url)
ValueError: Invalid endpoint: https://s3..amazonaws.com

Any idea why it happens?

(Versions: boto3==1.7.29, botocore==1.10.29)

Upvotes: 27

Views: 53028

Answers (3)

Gavin
Gavin

Reputation: 414

In my case, even though ~/.aws/config had the region set,

$ cat ~/.aws/config 
[default]
region = us-east-1

the env var AWS_REGION was set to an empty string

$ env | grep -i aws
AWS_REGION=

unset this env var and all was good again

$ unset AWS_REGION
$ aws sts get-caller-identity --output text --query Account
777***234534

(apologies for posting on a really old question, it did pop up in a Google search)

Upvotes: 4

ubi
ubi

Reputation: 4399

Set the region in your ~/.aws/credentials or ~/.aws/config files. You can set the region as an environment variable as well e.g.

In bash

export AWS_REGION="eu-west-2"

or in Powershell

$Env:AWS_REGION="eu-west-2"

Upvotes: 16

ChickenSoups
ChickenSoups

Reputation: 975

It looks like you have an invalid region. Check your ~/.aws/config

Upvotes: 29

Related Questions