Connecting to Amazon SQS key (SSE-SQS) Encrypted SQS Queue via Django Celery [sqs]

Question

I am unable to get my Celery working running while connecting to an SQS Encrypted Queue

This is my settings.py for my django project

SQS_AWS_ACCESS_KEY_ID = 'xxxx'
SQS_AWS_SECRET_ACCESS_KEY = 'xxxx'
SQS_AWS_QUEUE_NAME = 'sqs.eu-west-2.amazonaws.com/xxxx/xxx-celery-broker'

broker_url = f"sqs://{SQS_AWS_ACCESS_KEY_ID}:{SQS_AWS_SECRET_ACCESS_KEY}@{SQS_AWS_QUEUE_NAME}"
CELERY_BROKER_URL = broker_url
CELERY_RESULT_BACKEND = None
CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True 

When i run my worker i get this error, i assume as my Queue Name is not https://

botocore.exceptions.ClientError: An error occurred (InvalidSecurity) when calling the GetQueueAttributes operation: All requests to this queue must use HTTPS and SigV4.

when i change it to

SQS_AWS_QUEUE_NAME = 'https://sqs.eu-west-2.amazonaws.com/xxxx/xxx-celery-broker'

I get this error Cannot connect to sqs://xxx:**@https//sqs.eu-west-2.amazonaws.com/xxx/xxx-celery-broker: Could not connect to the endpoint URL: "http://https/".

The Config seems to add a random http onto the url which is why i assume its failing

When i connect to a non encrypted queue it works fine as seen as

This is my celery.py config

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "proj.settings")

# Create a Celery instance and configure it to use SQS
app = Celery("proj")

# Load task modules from all registered Django app configs.
app.config_from_object("django.conf:settings", namespace="CELERY")

# Auto-discover tasks in all installed apps

app.autodiscover_tasks(settings.INSTALLED_APPS)
app.conf.task_default_queue = 'xxx-celery-broker'

I'm at my wits end trying to fix this im assuming there is a small config change that will sort this

Thanks in Advance

This is from the logs when i set loglevel to debug

 Making request for OperationModel(name=GetQueueAttributes) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'Boto3/1.26.135 Python/3.11.6 Linux/6.2.0-1017-aws Botocore/1.29.165'}, 'body': {'Action': 'GetQueueAttributes', 'Version': '2012-11-05', 'QueueUrl': 'https://sqs.eu-west-2.amazonaws.com/id/name', 'AttributeName.1': 'ApproximateNumberOfMessages'}, 'url': 'http://https/', 'context': {'client_region': 'eu-west-2', 'client_config': <botocore.config.Config object at 0x7f4d28836b50>, 'has_streaming_input': False, 'auth_type': None}} 

Answer 1

In BROKER_TRANSPORT_OPTIONS, add the following line:

"is_secure": True

More on this issue here and credits to github/goatwu1993.