CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-glue
Debojyoti Mukherjee
Debojyoti Mukherjee

Reputation: 19

AWS Glue Error - An error occurred (403) when calling the HeadObject operation: Forbidden

I have a simple Glue pythonshell job and for testing purpose I just have print("Hello World") in it.

I have given it the required AWSGlueServiceRole. When I am trying to run the job it throws the following error:

Traceback (most recent call last):
  File "/tmp/runscript.py", line 114, in <module>
    temp_file_path = download_user_script(args.scriptLocation)
  File "/tmp/runscript.py", line 91, in download_user_script
    download_from_s3(args.scriptLocation, temp_file_path)
  File "/tmp/runscript.py", line 81, in download_from_s3
    s3.download_file(bucket_name, s3_key, new_file_path)
  File "/usr/local/lib/python3.6/site-packages/boto3/s3/inject.py", line 172, in download_file
    extra_args=ExtraArgs, callback=Callback)
  File "/usr/local/lib/python3.6/site-packages/boto3/s3/transfer.py", line 307, in download_file
    future.result()
  File "/usr/local/lib/python3.6/site-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/usr/local/lib/python3.6/site-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/usr/local/lib/python3.6/site-packages/s3transfer/tasks.py", line 255, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/s3transfer/download.py", line 345, in _submit
    **transfer_future.meta.call_args.extra_args
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

When I add S3 full access policy to the role, then the job runs successfully. I am not able to debug what is wrong

Upvotes: 1

Views: 2291

Answers (2)

IamAshay
IamAshay

Reputation: 1491

You need to have the permissions to the bucket and the script resource.

Try adding the following inline policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::myBucket/*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::myBucket",
            "Effect": "Allow"
        }
    ]
}

Upvotes: 0

Walker Rowe
Walker Rowe

Reputation: 983

In Glue you need to attach S3 policies to the Amazon Glue Role that you are using to run the job. When you define the job you select the role. In this example it is AWSGlueServiceRole-S3IAMRole. That does not have S3 access until you assign it.

enter image description here

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "*"
        }
    ]
}

Upvotes: 0

Related Questions