aws lambda development - s3 access not working in local debug environment, only on deployed

Question

I am trying to get local AWS Lambda development/debugging with Python3.8 working. I am using vscode and I have the AWS Toolkit extension enabled. It worked fine for the basic "hello world" lambda function that they give you. Now I want to modify the example to read some text from a file in S3, and when I try to debug it locally I get the error message "An error occurred (AccessDenied) when calling the GetObject operation: Access Denied" but if I deploy the SAM application to AWS, it works fine in the actual AWS environment.

As far as I can tell, I added the S3ReadPolicy correctly in the template.yaml file, because (as stated) it works fine in AWS when it gets deployed - the role it creates has the S3 read permissions correctly added. But local runs crash and burn.

What am I doing wrong?

Here's the good results that I see when I test it on AWS after deploying it to AWS:

START RequestId: 8841bcdb-1f3c-4772-82a3-fb47c29ec594 Version: $LATEST
About to get data from s3.
Got some stuff out of s3:
Hello. This is a text file.
May the odds be ever in your favor.
END RequestId: 8841bcdb-1f3c-4772-82a3-fb47c29ec594
REPORT RequestId: 8841bcdb-1f3c-4772-82a3-fb47c29ec594  Duration: 1884.15 ms    Billed Duration: 1900 ms    Memory Size: 128 MB Max Memory Used: 77 MB  Init Duration: 528.07 ms

Here's the output and error message that I see when I run it locally:

Local invoke of SAM Application has ended.
Preparing to debug 'app___vsctk___debug.lambda_handler' locally...
Building SAM Application...
Build complete.
Starting the SAM Application locally (see Terminal for output)
Running command: [/usr/local/bin/sam local invoke awsToolkitSamLocalResource --template /tmp/aws-toolkit-vscode/vsctkdmFPUi/output/template.yaml --event /tmp/aws-toolkit-vscode/vsctkdmFPUi/event.json --env-vars /tmp/aws-toolkit-vscode/vsctkdmFPUi/env-vars.json -d 5858]
Invoking app___vsctk___debug.lambda_handler (python3.8)
Skip pulling image and use local one: amazon/aws-sam-cli-emulation-image-python3.8:rapid-1.6.2.

Mounting /tmp/aws-toolkit-vscode/vsctkdmFPUi/output/awsToolkitSamLocalResource as /var/task:ro,delegated inside runtime container
START RequestId: d3ff0f84-6f93-1065-b44d-ab9f5f174fdd Version: $LATEST
Waiting for debugger to attach...
Waiting for SAM Application to start before attaching debugger...
Attaching debugger to SAM Application...
Debugger attached
...debugger attached
About to get data from s3.
An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
[ERROR] ClientError: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
Traceback (most recent call last):
  File "/var/task/app___vsctk___debug.py", line 17, in lambda_handler
    return _handler(event, context)
  File "/var/task/app.py", line 17, in lambda_handler
    raise(e)
  File "/var/task/app.py", line 13, in lambda_handler
    data = s3.get_object(Bucket=bucket, Key=key)
  File "/var/task/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/var/task/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)
END RequestId: d3ff0f84-6f93-1065-b44d-ab9f5f174fdd
REPORT RequestId: d3ff0f84-6f93-1065-b44d-ab9f5f174fdd  Init Duration: 1763.86 ms   Duration: 1444.53 ms    Billed Duration: 1500 ms    Memory Size: 128 MB Max Memory Used: 54 MB  

{"errorType":"ClientError","errorMessage":"An error occurred (AccessDenied) when calling the GetObject operation: Access Denied","stackTrace":["  File \"/var/task/app___vsctk___debug.py\", line 17, in lambda_handler\n    return _handler(event, context)\n","  File \"/var/task/app.py\", line 17, in lambda_handler\n    raise(e)\n","  File \"/var/task/app.py\", line 13, in lambda_handler\n    data = s3.get_object(Bucket=bucket, Key=key)\n","  File \"/var/task/botocore/client.py\", line 357, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n","  File \"/var/task/botocore/client.py\", line 676, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"]}
Local invoke of SAM Application has ended.

Here's the relevant files:

lambda_test/hello_world/app.py

import boto3
import json
import time

def lambda_handler(event, context):

    s3 = boto3.client('s3')
    bucket = "rtb-imaginary-bucket"
    key = "a-text-file.txt"

    print('About to get data from s3.')
    try:
        data = s3.get_object(Bucket=bucket, Key=key)
        file_content = data['Body'].read().decode('utf-8')
    except Exception as e:
        print(e)
        raise(e)

    print('Got some stuff out of s3:')
    print(file_content)

    return {
        "statusCode": 200,
        "body": json.dumps({
            "message": "hello world",
            "file_content": file_content

        }),
    }

lambda_test/template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  lambda_test

  Sample SAM Template for lambda_test

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.8
      Policies:
        - S3ReadPolicy:
            BucketName: rtb-imaginary-bucket

      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

.vscode/launch.json

{
    "configurations": [
        {
            "type": "aws-sam",
            "request": "direct-invoke",
            "name": "lambda_test:app.lambda_handler (python3.8)",
            "invokeTarget": {
                "target": "code",
                "projectRoot": "lambda_test/hello_world",
                "lambdaHandler": "app.lambda_handler"
            },
            "lambda": {
                "runtime": "python3.8",
                "payload": {
                    "json": {
                        "key1": "value1",
                        "key2": "value2",
                        "key3": "value3"
                    }
                },
                "environmentVariables": {}
            }
        }
    ]
}

Answer 1

As far as I know, the SAM functionality does not inherit the credentials profile that you set for the AWS Toolkit.

You can set the profile explicitly for SAM in your launch.json, for example:

{
    "configurations": [
        {
            "type": "aws-sam",
            "invokeTarget": { ... },
            "lambda": { ... },
            "aws": {
                "credentials": "profile:knievel"
            }
        }
    ]
}