AWS API gateway Datestamp is not JSON serializable

Question

I am using lambda function and triggering it using an API gateway rest api. While testing I am getting the below error. I need to pass the Datestamp in the request body which behaves as an input

I googled it and found this link and tried this solution but it didn't work -https://code-maven.com/serialize-datetime-object-as-json-in-python

My index.py -

dynamodb = boto3.resource('dynamodb')
resultstablename=('%s-%s-results' % (Project, Environment))
resultstable = dynamodb.Table(resultstablename)


def lambda_handler(event, context):
    try:
        result = []
        queryresponse = resultstable.scan()
        for i in queryresponse['Items']:
            result.append(i['reports'])

        block=['{"type": "section","text": {"type": "mrkdwn","text": "*Found below reports using Datestamp provided*"}}']
        for item in result:
            print(item)
            print(event['Datestamp'])
            if item.find(event['Datestamp']) != -1:
                itema='{"type": "section","text": {"type": "mrkdwn","text": "<https://%s/%s/%s/index.html|%s>"}}' % (SiteSpeedURL,SiteSpeedPreFix,item,item)
                block.append(itema)

block=('[{}]'.format(', '.join(block)))
        
        url = "%s" % (SlackURL)           
        msg = {
        "channel": "%s" % (ChannelName),
        "username": "%s" % (Username),
        "icon_emoji": ":robot_face:",
        "blocks": "%s" % (block),
            }
        encoded_msg = json.dumps(msg).encode('utf-8')
        **resp = http.request('POST',url, body=encoded_msg)**
        
        return("200")

The same code works fine when i send the command to chatbot like this -

lambda invoke --function-name monitoring-chatbot-shashank --payload {“Datestamp”:“2020-12-23"}

BUT The same payload like this when i am using in the request body while invoking this function from API gateway, it throws the error.

Error I am getting when invoking using API gateway using rest end point -

Wed Jan 06 11:47:05 UTC 2021 : Lambda execution failed with status 200 due to customer function error: Unable to marshal response: KeyError('Datestamp') is not JSON serializable. Lambda request id: 8baea1a2-517a-4f69-8a8f-6f17746dd360
Wed Jan 06 11:47:05 UTC 2021 : Method completed with status: 502

I don't know what else to do. Let me know if anyone got any idea on this please.

Answer 1

Your issue here is that you are expecting event to be the request body, but it isn't. Instead of the object you are passing via the CLI,

event == {"Datestamp": "2020-12-23"}

you will actually get an object generated by API gateway that describes the request (see docs), which includes the request body as a string.

event == {
    "resource": "/",
    "path": "/",
    "httpMethod": "GET",
    "requestContext": {
        "resourcePath": "/",
        "httpMethod": "GET",
        "path": "/Prod/",
        ...
    },
    "headers": {
        "accept": "text/html",
        ...
    },
    "multiValueHeaders": {
        ...
    },
    "queryStringParameters": null,
    "multiValueQueryStringParameters": null,
    "pathParameters": null,
    "stageVariables": null,
    "body": "{\"Datestamp\": \"2020-12-23\"}",
    "isBase64Encoded": false
}

So to get the JSON body as a Python dictionary, try this:

body = json.loads(event["body"])

...

if item.find(body['Datestamp']) != -1:
    ...

---

As the previous answer indicated, the "not JSON serializable" bit is referring to the KeyError that is raised when "Datestamp" is absent from the event. As you've seen, API gateway will treat failed Lambda executions (e.g. from uncaught exceptions) as a gateway error and return 502. This may be fine, but if you want to return a different error (and/or log more information) then it can be helpful to wrap all your code in a try/except block. I also often like to print the whole event at the start of each function, to help debug the HTTP bits.

def handler(event, context):
    try:
        print(event)  # helps debug input, not for production
        
        # main app code
        ...
    except Exception as e:
        print(repr(e))  # helps debug other issues, not for production

        return {
            "statusCode": 500,
            "body": "Internal server error: " + repr(e),  # DEFINITELY not for production
        }

Make sure only to do this while debugging though - when you print this stuff anyone with access to your logs can get information about application state, which may include sensitive data, and when you include it in the HTTP response then anyone with access to your API can do the same.

Answer 2

This is not related to datetime itself, problem likely lies here:

if item.find(event[Date2]) != -1

The event dict doesn't have '2020-12-22' key, and it raises KeyError. API gateway tries to treat this traceback message as a JSON response and understandably cannot serialize it.