I can't pass JSON to an AWS lambda function in a POST request without the lambda function inserting a backslash before every double quote in the JSON

Question

I have an AWS lambda function that I created through the sam cli tool. I started with a basic hello world template that I converted into a find anagrams function that accepts a JSON array of words and detects anagrams in the array. Right now, I'm just passing through the JSON input for debugging purposes. The template.yaml file looks like this:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  lambda-tester-two

  Sample SAM Template for lambda-tester-two
  
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3
    MemorySize: 128

Resources:
  HttpApi:
      Type: AWS::Serverless::HttpApi
      Properties:
        StageName: nonprod
  FindAnagramsFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: find-anagrams/
      Handler: app.lambdaHandler
      Runtime: nodejs16.x
      Architectures:
        - x86_64
      Events:
        PostWords:
          Type: HttpApi 
          Properties:
            Path: /anagram
            Method: post
            ApiId:
              Ref: HttpApi
    Metadata: # Manage esbuild properties
      BuildMethod: esbuild
      BuildProperties:
        Minify: true
        Target: "es2020"
        # Sourcemap: true # Enabling source maps will create the required NODE_OPTIONS environment variables on your lambda function during sam build
        EntryPoints: 
        - app.ts

The app.ts file looks like this:

import { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda';

/**
 *
 * Event doc: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-input-format
 * @param {Object} event - API Gateway Lambda Proxy Input Format
 *
 * Return doc: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html
 * @returns {Object} object - API Gateway Lambda Proxy Output Format
 *
 */

export const lambdaHandler = async (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> => {
    let response: APIGatewayProxyResult;
    try {
        const words = event.body;
        let result = {}
        
        for (const word of words) {
            const sorted = word.split("").sort().join("");
        
            if (sorted in result) {
                result[sorted].push(word);
            } else {
                result[sorted] = [word];
            }
        }
        response = {
            statusCode: 200,
            body: JSON.stringify({
                message: words,
            }),
        };
    } catch (err: unknown) {
        console.error(err);
        response = {
            statusCode: 500,
            body: JSON.stringify({
                message: err instanceof Error ? err.message : 'some error happened',
            }),
        };
    }

    return response;
};

I run the code with sam build then sam local start-api. I always have Docker Desktop running in the background. I expect this running code to accept a POST request at http://127.0.0.1:3000/anagram and print out the json sent in the body of the request. But the JSON that is returned looks weird... This is what my Insomnia window looks like:

Why is it adding all the \n \ characters before the " characters?

I tried making the input just a minified string with no spaces but it still returned weird...

Finally I added this code to replace const words = event.body; in order to strip out the \ characters:

const wordsWithSlashes = event.body;
const words = wordsWithSlashes.replace(/\\/g,"-");

And it ignored my regex and still returned weird JSON with \s before the " characters:

So how do I get my AWS lambda function to accept the correct JSON sent in the body of the request without adding \ characters?

Answer 1

For those who are working with the SAM template and Python, I can confirm that this works:

              x-amazon-apigateway-integration:
                type: aws
                uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${WebhookLambda.Arn}/invocations
                httpMethod: POST
                passthroughBehavior: when_no_match
                requestTemplates:
                  application/json: >
                    {
                      "body": "$util.base64Encode($input.body)",
                      "headers": {
                        #foreach($param in $input.params().header.keySet())
                        "$param": "$util.escapeJavaScript($input.params().header.get($param))"
                        #if($foreach.hasNext),#end
                        #end
                      }
                    }

and in your lambda:

stripe_webhook_secret = 'YOUR_WEBHOOK_SECRET'.   # See note below
def lambda_handler(event, context):
    print('Webhook event received. :', event)
    header_signature = event['headers']['Stripe-Signature']
    body_str = base64.b64decode(event['body']).decode('utf-8')
    try:
        event = stripe.Webhook.construct_event(
            body_str, header_signature, stripe_webhook_secret
        )
    except ValueError:
        print('invalid payment')
        return {
            'statusCode': 400,
            'body': json.dumps({'error': 'Invalid payload'}),
            'headers': {'Content-Type': 'application/json'}
        }
    except stripe.error.SignatureVerificationError:
        print('invalid signature')
        return {
            'statusCode': 400,
            'body': json.dumps({'error': 'Invalid signature'}),
            'headers': {'Content-Type': 'application/json'}
        }
    print('Successfully verified the payment signature')

Also in the stripe console, DO not copy the small text that looks like a signing key in front of the webhook URL. Make sure You click on reveal secret to show the webhook signing key and use that in your code

Answer 2

The problem was actually in the response code with JSON.stringify(... First, let me post the entire fixed app.ts file:

import { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda';

/**
 *
 * Event doc: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html#api-gateway-simple-proxy-for-lambda-input-format
 * @param {Object} event - API Gateway Lambda Proxy Input Format
 *
 * Return doc: https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html
 * @returns {Object} object - API Gateway Lambda Proxy Output Format
 *
 */

export const lambdaHandler = async (event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> => {
    let response: APIGatewayProxyResult;
    try {
        const words = JSON.parse(event.body).listedwords;
        let result = {}
        
        /* for (const word of words) {
            const sorted = word.split("").sort().join("");
        
            if (sorted in result) {
                result[sorted].push(word);
            } else {
                result[sorted] = [word];
            }
        } */
        response = {
            statusCode: 200,
            body: words
        };
    } catch (err: unknown) {
        console.error(err);
        response = {
            statusCode: 500,
            body: JSON.stringify({
                message: err instanceof Error ? err.message : 'some error happened',
            }),
        };
    }

    return response;
};

The notable change is this:

response = {
    statusCode: 200,
    body: words
};

Notice I'm not putting JSON.stringify(words) in the response body, I'm just putting words in the response body. So this is what the function now returns:

It's also worth noting that you do have to actually parse the input JSON in the POST request body like I did with this line: const words = JSON.parse(event.body).listedwords;. The JSON is now formatted properly.