Create API gateway in localstack

Question

I was able to setup localstack (https://github.com/atlassian/localstack) and also create lambda function in it (using create-function ... command). However, I couldnt find a way to create an APIGateway in localstack so that the lambda function can be called using it. Basically, I need an APIGateway(and its arn), so that using that the lambda function can be called.

Answer 1

The answer of @SteffenBach is very clear, but I think my recent experience could help some users. The lambda I am integrating is written in Java and implements the RequestStreamHandler interface.

IMPORTANT: The procedure I will show is based on the tests I found on the localstack repository. In the Coverage page, search for the Api Gateway service and you will find all the tests they have done for that service. Then you should look for the test you are interested in in the Localstack repo. I suggest you to take a look at it every time you need to use some aws service with Localstack. The tests are written in python but the client commands are very similar to the cli commands.

---

For my lambda integration, I am using a RequestStreamHandler, which does not always work with the AWS_PROXY integration type (with Localstack). In fact, Localstack only supports the payload format v1 for Lambda Integration. This means that we either return the response in the following format (and in this case we can use the AWS_PROXY integration type)

{
    "isBase64Encoded": true|false,
    "statusCode": httpStatusCode,
    "headers": { "headername": "headervalue", ... },
    "multiValueHeaders": { "headername": ["headervalue", "headervalue2", ...], ... },
    "body": "..."
}

Or we can return the body directly but using the AWS integration type, which is what I am showing next.

---

I am running Localstack on docker, so the next snippets come from the setup.sh file I have written:

1. Configure the environment with aws credentials and other options:

   printf "%s\n%s\n%s\n%s\n" "test_key_id" "test_access_key" "$REGION" "text" | aws configure
   

   In this way, I do not have to specify the region every time. "text" refers to the default output format (default is json).

2. Create the lambda function and store its id:

   LAMBDA_ARN="$(
       awslocal lambda create-function \
           ... \
           --query "FunctionArn"
   )"
   

   Note: If you are creating and integrating the lambda inside a script, don't forget to wait for the lambda to become Active:

   awslocal lambda wait function-active-v2 --function-name "$LAMBDA_NAME"
   

3. Create the rest api and get the resource id (see here for more info):

   REST_API_ID="$(
       awslocal apigateway create-rest-api \
           --name "$API_NAME" \
           --query "id"
   )"
   
   API_ROOT_RESOURCE_ID="$(
       awslocal apigateway get-resources \
           --rest-api-id "$REST_API_ID" \
           --query "items[0].id"
   )"
   
   RESOURCE_ID="$(
       awslocal apigateway create-resource --rest-api-id "$REST_API_ID" \
           --parent-id "$API_ROOT_RESOURCE_ID" \
           --path-part "$RESOURCE_PATH_PART" \
           --query "id"
   )"
   

4. Create the lambda integration:

   awslocal apigateway put-method \
       --rest-api-id "$REST_API_ID" \
       --resource-id "$RESOURCE_ID" \
       --http-method POST \
       --authorization-type "NONE"
   
   awslocal apigateway put-method-response \
       --rest-api-id "$REST_API_ID" \
       --resource-id "$RESOURCE_ID" \
       --http-method POST \
       --status-code 200
   
   awslocal apigateway put-integration \
       --rest-api-id "$REST_API_ID" \
       --resource-id "$RESOURCE_ID" \
       --http-method POST \
       --type AWS \
       --integration-http-method POST \
       --uri "arn:aws:apigateway:$REGION:lambda:path/2015-03-31/functions/$LAMBDA_ARN/invocations" \
       --passthrough-behavior WHEN_NO_MATCH
   
   awslocal apigateway put-integration-response \
       --rest-api-id "$REST_API_ID" \
       --resource-id "$RESOURCE_ID" \
       --http-method POST \
       --status-code 200
   

5. Create the deployment environment:

   awslocal apigateway create-deployment --rest-api-id "$REST_API_ID" --stage-name "$DEPLOYMENT_NAME"
   

The lambda is ready to be called, like with curl:

curl -d '{"key1":"value1", "key2":"value2"}' -H "Content-Type: application/json" -X POST \
"http://localhost:4566/restapis/$REST_API_ID/$DEPLOYMENT_NAME/_user_request_/$RESOURCE_PATH_PART"

The input stream of my lambda is exactly what I put in the request body (-d '{"key1":"value1", "key2":"value2"}').

Answer 2

Looks like there is an open issue related to setting up API Gateway with localstack on GitHub:

https://github.com/localstack/localstack/issues/129

You could try following the steps provided in the answer there.

Copied from the GitHub issue:

"""

One option would be to use the serverless framework (https://github.com/serverless/serverless). Otherwise, you can call the LocalStack services directly (via the CLI or an SDK) to create an API Gateway resource+method+integration, and connect them to your Lambda function.

Here are a few pointers that might be helpful: https://ig.nore.me/2016/03/setting-up-lambda-and-a-gateway-through-the-cli/ (the "Creating a role" part can be skipped) https://github.com/atlassian/localstack/issues/101 https://github.com/temyers/serverless-localstack """

Answer 3

Walkthrough for creating a NodeJS Lambda together with API Gateway per CLI:

First we create a simple NodeJS Lambda:

const apiTestHandler = (payload, context, callback) => {
console.log(`Function apiTestHandler called with payload ${JSON.stringify(payload)}`);
callback(null, {
    statusCode: 201,
    body: JSON.stringify({
        somethingId: payload.pathParameters.somethingId
    }),
    headers: {
        "X-Click-Header": "abc"
    }
}); 
}
module.exports = {
    apiTestHandler,
}

Put that into a zip File called apiTestHandler.zip and upload it to localstack:

aws lambda create-function \
--region us-east-1 \
--function-name api-test-handler \
--runtime nodejs6.10 \
--handler index.apiTestHandler \
--memory-size 128 \
--zip-file fileb://apiTestHandler.zip \
--role arn:aws:iam::123456:role/role-name --endpoint-url=http://localhost:4574

Now we can create our Rest-Api:

aws apigateway create-rest-api --region us-east-1 --name 'API Test' --endpoint-url=http://localhost:4567

This gives the following response:

{
"name": "API Test",
"id": "487109A-Z548",
"createdDate": 1518081479
}

With the ID we got here, we can ask for its parent-ID:

aws apigateway get-resources --region us-east-1 --rest-api-id 487109A-Z548 --endpoint-url=http://localhost:4567

Response:

{
"items": [
    {
        "path": "/",
        "id": "0270A-Z23550",
        "resourceMethods": {
            "GET": {}
        }
    }
]
}

Now we have everything to create our resource together with its path:

aws apigateway create-resource \
--region us-east-1 \
--rest-api-id 487109A-Z548 \
--parent-id 0270A-Z23550 \
--path-part "{somethingId}" --endpoint-url=http://localhost:4567

Response:

{
"resourceMethods": {
    "GET": {}
},
"pathPart": "{somethingId}",
"parentId": "0270A-Z23550",
"path": "/{somethingId}",
"id": "0662807180"
}

The ID we got here is needed to create our linked GET Method:

aws apigateway put-method \
 --region us-east-1 \
 --rest-api-id 487109A-Z548 \
 --resource-id 0662807180 \
 --http-method GET \
 --request-parameters "method.request.path.somethingId=true" \
 --authorization-type "NONE" \
--endpoint-url=http://localhost:4567

We are almost there - one of the last things to do is to create our integration with the already uploaded lambda:

aws apigateway put-integration \
 --region us-east-1 \
 --rest-api-id 487109A-Z548 \
 --resource-id 0662807180 \
 --http-method GET \
 --type AWS_PROXY \
 --integration-http-method POST \
 --uri arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:000000000000:function:api-test-handler/invocations \
 --passthrough-behavior WHEN_NO_MATCH \
 --endpoint-url=http://localhost:4567

Last but not least: Deploy our API to our desired stage:

aws apigateway create-deployment \
 --region us-east-1 \
 --rest-api-id 487109A-Z548 \
 --stage-name test \
 --endpoint-url=http://localhost:4567

Now we can test it:

curl http://localhost:4567/restapis/487109A-Z548/test/_user_request_/HowMuchIsTheFish

Response:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
100    34  100    34    0     0      9      0  0:00:03  0:00:03 --:--:--     9
{"somethingId":"HowMuchIsTheFish"}

I hope this helps.

Hint 1: For easier use I recommend to install AWSCLI Local ( https://github.com/localstack/awscli-local ) - with this tool you can use the command "awslocal" and don't have to type "--endpoint-url= ..." for each command

Walkthrough for using Serverless Framework and Localstack:

You can also use the Serverless Framework (https://serverless.com/).

First install it via npm:

npm install serverless -g

Now you can create a sample application based on a nodejs-aws template:

serverless create --template aws-nodejs

In order to have an HTTP endpoint, you have to edit the serverless.yml and add the corresponding event :

functions:
  hello:
    handler: handler.hello
    events:
      - http:
          path: ping
          method: get

In order to run this against your localstack installation you have to use the serverless-localstack plugin ( https://github.com/temyers/serverless-localstack):

npm install serverless-localstack

Now you have to edit your serverless.yml again, add the plugin and adjust your endpoints. In my case localstack is running inside the Docker toolbox, so it's IP is 192.168.99.100 - you may have to change this to localhost, depending on your use:

plugins:
 - serverless-localstack
custom:
  localstack:
    debug: true
    stages:
      - local
      - dev
    host: http://192.168.99.100
    endpoints:
      S3: http://192.168.99.100:4572
      DynamoDB: http://192.168.99.100:4570
      CloudFormation: http://192.168.99.100:4581
      Elasticsearch: http://192.168.99.100:4571
      ES: http://192.168.99.100:4578
      SNS: http://192.168.99.100:4575
      SQS: http://192.168.99.100:4576
      Lambda: http://192.168.99.100:4574
      Kinesis: http://192.168.99.100:4568

Now you can try to deploy it:

serverless deploy --verbose --stage local

This will create an S3 bucket, upload your lambda and create a cloudformation stack. However, the process will fail due to some inconsistencies of localstack when compared against AWS. Don't be dismayed though, the created cloudformation template works fine and you just need an additional request and you are done:

awslocal cloudformation update-stack --template-body file://.serverless/cloudformation-template-update-stack.json --stack-name aws-nodejs-local

Now your lambda is deployed and can be tested:

curl http://192.168.99.100:4567/restapis/75A-Z278430A-Z/local/_user_request_/ping

Response:

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current Dload  Upload   Total   Spent    Left  Speed
100   364  100   364    0     0    111      0  0:00:03  0:00:03 --:--:--   111
{"message":"Go Serverless v1.0! Your function executed successfully!","input":{"body":null,"headers":{"host":"192.168.99.100:4567","accept":"*/*","user-agent":"curl/7.49.1"},"resource":"/restapis/75A-Z278430A-Z/local/_user_request_/ping","queryStringParameters":{},"httpMethod":"GET","stageVariables":{},"path":"/ping","pathParameters":{},"isBase64Encoded":false}}

Hope this helps.