CODEWITHSUNDEEP
jsonamazon-web-servicesamazon-dynamodbaws-cloudformation
gcg27
gcg27

Reputation: 77

Is there a way to populate an Amazon DynamoDB table from a CloudFormation template?

I need to create a DynamoDB table were one of the attribute values its passed as a parameter in a CloudFormation template:

The DynamoDB table should look like this:

PhoneNumber     |       OrderNumber

223546421               11545154
784578745               11547854
223458784               11547487
XXXXXXXXX               11578451

The attribute value "XXXXXXXXX" needs to be passed as a parameter from a CloudFormation template were the DynamoDB table is going to be created and populated with the information above.

This is the current CF template which builds the DynamoDB table:

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Resources": {
        "OrdersTable": {
            "Type": "AWS::DynamoDB::Table",
            "Properties": {
                "TableName": "ClientOrders",
                "AttributeDefinitions": [
                    {
                        "AttributeName": "PhoneNumber",
                        "AttributeType": "S"
                    },
                    {
                        "AttributeName": "OrderNumber",
                        "AttributeType": "S"
                    }
                ],
                "KeySchema": [
                    {
                        "AttributeName": "PhoneNumber",
                        "KeyType": "HASH"
                    },
                    {
                        "AttributeName": "OrderNumber",
                        "KeyType": "RANGE"
                    }
                ],
                "TimeToLiveSpecification": {
                    "AttributeName": "ExpirationTime",
                    "Enabled": true
                },
                "ProvisionedThroughput": {
                    "ReadCapacityUnits": "10",
                    "WriteCapacityUnits": "5"
                }
            },
            "DependsOn": [
                "DynamoDBQueryPolicy"
            ]
        },
        "DynamoDBQueryPolicy": {
            "Type": "AWS::IAM::Policy",
            "Properties": {
                "PolicyName": "DynamoDBQueryPolicy",
                "PolicyDocument": {
                    "Version": "2012-10-17",
                    "Statement": [
                        {
                            "Effect": "Allow",
                            "Action": "dynamodb:Query",
                            "Resource": "*"
                        }
                    ]
                },
                "Roles": [
                    {
                        "Ref": "OrdersTableQueryRole"
                    }
                ]
            }
        },
        "OrdersTableQueryRole": {
            "Type": "AWS::IAM::Role",
            "Properties": {
                "AssumeRolePolicyDocument": {
                    "Version": "2012-10-17",
                    "Statement": [
                        {
                            "Effect": "Allow",
                            "Principal": {
                                "Service": [
                                    "dynamodb.amazonaws.com"
                                ]
                            },
                            "Action": [
                                "sts:AssumeRole"
                            ]
                        }
                    ]
                },
                "Path": "/"
            }
        }
    }
}

I need help to populate the table with the mentioned values above and how to pass mentioned attribute value as a parameter.

Any help on this would be appreciated.

Thanks

Upvotes: 3

Views: 1658

Answers (2)

Balu Vyamajala
Balu Vyamajala

Reputation: 10333

We need custom cloudformation resource to add entries into Dynamo Table.

From Docs:

With Lambda functions and custom resources, you can run custom code in response to stack events (create, update, and delete).

This custom resource invokes a Lambda function and sends it the StackName property as input. The function uses this property to get outputs from the appropriate stack.

Here is a sample I created for 1 key, need some minor changes:

We need two things.

One, a stack which holds the Lambda function. With this we not have a cloudformation resource which calls this Lambda behind the scenes when we add/remove/update a resource. This typically goes into separate stack all together.

AWSTemplateFormatVersion: '2010-09-09'
Resources:
    DynamoCfnLambdaRole:
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Version: '2012-10-17'
                Statement:
                    - Effect: Allow
                      Principal:
                          Service:
                              - lambda.amazonaws.com
                      Action:
                          - sts:AssumeRole
            Path: '/'
            Policies:
                - PolicyName: dynamodbAccessRole
                  PolicyDocument:
                      Version: '2012-10-17'
                      Statement:
                          - Effect: Allow
                            Action:
                                - dynamodb:*
                            Resource: '*'
                          - Effect: Allow
                            Action:
                                - logs:*
                            Resource: '*'
    CfnCrtUpdDltDynamodbDocumentLambda:
        Type: AWS::Lambda::Function
        Properties:
            FunctionName: 'cfn-crt-upd-dlt-dynamodb-document'
            Code:
                ZipFile: >
                    const AWS = require("aws-sdk");
                    const response = require("./cfn-response");
                    const docClient = new AWS.DynamoDB.DocumentClient();
                    exports.handler = function(event, context) {
                      console.log(JSON.stringify(event, null, 2));
                      var item = JSON.parse(event.ResourceProperties.DynamoItem);
                      var keyProperty = event.ResourceProperties.DynamoKeyProperty;
                      var tableName = event.ResourceProperties.DynamoTableName;
                      if (event.RequestType == "Create" || event.RequestType == "Update") {
                        console.log("item:", item);
                        var params = {
                          TableName: tableName,
                          Item: item
                        };
                        console.log("Creating or Updating Document");
                        docClient.put(params, function(err, data) {                
                          if (err) {
                            console.log('error creating/updating document', err);
                            response.send(event, context, "FAILED", {}, tableName + '_' + item[keyProperty]);
                          } else {
                            response.send(event, context, "SUCCESS", {}, tableName + '_' + item[keyProperty]);
                          }
                        });
                      }

                      if (event.RequestType == "Delete") {
                        console.log("Deleting a Document");
                        var params = {
                          TableName: tableName,
                          Key: {
                            [keyProperty]: item[keyProperty]
                          }
                        };
                        docClient.delete(params, function(err, data) {
                          if (err) {
                            response.send(event, context, "FAILED", {});
                          } else {
                            response.send(event, context, "SUCCESS", {});
                          }
                        });
                      }
                    };
            Handler: index.handler
            Role: !GetAtt DynamoCfnLambdaRole.Arn
            Runtime: nodejs10.x
            Timeout: 60

Second, we need to add resource blocks Custom::ThisCouldBeAnything with ServiceToken pointing to Lambda Arn we created above. Whenever we

  • Add a new block in any cloudformation, Lambda will be called with Create, which creates an entry in to Dynamo table.
  • Remove the block from Cloudformation, Lambda will be called with Delete, which delete entry from Dynamo Table.
  • Change something in resource block, example change something in DynamoItem, Lambda will be called with Update.

We need one resource block for every record that we need to insert into Dynamo.

Resources:
    MyPhone223546421:
        Type: Custom::CrtUpdDltDynamodbDocumentLambda
        Properties:
            ServiceToken: !GetAtt CfnCrtUpdDltDynamodbDocumentLambda.Arn
            DynamoTableName: My-Table
            DynamoKeyProperty: 'PhoneNumber'
            DynamoItem: |
                {
                  "PhoneNumber": "223546421",
                  "OrderNumber": "11545154",
                  "someKey": "someValue"
                }
    MyPhone784578745:
        Type: Custom::CrtUpdDltDynamodbDocumentLambda
        Properties:
            ServiceToken: !GetAtt CfnCrtUpdDltDynamodbDocumentLambda.Arn
            DynamoTableName: My-Table
            DynamoKeyProperty: 'PhoneNumber'
            DynamoItem: |
                {
                  "PhoneNumber": "784578745",
                  "OrderNumber": "11547854",
                  "someKey": "someValue"
                }

Upvotes: 3

Jason Wadsworth
Jason Wadsworth

Reputation: 8887

There isn't a way, natively, to add rows to a DynamoDB table via CloudFormation. CloudFormation is a tool designed to create/manage infrastructure, not data. I would recommend doing that in a separate process. For example, if you are deploying via CodePipeline you could have a CodeBuild job that runs after the CloudFormation is created. That code can do just about anything you want it to do.

There is the option of creating either a custom resource or a resource provider, but while it would be possible I don't think this is within its intended purpose.

Upvotes: 1

Related Questions