CODEWITHSUNDEEP
aws-cloudformation
MikeJansen
MikeJansen

Reputation: 3476

"Importing" non-importable resources into a CloudFormation stack

With AWS CloudFormation, you can import existing resources of supported types into a new or existing Stack. Some resources such as Routes and various associations and attachments are not supported. I'm guessing many of these are not "full fledged" resources and exist behind the scenes as simply a component of another resource.

I have found that I can "fake import" an existing VPCGatewayAttachment simply by adding it to the templates and creating and executing an UPDATE ChangeSet after successfully IMPORTing the VPC and Internet Gateway via an IMPORT ChangeSet. The VPCGatewayAttachment is added with no error and also becomes a part of the Stack. In the demonstration below, you can note that the PhysicalResourceId of the VPCGatewayAttachment in question changes between its initial creation and subsequent removal and re-adding into the Stack. (NOTE: The initial creation via template is to simplify the example -- normally this would have been an existing resource not in a Stack). I'm not sure if this reflects actual destruction of the existing attachment and re-creation of a new one, or if it's simply that the attachment has no actual PhysicalResourceId but one is randomly assigned when added into a Stack.

My questions are:

  1. Is the 'fake import' of the VPCGatewayAttachment non-destructive, i.e. non-disruptive, in a production environment?

  2. If it is non-disruptive, what other resources that are not supported for Import, can also effectively be brought into the Stack in a non-disruptive manner using the same technique: simply adding the equivalent resource in the template and creating and executing an UPDATE ChangeSet. I'm thinking mainly of Routes and other associations and attachments.

Below is a demonstration of this. To run it, place the first 4 files (.ps1 and .yaml) in files with the specified name in the same directory. You must have AWS CLI installed and configured for a profile with proper permissions to manipulate stacks and resources. Run the PowerShell (.ps1) file. You probably want to replace the name of the S3 bucket with something unique. The script cleans up all created resources (stack and s3 bucket).

If you want to skip running it, I have included the output from a local run as the last file below.

case-XXXXXXXXXX-example.ps1:

echo "---------------------------------------------------------------------------"
echo "---------------------------------------------------------------------------"
echo "- Demonstration for Case XXXXXXXXXX"
echo "---------------------------------------------------------------------------"
echo "---------------------------------------------------------------------------"
echo "-"
echo "---------------------------------------------------------------------------"
echo "Create S3 bucket and upload templates"
echo "---------------------------------------------------------------------------"
aws s3api create-bucket --bucket case-XXXXXXXXXX --no-paginate --no-cli-pager
aws s3 sync . s3://case-XXXXXXXXXX --exclude * --include *.yaml --no-paginate --no-cli-pager

echo "---------------------------------------------------------------------------"
echo "- Create stack with VPC, Internet Gateway, and Gateway Attachment"
echo "-  (the latter has DeletionPolicy: Retain)"
echo "---------------------------------------------------------------------------"
echo "- Create stack"
aws cloudformation create-stack --stack-name case-XXXXXXXXXX --template-url https://case-XXXXXXXXXX.s3.amazonaws.com/case-XXXXXXXXXX-example-1.yaml --no-paginate --no-cli-pager
echo "- Wait stack create complete"
aws cloudformation wait stack-create-complete --stack-name case-XXXXXXXXXX --no-paginate --no-cli-pager
echo "- Describe stack and resources"
echo "- Note the PhysicalResourceId of the Gateway Attachment."
aws cloudformation describe-stacks --stack-name case-XXXXXXXXXX --no-paginate --no-cli-pager
aws cloudformation describe-stack-resources --stack-name case-XXXXXXXXXX --no-paginate --no-cli-pager

echo "---------------------------------------------------------------------------"
echo "- Create and execute a change-set that removes the Gateway Attachment"
echo "- This leaves us in a state simulating having IMPORTed the VPC and"
echo "- Internet Gateway, but the Gateway Attachment is not in the stack."
echo "- This sets up the next part which actually demonstrates a 'fake import'"
echo "- of the Gateway Attachment"
echo "---------------------------------------------------------------------------"
echo "- Create change-set"
aws cloudformation create-change-set --stack-name case-XXXXXXXXXX --change-set-name delete-igw-attach --template-url https://case-XXXXXXXXXX.s3.amazonaws.com/case-XXXXXXXXXX-example-2.yaml --no-paginate --no-cli-pager
echo "- Wait change-set create complete"
aws cloudformation wait change-set-create-complete --stack-name case-XXXXXXXXXX --change-set-name delete-igw-attach --no-paginate --no-cli-pager
echo "- Describe change-set"
aws cloudformation describe-change-set --stack-name case-XXXXXXXXXX --change-set-name delete-igw-attach --no-paginate --no-cli-pager
echo "- Execute change-set"
aws cloudformation execute-change-set --stack-name case-XXXXXXXXXX --change-set-name delete-igw-attach --no-paginate --no-cli-pager
echo "- Wait stack update complete"
aws cloudformation wait stack-update-complete  --stack-name case-XXXXXXXXXX --no-paginate --no-cli-pager
echo "---------------------------------------------------------------------------"
echo "- Note the Gateway Attachment is not in the stack, but the Internet Gateway"
echo "- is still attached to the VPC"
echo "---------------------------------------------------------------------------"
aws cloudformation describe-stack-resources --stack-name case-XXXXXXXXXX --no-paginate --no-cli-pager
aws ec2 describe-internet-gateways --filter "Name=tag:Name,Values=Case-XXXXXXXXXX" --no-paginate --no-cli-pager

echo "---------------------------------------------------------------------------"
echo "- THE WHOLE POINT OF THIS DEMONSTRATION IS NEXT"
echo "- 'Fake Import' the Gateway Attachment just by adding it to the template and"
echo "- creating and executing an UPDATE change-set."
echo "---------------------------------------------------------------------------"
echo "- Create change-set"
aws cloudformation create-change-set --stack-name case-XXXXXXXXXX --change-set-name fake-import-igw-attach --template-url https://case-XXXXXXXXXX.s3.amazonaws.com/case-XXXXXXXXXX-example-3.yaml --no-paginate --no-cli-pager
echo "- Wait change-set create complete"
aws cloudformation wait change-set-create-complete --stack-name case-XXXXXXXXXX --change-set-name fake-import-igw-attach --no-paginate --no-cli-pager
echo "- Describe change-set"
aws cloudformation describe-change-set --stack-name case-XXXXXXXXXX --change-set-name fake-import-igw-attach --no-paginate --no-cli-pager
echo "- Execute change-set"
aws cloudformation execute-change-set --stack-name case-XXXXXXXXXX --change-set-name fake-import-igw-attach  --no-paginate --no-cli-pager
echo "- Wait stack update complete"
aws cloudformation wait stack-update-complete  --stack-name case-XXXXXXXXXX --no-paginate --no-cli-pager
echo "---------------------------------------------------------------------------"
echo "- Note that the Gateway Attachment is now in the stack and the Internet"
echo "- Gateway is still attached, and there weren't any errors."
echo "- The PhysicalResourceId did change, however."
echo "---------------------------------------------------------------------------"
aws cloudformation describe-stack-resources --stack-name case-XXXXXXXXXX --no-paginate --no-cli-pager
aws ec2 describe-internet-gateways --filter "Name=tag:Name,Values=Case-XXXXXXXXXX" --no-paginate --no-cli-pager

echo "---------------------------------------------------------------------------"
echo "- Delete stack"
echo "---------------------------------------------------------------------------"
aws cloudformation delete-stack --stack-name case-XXXXXXXXXX --no-paginate --no-cli-pager
echo "- Wait stack delete complete"
aws cloudformation wait stack-delete-complete --stack-name case-XXXXXXXXXX --no-paginate --no-cli-pager

echo "---------------------------------------------------------------------------"
echo "- Delete S3 bucket with templates"
echo "---------------------------------------------------------------------------"
aws s3 rb s3://case-XXXXXXXXXX --force --no-paginate --no-cli-pager

echo "---------------------------------------------------------------------------"
echo "- DONE"
echo "---------------------------------------------------------------------------"

case-XXXXXXXXXX-example-1.yaml

Description: >
  Create the VPC, Internet Gateway, and attach the gateway 
  to the VPC, with a DeletionPolicy of Retain so that we can
  remove it from the stack without deleting it.  Run with
  aws cloudformation create-stack.

Resources:
  VPC:
    Type:  AWS::EC2::VPC
    Properties:
      CidrBlock: 10.187.32.0/24
      Tags:
      - Key: Name
        Value: Case-XXXXXXXXXX

  IGW:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
      - Key: Name
        Value: Case-XXXXXXXXXX

  IGWassoc:
    Type: AWS::EC2::VPCGatewayAttachment
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref IGW

case-XXXXXXXXXX-example-2.yaml

Description: >
  Delete the gateway attachment, but it will be retained 
  so we can import it next. Run with aws cloudformation
  create-change-set --change-set-type UPDATE

Resources:
  VPC:
    Type:  AWS::EC2::VPC
    Properties:
      CidrBlock: 10.187.32.0/24
      Tags:
      - Key: Name
        Value: Case-XXXXXXXXXX

  IGW:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
      - Key: Name
        Value: Case-XXXXXXXXXX

case-XXXXXXXXXX-example3.yaml

Description: >
  Create the gateway attachment.  It already exists, and is
  not importable but this action succeeds and SEEMS to be
  non-destructive. Run with aws cloudformation
  create-change-set --change-set-type UPDATE

Resources:
  VPC:
    Type:  AWS::EC2::VPC
    Properties:
      CidrBlock: 10.187.32.0/24
      Tags:
      - Key: Name
        Value: Case-XXXXXXXXXX

  IGW:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
      - Key: Name
        Value: Case-XXXXXXXXXX

  IGWassoc:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref IGW

output.txt

---------------------------------------------------------------------------
---------------------------------------------------------------------------
- Demonstration for Case XXXXXXXXXX
---------------------------------------------------------------------------
---------------------------------------------------------------------------
-
---------------------------------------------------------------------------
Create S3 bucket and upload templates
---------------------------------------------------------------------------
{
    "Location": "/case-XXXXXXXXXX"
}
upload: .\case-XXXXXXXXXX-example-2.yaml to s3://case-XXXXXXXXXX/case-XXXXXXXXXX-example-2.yaml
upload: .\case-XXXXXXXXXX-example-1.yaml to s3://case-XXXXXXXXXX/case-XXXXXXXXXX-example-1.yaml
upload: .\case-XXXXXXXXXX-example-3.yaml to s3://case-XXXXXXXXXX/case-XXXXXXXXXX-example-3.yaml
---------------------------------------------------------------------------
- Create stack with VPC, Internet Gateway, and Gateway Attachment
-  (the latter has DeletionPolicy: Retain)
---------------------------------------------------------------------------
- Create stack
{   
    "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829"
}
- Wait stack create complete
- Describe stack and resources
- Note the PhysicalResourceId of the Gateway Attachment.
{   
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
            "StackName": "case-XXXXXXXXXX",
            "Description": "Create the VPC, Internet Gateway, and attach the gateway  to the VPC, with a DeletionPolicy of Retain so that we can remove it from the stack without deleting it.  Run with aws cloudformation create-stack.\n",
            "CreationTime": "2021-11-03T15:05:03.251000+00:00",
            "RollbackConfiguration": {},
            "StackStatus": "CREATE_COMPLETE",
            "DisableRollback": false,
            "NotificationARNs": [],
            "Tags": [],
            "EnableTerminationProtection": false,
            "DriftInformation": {
                "StackDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}
{   
    "StackResources": [
        {
            "StackName": "case-XXXXXXXXXX",
            "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
            "LogicalResourceId": "IGW",
            "PhysicalResourceId": "igw-028cb469265fa34a8",
            "ResourceType": "AWS::EC2::InternetGateway",
            "Timestamp": "2021-11-03T15:05:49.880000+00:00",
            "ResourceStatus": "CREATE_COMPLETE",
            "DriftInformation": {
                "StackResourceDriftStatus": "NOT_CHECKED"
            }
        },
        {
            "StackName": "case-XXXXXXXXXX",
            "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
            "LogicalResourceId": "IGWassoc",
            "PhysicalResourceId": "case-IGWas-ZHZQ0DZ9KXLS",
            "ResourceType": "AWS::EC2::VPCGatewayAttachment",
            "Timestamp": "2021-11-03T15:06:08.293000+00:00",
            "ResourceStatus": "CREATE_COMPLETE",
            "DriftInformation": {
                "StackResourceDriftStatus": "NOT_CHECKED"
            }
        },
        {
            "StackName": "case-XXXXXXXXXX",
            "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
            "LogicalResourceId": "VPC",
            "PhysicalResourceId": "vpc-03b26a31ca1bca800",
            "ResourceType": "AWS::EC2::VPC",
            "Timestamp": "2021-11-03T15:05:28.179000+00:00",
            "ResourceStatus": "CREATE_COMPLETE",
            "DriftInformation": {
                "StackResourceDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}
---------------------------------------------------------------------------
- Create and execute a change-set that removes the Gateway Attachment
- This leaves us in a state simulating having IMPORTed the VPC and
- Internet Gateway, but the Gateway Attachment is not in the stack.
- This sets up the next part which actually demonstrates a 'fake import'
- of the Gateway Attachment
---------------------------------------------------------------------------
- Create change-set
{
    "Id": "arn:aws:cloudformation:us-east-1:606679984871:changeSet/delete-igw-attach/631b12ca-c8f4-407d-b248-b2766a730eba",
    "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829"
}
- Wait change-set create complete
- Describe change-set
{
    "ChangeSetName": "delete-igw-attach",
    "ChangeSetId": "arn:aws:cloudformation:us-east-1:606679984871:changeSet/delete-igw-attach/631b12ca-c8f4-407d-b248-b2766a730eba",
    "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
    "StackName": "case-XXXXXXXXXX",
    "CreationTime": "2021-11-03T15:06:40.618000+00:00",
    "ExecutionStatus": "AVAILABLE",
    "Status": "CREATE_COMPLETE",
    "NotificationARNs": [],
    "RollbackConfiguration": {},
    "Capabilities": [],
    "Changes": [
        {
            "Type": "Resource",
            "ResourceChange": {
                "Action": "Remove",
                "LogicalResourceId": "IGWassoc",
                "PhysicalResourceId": "case-IGWas-ZHZQ0DZ9KXLS",
                "ResourceType": "AWS::EC2::VPCGatewayAttachment",
                "Scope": [],
                "Details": []
            }
        }
    ],
    "IncludeNestedStacks": false
}
- Execute change-set
- Wait stack update complete
---------------------------------------------------------------------------
- Note the Gateway Attachment is not in the stack, but the Internet Gateway
- is still attached to the VPC
---------------------------------------------------------------------------
{
    "StackResources": [
        {
            "StackName": "case-XXXXXXXXXX",
            "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
            "LogicalResourceId": "IGW",
            "PhysicalResourceId": "igw-028cb469265fa34a8",
            "ResourceType": "AWS::EC2::InternetGateway",
            "Timestamp": "2021-11-03T15:05:49.880000+00:00",
            "ResourceStatus": "CREATE_COMPLETE",
            "DriftInformation": {
                "StackResourceDriftStatus": "NOT_CHECKED"
            }
        },
        {
            "StackName": "case-XXXXXXXXXX",
            "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
            "LogicalResourceId": "VPC",
            "PhysicalResourceId": "vpc-03b26a31ca1bca800",
            "ResourceType": "AWS::EC2::VPC",
            "Timestamp": "2021-11-03T15:05:28.179000+00:00",
            "ResourceStatus": "CREATE_COMPLETE",
            "DriftInformation": {
                "StackResourceDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}
{
    "InternetGateways": [
        {
            "Attachments": [
                {
                    "State": "available",
                    "VpcId": "vpc-03b26a31ca1bca800"
                }
            ],
            "InternetGatewayId": "igw-028cb469265fa34a8",
            "OwnerId": "606679984871",
            "Tags": [
                {
                    "Key": "aws:cloudformation:logical-id",
                    "Value": "IGW"
                },
                {
                    "Key": "Name",
                    "Value": "Case-XXXXXXXXXX"
                },
                {
                    "Key": "aws:cloudformation:stack-name",
                    "Value": "case-XXXXXXXXXX"
                },
                {
                    "Key": "aws:cloudformation:stack-id",
                    "Value": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829"
                }
            ]
        }
    ]
}
---------------------------------------------------------------------------
- THE WHOLE POINT OF THIS DEMONSTRATION IS NEXT
- 'Fake Import' the Gateway Attachment just by adding it to the template and
- creating and executing an UPDATE change-set.
---------------------------------------------------------------------------
- Create change-set
{
    "Id": "arn:aws:cloudformation:us-east-1:606679984871:changeSet/fake-import-igw-attach/95510e17-3f44-4ba4-be9e-4183cbb143ca",
    "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829"
}
- Wait change-set create complete
- Describe change-set
{
    "ChangeSetName": "fake-import-igw-attach",
    "ChangeSetId": "arn:aws:cloudformation:us-east-1:606679984871:changeSet/fake-import-igw-attach/95510e17-3f44-4ba4-be9e-4183cbb143ca",
    "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
    "StackName": "case-XXXXXXXXXX",
    "CreationTime": "2021-11-03T15:07:52.172000+00:00",
    "ExecutionStatus": "AVAILABLE",
    "Status": "CREATE_COMPLETE",
    "NotificationARNs": [],
    "RollbackConfiguration": {},
    "Capabilities": [],
    "Changes": [
        {
            "Type": "Resource",
            "ResourceChange": {
                "Action": "Add",
                "LogicalResourceId": "IGWassoc",
                "ResourceType": "AWS::EC2::VPCGatewayAttachment",
                "Scope": [],
                "Details": []
            }
        }
    ],
    "IncludeNestedStacks": false
}
- Execute change-set
- Wait stack update complete
---------------------------------------------------------------------------
- Note that the Gateway Attachment is now in the stack and the Internet
- Gateway is still attached, and there weren't any errors.
- The PhysicalResourceId did change, however.
---------------------------------------------------------------------------
{
    "StackResources": [
        {
            "StackName": "case-XXXXXXXXXX",
            "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
            "LogicalResourceId": "IGW",
            "PhysicalResourceId": "igw-028cb469265fa34a8",
            "ResourceType": "AWS::EC2::InternetGateway",
            "Timestamp": "2021-11-03T15:05:49.880000+00:00",
            "ResourceStatus": "CREATE_COMPLETE",
            "DriftInformation": {
                "StackResourceDriftStatus": "NOT_CHECKED"
            }
        },
        {
            "StackName": "case-XXXXXXXXXX",
            "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
            "LogicalResourceId": "IGWassoc",
            "PhysicalResourceId": "case-IGWas-3DBXKEM6SFPL",
            "ResourceType": "AWS::EC2::VPCGatewayAttachment",
            "Timestamp": "2021-11-03T15:08:48.657000+00:00",
            "ResourceStatus": "CREATE_COMPLETE",
            "DriftInformation": {
                "StackResourceDriftStatus": "NOT_CHECKED"
            }
        },
        {
            "StackName": "case-XXXXXXXXXX",
            "StackId": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829",
            "LogicalResourceId": "VPC",
            "PhysicalResourceId": "vpc-03b26a31ca1bca800",
            "ResourceType": "AWS::EC2::VPC",
            "Timestamp": "2021-11-03T15:05:28.179000+00:00",
            "ResourceStatus": "CREATE_COMPLETE",
            "DriftInformation": {
                "StackResourceDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}
{
    "InternetGateways": [
        {
            "Attachments": [
                {
                    "State": "available",
                    "VpcId": "vpc-03b26a31ca1bca800"
                }
            ],
            "InternetGatewayId": "igw-028cb469265fa34a8",
            "OwnerId": "606679984871",
            "Tags": [
                {
                    "Key": "aws:cloudformation:logical-id",
                    "Value": "IGW"
                },
                {
                    "Key": "Name",
                    "Value": "Case-XXXXXXXXXX"
                },
                {
                    "Key": "aws:cloudformation:stack-name",
                    "Value": "case-XXXXXXXXXX"
                },
                {
                    "Key": "aws:cloudformation:stack-id",
                    "Value": "arn:aws:cloudformation:us-east-1:606679984871:stack/case-XXXXXXXXXX/6bf590e0-3cb7-11ec-b30d-0a5d84963829"
                }
            ]
        }
    ]
}
---------------------------------------------------------------------------
- Delete stack
---------------------------------------------------------------------------
- Wait stack delete complete
---------------------------------------------------------------------------
- Delete S3 bucket with templates
---------------------------------------------------------------------------
delete: s3://case-XXXXXXXXXX/case-XXXXXXXXXX-example-3.yaml
delete: s3://case-XXXXXXXXXX/case-XXXXXXXXXX-example-2.yaml
delete: s3://case-XXXXXXXXXX/case-XXXXXXXXXX-example-1.yaml
remove_bucket: case-XXXXXXXXXX
---------------------------------------------------------------------------
- DONE
---------------------------------------------------------------------------

Upvotes: 0

Views: 975

Answers (1)

MikeJansen
MikeJansen

Reputation: 3476

Tried this with an AWS::EC2::Route and it failed with "route already exists."

So while I might be able to get away with brute forcing a VPCGatewayAttachment into the Stack, I won't be able to with Routes, and likely other resource types.

The amount of time to investigate what resources might be possible is not worth the effort for an undocumented approach.

The best way forward for getting resources into a Stack that don't support import will be to have a script that deletes existing non-importable resources and re-creates them with a change-set. This will have to be done during a maintenance window as there will certainly be outages in a non-redundant system.

Upvotes: 0

Related Questions