CloudFormation GUI Parameter Override

Question

I have a CloudFormation template with 2 parameters. One for the EC2 name and the 2nd for an URL. The UserData curls the necessary URL on the EC2 after it spins up. This URL changes based on the user's needs. I added tags to ensure the CloudFormation was picking up the change. When I do an update through the GUI, both the tags are updated. However, the URL in the UserData's curl does not. I am trying to create instructions on how to update this template via the CloudFormation GUI. Can the parameter override be done in the UserData section for the URL parameter via the GUI? Here is a snippet of my CloudFormation template.

{
"Parameters":{
    "ArtifactURL" : {
        "Type" : "String",
        "Default" : "https://stuff.war",
        "Description": "Enter the Artifact's URL to be installed on this EC2.  Default is the latest stuff"
    },
    "EC2Name" : {
        "Type" : "String",
        "Default" : "MyCloudFormationInstance"
    }
},
"Resources" : {
    "MyCloudFormationInstance" : {
        "Type" : "AWS::EC2::Instance",
        "Properties" : {
            "AvailabilityZone" : "...",
            "ImageId" : "...",
            "SecurityGroupIds" :[ "..." ],
            "KeyName" : "...",
            "Tags" : [
                {
                    "Key" : "Name",
                    "Value" : { "Ref": "EC2Name"}
                },
                {
                    "Key" : "Url",
                    "Value" : { "Ref": "ArtifactURL"}
                }
            ],
            "IamInstanceProfile" : "...",
            "InstanceType" : "t2.micro",
            "UserData" : {"Fn::Base64": {"Fn::Join" : ["", [
                "#!/bin/bash","\n",
                "warlink=\"", { "Ref": "ArtifactURL" },"\"\n",
                "echo \"warlink: $warlink\" > myFile.txt","\n",
                "\n","curl \"$warlink\" -O","\n",

Answer 1

Here is the edited final code

    {
"Parameters":{
    "ArtifactURL" : {
        "Type" : "String",
        "Default" : "https://stuff.war",
        "Description": "Enter the Artifact's URL"
    },
    "EC2Name" : {
        "Type" : "String",
        "Default" : "MyCloudFormationInstance"
    }
},
"Resources" : {
    "WaitHandle" : { 
        "Type" : "AWS::CloudFormation::WaitConditionHandle"
     },
    "MyCloudFormationInstance" : {
        "Type" : "AWS::EC2::Instance",
        "Metadata" : {
            "ArtifactURL": { "Ref": "ArtifactURL"},
            "AWS::CloudFormation::Init" : {
                "configSets": {
                    "myconfigs" : ["config-cfn-hup","install-artifact"]
                },
                "install-artifact" : {
                    "commands": {
                        "1-run-script" : {
                            "command" : { "Fn::Join" : [ "", [
                                "/bin/bash /etc/downloadandinstallfile.sh"
                            ]]}
                        }
                    }
                },
                "config-cfn-hup" : {
                    "files" : {
                        "/etc/downloadandinstallfile.sh" : {
                            "content" : { "Fn::Join" : ["", [
                                "#!/bin/bash","\n",
                                "warlink=\"", { "Ref": "ArtifactURL" },"\"\n",
                                "echo \"warlink: $warlink\" > myFile.txt","\n",
                                "\n","curl \"$warlink\" -O","\n"]]},
                            "mode" : "000777",
                            "owner" : "root",
                            "group" : "root"
                        },
                        "/etc/cfn/cfn-hup.conf" : {
                            "content" : { "Fn::Join" : ["", [
                                "[main]\n",
                                "stack=", { "Ref" : "AWS::StackId" }, "\n",
                                "region=", { "Ref" : "AWS::Region" }, "\n",
                                "interval=1\n"
                            ]]},
                            "mode"  : "000400",
                            "owner" : "root",
                            "group" : "root"
                        },
                        "/etc/cfn/hooks.d/cfn-auto-reloader.conf" : {
                            "content" : { "Fn::Join" : ["", [
                                "[cfn-auto-reloader-hook]\n",
                                "triggers=post.update\n",
                                "path=Resources.MyCloudFormationInstance.Metadata.AWS::CloudFormation::Init\n",
                                "action=cfn-init -v --configsets myconfigs --stack ", { "Ref" : "AWS::StackId" }, " --resource MyCloudFormationInstance ", " --region ", { "Ref" : "AWS::Region"},"\n",
                                "runas=root\n"
                            ]]}
                        },
                        "/etc/systemd/system/cfn-hup.service" : {
                            "content" : { "Fn::Join" : ["", [
                                "[Unit]\n",
                                "Description=CloudFormation helper daemon\n",
                                "\n",
                                "[Service]\n",
                                "ExecStart=/bin/cfn-hup\n",
                                "Restart=always\n",
                                "Type=simple\n",
                                "\n",
                                "[Install]\n",
                                "WantedBy=multi-user.target"
                            ]]}
                        }
                    },
                    "commands" : {
                        "01-install-set" : {
                            "command" : { "Fn::Join" : [ "", [
                                "set -xe\n"
                            ]]}
                        },
                        "02-daemon-reload" : {
                            "command" : { "Fn::Join" : [ "", [
                                "systemctl daemon-reload"
                            ]]}
                        },
                        "03-enable-service" : {
                            "command" : { "Fn::Join" : [ "", [
                                "systemctl enable cfn-hup.service\n"
                            ]]}
                        },
                        "04-start-service" : {
                            "command" : { "Fn::Join" : [ "", [
                                "service cfn-hup.service start\n"
                            ]]}
                        }
                    }
                }
            }
        },
        "Properties" : {
            "AvailabilityZone" : "xxxxxxxx",
            "ImageId" : "xxxxxxxxxx",
            "SecurityGroupIds" :[ "xxxxxxx" ],
            "KeyName" : "xxx.pem",
            "Tags" : [
                {
                    "Key" : "Name",
                    "Value" : { "Ref": "EC2Name"}
                },
                {
                    "Key" : "Url",
                    "Value" : { "Ref": "NexusArtifactURL"}
                }
            ],
            "IamInstanceProfile" : "ec2LoggingRole",
            "InstanceType" : "t2.micro",
            "UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
                "#!/bin/bash -v\n",
                "# Function to return error code to the wait handle\n",
                "function handle_error\n",
                "{\n",
                " cfn-signal -e 1 -r \"$1\" '", { "Ref" : "WaitHandle"}, "'\n",
                " exit 1\n",
                "}\n",

                "yum update -y \n",
                "pip3 install pystache\n",
                "pip3 install argparse\n",
                "pip3 install python-daemon\n",
                "pip3 install requests\n",
                "curl -sSL https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz -O\n",
                "# tar -xpf aws-cfn-bootstrap-latest.tar.gz -C /opt\n",
                "# cd /opt/aws-cfn-bootstrap-1.4/\n",
                "# sudo python setup.py build\n",
                "# sudo python setup.py install\n",
                "easy_install aws-cfn-bootstrap-latest.tar.gz\n",
                "#ln -s /usr/init/redhat/cfn-hup /etc/init.d/cfn-hup\n",
                "#chmod 775 /usr/init/redhat/cfn-hup\n",
                "#cd /opt\n",
                "#mkdir aws\n",
                "#cd aws\n",
                "#mkdir bin\n",
                "#ln -s /usr/bin/cfn-hup /opt/aws/bin/cfn-hup\n",
                
                "sudo cfn-init -v -c myconfigs -s ", { "Ref" : "AWS::StackId" }, " -r MyCloudFormationInstance ", " --region ", { "Ref" : "AWS::Region" }, " || handle_error 'Failed to run cfn-init'\n",

                "# Return success\n",

                "cfn-signal -e 0 -r \"Stack Complete\" '", { "Ref" : "WaitHandle" }, "'\n"
                ]]}
            }
        }
    }
},
"Outputs":{
    "InstanceID" : {
        "Description" : "The Instance ID",
        "Value": { "Ref" : "MyCloudFormationInstance" }
    },
    "PublicIp" : {
        "Description" : "Instance's Public Ip",
        "Value" : { "Fn::GetAtt" : [ "MyCloudFormationInstance", "PublicIp"]}
    },
    "SSHLogin" : {
        "Description" : "SSH command to log into your instance",
        "Value" : {"Fn::Join" : ["", ["ssh -i \"xxx.pem\" ec2-user@", {"Fn::GetAtt": ["MyCloudFormationInstance","PublicDnsName"]}]]}
    }
}}

Answer 2

I'm going to make some assumptions here:

• I assume your goal is that an update to the CloudFormation parameter ArtifactURL should make the instance download the new URL;
• Also, I assume that you didn't directly verify whether UserData was updated or not, but you indirectly verified it by noticing that the new URL wasn't downloaded and thought the reason was that UserData wasn't updated.

If those assumptions are invalid, please clarify in the question what your end goal is and what exactly you observed.

The ArtifactURL in the UserData script most likely is being updated — it's just how CloudFormation is supposed to work. What you probably observed is that the new URL is not being downloaded, which is quite different and expected.

EC2 instances support having their UserData modified, and CloudFormation supports making that change. However, by default in most AMIs, the OS (and, specifically, the cloud-init package) is configured to only run the UserData script once-per-instance (rather than once-per-boot).

This means that, by default, changes made to the UserData script will not execute.

If you look at the CloudFormation documentation for resource type AWS::EC2::Instance, you'll see that both the properties Tags and UserData can be updated without replacement. Specifically, Tags is updated without any interruption at all, while UserData is updated with a Stop -> Start cycle. When the instance reboots, the cloud-init package verifies that it has already executed UserData for that instance once before, so it doesn't execute it again — regardless of the fact that it's been modified.

So, what you need is to trigger the re-execution of UserData when it is modified.

There are a variety of ways to make that happen. Since you're using CloudFormation, I'd advise you to use cfn-hup:

The cfn-hup helper is a daemon that detects changes in resource metadata and runs user-specified actions when a change is detected.

This tool was specifically designed to detect changes in UserData and re-run them. There's an example in the linked documentation page, and you'll also find plenty of other examples around by searching for cfn-hup.

As a final note, there are a few other ways to achieve the same goal (re-executing UserData scripts if they are modified, or every boot, etc). Another good option is to learn more about cloud-init and modify its configuration. Always make sure that the option you read about is documented and supported by the infrastructure or software you're using (e.g., you'll find a lot of places suggesting that you can remove a lock file to force re-execution of UserData, but that's bad advice).