How to update a file application.properties in EC2 instance from an external file (e.g from S3 bucket) using CloudFormation script?

Question

I can update the application.properties and restart the service using the following CloudFormation template. But I want to update the application.properties file from an external file (e.g: S3 or git) instead of a script.

How can I achieve this?

My CF template,

AWSTemplateFormatVersion: '2010-09-09'
Description: AWS CloudFormation Template with EC2InstanceWithSecurityGroup
Parameters:
  KeyName:
    Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
    Type: AWS::EC2::KeyPair::KeyName
    ConstraintDescription: must be the name of an existing EC2 KeyPair.
  InstanceType:
    Description: EC2 instance type
    Type: String
    Default: t2.medium
    AllowedValues:
      - t2.medium
      - t2.large
    ConstraintDescription: must be a valid EC2 instance type.
  RemoteAccessLocation:
    Description: The IP address range that can be used to access to the EC2 instances
    Type: String
    MinLength: '9'
    MaxLength: '18'
    Default: 0.0.0.0/0
    AllowedPattern: (\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})
    ConstraintDescription: must be a valid IP CIDR range of the form x.x.x.x/x.
Resources:
  S3Bucket:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Retain
    Properties:
      BucketName: my-test-bucket
      AccessControl: Private 
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: !Ref 'InstanceType'
      SecurityGroups:
        - !Ref 'InstanceSecurityGroup'
      KeyName: !Ref 'KeyName'
      ImageId: ami-xxxxxxxxxxxxxxxx
      UserData:
        Fn::Base64: !Sub |
              #!/bin/bash -ex

              cat >/usr/local/application.properties <<EOL
              amazon.s3.bucket-name=${S3Bucket}
              amazon.s3.region=ap-south-1
              amazon.s3.access-key=xxxxxxxxxxxxxxxxxxx
              amazon.s3.secret-key=yyyyyyyyyyyyyyyyxxxxxxxxx

              ## H2 Config
              spring.datasource.url=jdbc:h2:mem:testdb
              spring.datasource.driverClassName=org.h2.Driver
              spring.datasource.username=username
              spring.datasource.password=password
              spring.jpa.show-sql=true
              spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.H2Dialect
              spring.jpa.hibernate.ddl-auto=update
              EOL

              ## Restart our service
              sudo systemctl restart myapplication.service

  InstanceSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Enable SSH (22), HTTP (8080)
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: '22'
          ToPort: '22'
          CidrIp: !Ref 'RemoteAccessLocation'
        - CidrIp: 0.0.0.0/0
          FromPort: '8080'
          IpProtocol: tcp
          ToPort: '8080'
Outputs:
  InstanceId:
    Description: InstanceId of the newly created EC2 instance
    Value: !Ref 'EC2Instance'
  AZ:
    Description: Availability Zone of the newly created EC2 instance
    Value: !GetAtt 'EC2Instance.AvailabilityZone'
  PublicDNS:
    Description: Public DNSName of the newly created EC2 instance
    Value: !GetAtt 'EC2Instance.PublicDnsName'
  PublicIP:
    Description: Public IP address of the newly created EC2 instance
    Value: !GetAtt 'EC2Instance.PublicIp'

Any inputs here really appreciated.

Answer 1

You have to create AWS::IAM::InstanceProfile with AWS::IAM::Role that has read permissions to S3 or to a specific bucket/object. Then you use aws CLI in your user data to get the file from s3 into your instance. I don't know what is your AMI, but if its standard Amazon Linux 2 or Ubuntu, it will have AWS CLI already installed.

AWSTemplateFormatVersion: '2010-09-09'
Description: AWS CloudFormation Template with EC2InstanceWithSecurityGroup
Parameters:
  KeyName:
    Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
    Type: AWS::EC2::KeyPair::KeyName
    ConstraintDescription: must be the name of an existing EC2 KeyPair.
  InstanceType:
    Description: EC2 instance type
    Type: String
    Default: t2.medium
    AllowedValues:
      - t2.medium
      - t2.large
    ConstraintDescription: must be a valid EC2 instance type.
  RemoteAccessLocation:
    Description: The IP address range that can be used to access to the EC2 instances
    Type: String
    MinLength: '9'
    MaxLength: '18'
    Default: 0.0.0.0/0
    AllowedPattern: (\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})
    ConstraintDescription: must be a valid IP CIDR range of the form x.x.x.x/x.
Resources:

  S3Bucket:
    Type: 'AWS::S3::Bucket'
    DeletionPolicy: Retain
    Properties:
      BucketName: my-test-bucket
      AccessControl: Private 

  MyInstanceRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'               
        Statement:
          - Effect: Allow
            Principal: {'Service': ['ec2.amazonaws.com']}
            Action: ['sts:AssumeRole']
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess   
      Path: '/'  

  MyInstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties: 
      Roles: 
        - !Ref MyInstanceRole

  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: !Ref 'InstanceType'
      SecurityGroups:
        - !Ref 'InstanceSecurityGroup'
      KeyName: !Ref 'KeyName'
      ImageId: ami-xxxxxxxxxxxxxxxx
      IamInstanceProfile: !Ref MyInstanceProfile
      UserData:
        Fn::Base64: !Sub |
              #!/bin/bash -ex

              aws s3 cp s3://<your-bucket-with-settings>/application.properties /usr/local/application.properties

              ## Restart our service
              sudo systemctl restart myapplication.service

  InstanceSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Enable SSH (22), HTTP (8080)
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: '22'
          ToPort: '22'
          CidrIp: !Ref 'RemoteAccessLocation'
        - CidrIp: 0.0.0.0/0
          FromPort: '8080'
          IpProtocol: tcp
          ToPort: '8080'

Outputs:
  InstanceId:
    Description: InstanceId of the newly created EC2 instance
    Value: !Ref 'EC2Instance'
  AZ:
    Description: Availability Zone of the newly created EC2 instance
    Value: !GetAtt 'EC2Instance.AvailabilityZone'
  PublicDNS:
    Description: Public DNSName of the newly created EC2 instance
    Value: !GetAtt 'EC2Instance.PublicDnsName'
  PublicIP:
    Description: Public IP address of the newly created EC2 instance
    Value: !GetAtt 'EC2Instance.PublicIp'