AWS Boto3 Python run_instances method not parsing UserData parameter

Question

The command runs and does create the instance, I can log in, run updates, etc. but the UserData file is not passed in.

Here is my script:

#!/usr/bin/python

import boto3



def main():


    dev_server_ami_id = 'ami-0b33d91d'  # This is currently the Amazon Linux base AMI.
    dev_server_sec_group = 'xxxxxxxxxx'
    dev_server_az = 'us-east-1a'
    dev_server_subnet_id = 'xxxxxxxxxxx'
    dev_server_name = 'test_server_name'
    dev_instance_type = 't2.large'
    slash_sites_size = 16
    slash_scratch_size = 5

    ec2client = boto3.client('ec2',aws_access_key_id='asdfasdfasdf',aws_secret_access_key='asdfasdfasdfasdf')

    creation_response = ec2client.run_instances(DryRun=False,MinCount=1,ImageId=dev_server_ami_id,
        MaxCount=1,KeyName='mcp_demo_dev',SecurityGroupIds=[dev_server_sec_group],
        InstanceType=dev_instance_type,Placement={'AvailabilityZone': dev_server_az},SubnetId=dev_server_subnet_id,UserData="file://C:\\Users\\xxxxx\\Dev\\Site Where My Script Is\\base_server_bootstrap.sh",
        BlockDeviceMappings=[{'DeviceName':'/dev/xvdb','Ebs':{'VolumeSize':slash_sites_size,'DeleteOnTermination':True}},
            {'DeviceName':'/dev/xvdc','Ebs':{'VolumeSize':slash_scratch_size,'DeleteOnTermination':True}}])
    instance_id = creation_response['Instances'][0]['InstanceId']
    ec2client.create_tags(Resources=[instance_id,],Tags=[{'Key':'Name','Value': dev_server_name,},],)



if __name__ == "__main__": main()

This is all a work in progress as far as the script is concerned and I've tried a few different options for passing in the file including just the filename since the python script and my shell script are in the same directory, passing in just the shell script name without the full path with the "file://" in front of it, as well as the full path and script name with no "file://" in front of it.

Any tricks would be appreciated, it seems that the run_instances method is just ignoring that parameter.

For reference here is the shell script I am trying to pass into the run_instance() method. It's not being called.

#!/bin/bash

#
# These variables will be used to create directories and name everything client specific.
# All files that are pulled from S3 have to follow the naming convention and are client specific.
#

clientName="demo"
# If both author and public are true then we are in DEV.
magnoliaPublic=true 
magnoliaAuthor=true
# Set this if the client is doing light-module development work.
lightModule=true
lightModuleFileName="one-pager-module.zip"

# build the additional filesytems and mount points
sudo mkfs -t ext4 /dev/xvdb
sudo mkfs -t ext4 /dev/xvdc
sudo mkdir /sites
sudo mkdir /scratch
sudo mount /dev/xvdb /sites
sudo mount /dev/xvdc /scratch

# add them to the fstab so that they will be there after a reboot
sudo cat /etc/fstab > /home/ec2-user/fstab_temp
sudo echo -e "/dev/xvdb\t/sites\text4\tdefaults,nofail\t0\t2" >> /home/ec2-user/fstab_temp
sudo echo -e "/dev/xvdc\t/scratch\text4\tdefaults,nofail\t0\t2" >> /home/ec2-user/fstab_temp
sudo cp /home/ec2-user/fstab_temp /etc/fstab
sudo rm /home/ec2-user/fstab_temp

# Set up of the base environment with Java and Tomcat.

# Need to figure out how to set the specific version of the JDK that we install.  Either copy it to S3 or direct it through yum.
sudo yum -y install java-1.8.0-openjdk
sudo groupadd tomcat
sudo useradd -g tomcat tomcat
sudo wget -O /home/tomcat/apache-tomcat-8.5.9.tar.gz http://mirror.stjschools.org/public/apache/tomcat/tomcat-8/v8.5.9/bin/apache-tomcat-8.5.9.tar.gz
sudo tar -xf /home/tomcat/apache-tomcat-8.5.9.tar.gz -C /opt
sudo rm /home/tomcat/apache-tomcat-8.5.9.tar.gz
sudo chown -R tomcat:tomcat /opt/apache-tomcat-8.5.9/

# Create our individual JVM directory structure.
sudo mkdir /sites/
sudo mkdir /sites/${clientName}
sudo mkdir /sites/${clientName}/magnolia-base/
sudo mkdir /sites/${clientName}/light-module/
# If there is content to deploy to the light-module directory then grab it.
if $lightModule; then
    sudo aws s3 cp s3://mcp-${clientName}-light-module/${lightModuleFileName} /sites/${clientName}/light-module
    sudo unzip /sites/${clientName}/light-module/${lightModuleFileName} -d /sites/${clientName}/light-module/
fi
# Build the base tomcat directories for this client.
sudo mkdir /sites/${clientName}/magnolia-base/common /sites/${clientName}/magnolia-base/conf /sites/${clientName}/magnolia-base/logs /sites/${clientName}/magnolia-base/server /sites/${clientName}/magnolia-base/shared /sites/${clientName}/magnolia-base/temp /sites/${clientName}/magnolia-base/work
# Copy configs down from S3. mcp-demo-configs
sudo aws s3 cp s3://mcp-${clientName}-configs/${clientName}_conf.zip /sites/${clientName}/magnolia-base/
sudo unzip /sites/${clientName}/magnolia-base/${clientName}_conf.zip -d /sites/${clientName}/magnolia-base/

# Set one or more appBase directories and copy our Magnolia WAR files in.
if $magnoliaPublic; then
    sudo mkdir /sites/${clientName}/magnolia-base/webapps_public
    sudo aws s3 cp s3://mcp-${clientName}-magnolia-wars/demo-mcpLive-2.3.war /sites/${clientName}/magnolia-base/webapps_public
fi
if $magnoliaAuthor; then
    sudo mkdir /sites/${clientName}/magnolia-base/webapps_author
    sudo aws s3 cp s3://mcp-${clientName}-magnolia-wars/demo-mcpEdit-2.3.war /sites/${clientName}/magnolia-base/webapps_author
fi

sudo chown -R tomcat:tomcat /sites

# From here on out everything is done as the tomcat user.
sudo su - tomcat

# Set up the environment variables.
echo export "JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.121-0.b13.29.amzn1.x86_64" >> /home/tomcat/.bash_profile
echo export JRE_HOME=\$JAVA_HOME/jre  >> /home/tomcat/.bash_profile

# Create our Tomcat setenv.sh file
touch /opt/apache-tomcat-8.5.9/bin/setenv.sh
echo export CATALINA_OPTS="$CATALINA_OPTS -XX:MaxPermSize=256m -Xms64M -Xmx1024M -Djava.awt.headless=true" >> /opt/apache-tomcat-8.5.9/bin/setenv.sh
echo export CATALINA_HOME=/opt/apache-tomcat-8.5.9 >> /opt/apache-tomcat-8.5.9/bin/setenv.sh
echo export CATALINA_BASE=/sites/${clientName}/magnolia-base >> /opt/apache-tomcat-8.5.9/bin/setenv.sh
chmod 755 /opt/apache-tomcat-8.5.9/bin/setenv.sh

sudo -S -u tomcat -i /bin/bash -l -c '/opt/apache-tomcat-8.5.9/bin/startup.sh'

Thanks in advance.

Answer 1

Pass the script contents as a string to UserData

eg:

ec2client.run_instances(,...,UserData=open("C:\\Users\\xxxxx\\Dev\\Site Where My Script Is\\base_server_bootstrap.sh").read(),...)

Answer 2

Your user data is specified incorrectly. The first line of your user data should specify whether it is a shell script or a cloud init script.

For a bash script, the first line should be:

#!/bin/bash

User data shell scripts must start with the #! characters and the path to the interpreter you want to read the script (commonly /bin/bash). For a great introduction on shell scripting, see the BASH Programming HOW-TO at the Linux Documentation Project (tldp.org).

For cloud init

#cloud-config

The cloud-init user directives can be passed to an instance at launch the same way that a script is passed, although the syntax is different. For more information about cloud-init, go to http://cloudinit.readthedocs.org/en/latest/index.html.