How to programmatically/remotely execute a program in EC2 Windows instance

Question

I'd like to launch an EC2 Windows instance, upload an EXEecutable & execute it (all in an automated fashion, this is important)

So far I was able to programmatically launch EC2 Windows instance & get its parameters (password / IP), now I'd like to find a way to upload this Executable (from my Windows machine or from my other EC2 linux instance) & run it.

I thought about launching an RDP connection & using a macro software to upload & execute the file, but based on previous experiences this is a poor/fragile approach to say the least.

I also thought about uploading this EXE to a server, then do something like this on Windows:

wget http://www.domain.com/my-file.exe

Except that Windows doesn't have wget!

So my question is: is there a way to programmatically upload & execute an EXEcutable in EC2 Windows instance?

Answer 1

I had do a similar automation for AWS for enterprise deployment back in 2011. Amazon cloud formation and opsworks where still under construction then.However we had successfully done deployment automation using dotnet for both linux and windows platform.The powershell and ftp model where ruled out becasue it was an enterprise environment and there was port restriction. Below are the approaches i used.

Note:This was asp.net web application

For linux deployment.

We used an open source project called sharpshell(sharpSSH).This is c# application which simpulates shell communication between windows and linux.Just need to supply the target aws address and security key to connect. We customized the app around our requirement

For windows

Having said that cloud formation API where still not available the and less documentation available by AWS. We used a workaround, webservice approach. Crearted a webservice which basically uploads a file to server and deploy. With this webserver hosted on amazaon windows server.Created a base image out of this and certificate. Finally a new instances created out of this image would have webservices hosted which could be called to upload the deployment package and install on that system.

Though the above startergies where not fool proof and had less controll. we achieved the functionality of cross platform direct deployment from windows to linux without using S3 buckets or powershell.

Please let me know if you require any clarification.

Cheers! Charles

Answer 2

The command ec2-run-instances has two additional arguments that can be used when running the command. The user-data command and user-data-file both of these perform the same task just they read from different input. When you use this argument the contents of the user-data will be uploaded to a amazon hosted URI http://169.254.169.254/1.0/user-data only available to the instance that was launched.

The normal way to do this in the linux environment would be to upload a shell script to the instance to download the exe, your user-data-file might look something like this...

#! /bin/bash
wget http://www.domain.com/my-file.exe

In Windows there's no default service installed to execute the user-data-file when the instance is booted but there is an open-source project CloudInit.NET which simulates the same process but with a powershell script. The only requirements are .NET 4.0 and CloudInit.NET. Once installed it will execute the user-data-file when the instance is booted. It's very easy to download a file and execute it with a powershell script.

!# /powershell/
$wc = New-Object System.Net.WebClient
$wc.DownloadFile("http://www.domain.com/my-file.exe", "C:\my-file.exe");
& 'C:\my-file.exe'

Answer 3

This sounds like a perfect use case of CloudFormation. I created a template that demonstrates. To use, put your executable in an S3 bucket and create a new CloudFormation stack with the following template. It will download your executable from S3 and run it. Note: the template makes use of special AMIs with CloudFormationScripts built in.

{
    "AWSTemplateFormatVersion" : "2010-09-09",
    "Description" : "Spot Autoscaling for installing and running Windows Services.",
    "Parameters" : {
        "InstanceType" : {
            "Description" : "WebServer EC2 instance type",
            "Type" : "String",
            "Default" : "m1.small",
            "AllowedValues" : ["t1.micro", "m1.small", "m1.medium", "m1.large", "m1.xlarge", "m2.xlarge", "m2.2xlarge", "m2.4xlarge", "c1.medium", "c1.xlarge", "cc1.4xlarge", "cc2.8xlarge", "cg1.4xlarge"],
            "ConstraintDescription" : "must be a valid EC2 instance type."
        },
        "KeyName" : {
            "Description" : "The EC2 Key Pair to get Admin password to Instance.",
            "Type" : "String"
        },
        "DeployS3Bucket" : {
            "Description" : "The S3 Bucket where deploy files are stored",
            "Type" : "String"
        },
        "DeployS3Key" : {
            "Description" : "The exe file that runs on startup",
            "Type" : "String"
        }
    },
    "Mappings" : {
        "RegionToAMIMap" : {
            "us-east-1" : {
                "AMI" : "ami-60b90609"
            },
            "us-west-1" : {
                "AMI" : "ami-5bd6f11e"
            },
            "eu-west-1" : {
                "AMI" : "ami-07151573"
            },
            "ap-southeast-1" : {
                "AMI" : "ami-6ab5f538"
            },
            "ap-northeast-1" : {
                "AMI" : "ami-424ff043"
            }
        }
    },
    "Resources" : {
        "IAMUser" : {
            "Type" : "AWS::IAM::User",
            "Properties" : {
                "Path" : "/",
                "Policies" : [{
                    "PolicyName" : "root",
                    "PolicyDocument" : {
                        "Statement" : [{
                            "Effect" : "Allow",
                            "Action" : "*",
                            "Resource" : "*"
                        }]
                    }
                }]
            }
        },
        "IAMUserAccessKey" : {
            "Type" : "AWS::IAM::AccessKey",
            "Properties" : {
                "UserName" : {
                    "Ref" : "IAMUser"
                }
            }
        },
        "SecurityGroup" : {
            "Type" : "AWS::EC2::SecurityGroup",
            "Properties" : {
                "GroupDescription" : "Enable RDP",
                "SecurityGroupIngress" : [{
                    "IpProtocol" : "tcp",
                    "FromPort" : "3389",
                    "ToPort" : "3389",
                    "CidrIp" : "0.0.0.0/0"
                }]
            }
        },
        "RunExecutable" : {
            "Type" : "AWS::EC2::Instance",
            "Metadata" : {
                "AWS::CloudFormation::Init" : {
                    "config" : {
                        "files" : {
                            "c:\\ToRun\\executable.exe" : {
                                "source" : {
                                    "Fn::Join" : ["/", ["http://s3.amazonaws.com", {
                                        "Ref" : "DeployS3Bucket"
                                    }, {
                                        "Ref" : "DeployS3Key"
                                    }]]
                                },
                                "authentication" : "S3AccessCreds"
                            }
                        },
                        "commands" : {
                            "1-run-executable" : {
                                "command" : "c:\\ToRun\\executable.exe"
                            }
                        }
                    }
                },
                "AWS::CloudFormation::Authentication" : {
                    "S3AccessCreds" : {
                        "type" : "S3",
                        "accessKeyId" : {
                            "Ref" : "IAMUserAccessKey"
                        },
                        "secretKey" : {
                            "Fn::GetAtt" : ["IAMUserAccessKey", "SecretAccessKey"]
                        },
                        "buckets" : [{
                            "Ref" : "DeployS3Bucket"
                        }]
                    }
                }
            },
            "Properties" : {
                "KeyName" : {
                    "Ref" : "KeyName"
                },
                "ImageId" : {
                    "Fn::FindInMap" : ["RegionToAMIMap", {
                        "Ref" : "AWS::Region"
                    }, "AMI"]
                },
                "SecurityGroups" : [{
                    "Ref" : "SecurityGroup"
                }],
                "InstanceType" : {
                    "Ref" : "InstanceType"
                },
                "UserData" : {
                    "Fn::Base64" : {
                        "Fn::Join" : ["", ["<script>\n", "cfn-init.exe -v -s ", {
                            "Ref" : "AWS::StackName"
                        }, " -r RunExecutable ", " --access-key ", {
                            "Ref" : "IAMUserAccessKey"
                        }, " --secret-key ", {
                            "Fn::GetAtt" : ["IAMUserAccessKey", "SecretAccessKey"]
                        }, "\n", "</script>"]]
                    }
                }
            }
        }
    },
    "Outputs" : {}
}

Answer 4

An alternative approach is to use Windows powershell and WinRM - it allows for remote execution, a bit like ssh on Linux.

Here is a sample of a powershell script you can run on the client to remote execute a script (taken from: https://github.com/CloudifySource/cloudify/blob/master/esc/src/main/resources/clouds/ec2-win/upload/bootstrap-client.ps1):

param ([string]$target, [string]$username, [string]$password, [string]$command)

$ErrorActionPreference="Stop"

# Set up the password
$securePassword = ConvertTo-SecureString -AsPlainText -Force $password
$cred = New-Object System.Management.Automation.PSCredential $username, $securePassword

Write-Host "Connecting to management service of $target"
Connect-WSMan -Credential $cred $target 

set-item WSMan:\$target\Client\TrustedHosts -Value * -Force
set-item WSMan:\$target\Shell\MaxMemoryPerShellMB -Value 0 -Force

Write-Host Invoking command on Remote host $target
Invoke-Command -ComputerName $target -Credential $cred  -ScriptBlock {  
    Invoke-Expression $args[0]
} -ArgumentList $command
Write-Host "Command finished"

You can run this command from your own script with the following command:

powershell.exe -inputformat none -File PATH_TO_SCRIPT -target TARGET_IP -password PASSWORD -username USERNAME -command COMMAND_TO_EXECUTE

You should probably quote your strings, especially the password and command, as these will usually have special characters that powershell can interpret as something else.

The WinRM service is on by default on the EC2 Amazon Windows AMIs. All you need to do is open port 5985 (the WinRM port) in your security group.

Finally, if you have never used powershell remoting on your client machine before, there are a couple of commands you should execute to set it up (you only need to do this once):

set-item WSMan:\localhost\Client\TrustedHosts -Value * -Force
set-item WSMan:\localhost\Shell\MaxMemoryPerShellMB -Value 0 -Force
Enable-PSRemoting
Set-ExecutionPolicy unrestricted

Make sure to run these as an Administrator.

Answer 5

You can handle this in 2 ways,

• Using winscp in Windows SFTP program.

  To access your Amazon server using SFTP on Windows, download a Windows SFTP application. Using WinSCP you’ll establish an SFTP session with your server. WinSCP offers some nice features that make it easy to work with your EC2 server. For example, a command in the button bar opens a PuTTY SSH terminal session using the same credentials you used for your SFTP session. (You can also launch a PuTTY session by clicking CTRL+P.).

• Get an S3 bucket and mount on all your windows and linux EC2 instances. You should be able to upload and download the files to S3 bucket from your workstation, which are accessible to your instances.