CODEWITHSUNDEEP
bashamazon-web-servicesamazon-ec2
rubenhak
rubenhak

Reputation: 900

Start AWS EC2 instance, run commands, stream logs to console and terminate

Trying to run few steps of CI/CD in a EC2 instance. Please don't ask for reasons.

Need to: 1) Start an instance using AWS CLI. Set few environment variables. 2) Run few bash commands. 3) Stream the command from the above commands into the console of the caller script. 4) If any of the commands fail, need to fail the calling script as well. 5) Terminate the instance.

Upvotes: 0

Views: 1803

Answers (2)

Martin Löper
Martin Löper

Reputation: 6659

There is a SO thread which indicates that streaming the output is not as easy. [1] What I would do, if I had to implement this task:

  • Start the instance using the cli command aws ec2 run-instances and using an AMI which has the AWS SSM agent preinstalled. [2]

  • Run your commands using AWS SSM. [3] This has the benefit that you can run any number of commands you want - whenever you want (i.e. the commands must not be specified at instance launch, but can be chosen afterwards). You also get the status code of each command.[4]

  • Use the CloudWatch integration in SSM to stream your command output to CloudWatch logs. [5]

  • Stream the logs from CloudWatch to your own instance. [6]

Note: Instead of streaming the command output via CloudWatch, you could also periodically poll the SSM API by using aws ssm get-command-invocation. [7]

Reference

[1] How to check whether my user data passing to EC2 instance working or not?
[2] Working with SSM Agent - AWS Systems Manager
[3] Walkthrough: Use the AWS CLI with Run Command - AWS Systems Manager
[4] Understanding Command Statuses - AWS Systems Manager
[5] Streaming AWS Systems Manager Run Command output to Amazon CloudWatch Logs | AWS Management Tools Blog
[6] how to view aws log real time (like tail -f)
[7] get-command-invocation — AWS CLI 1.16.200 Command Reference

Upvotes: 2

mahendra rathod
mahendra rathod

Reputation: 1638

Approach 1.

Start an instance using AWS CLI.

 aws ec2 start-instances --instance-ids i-1234567890abcdef0

Set few environment variables.

Use user dat of ec2 to set env. & run commands

.. Run your other logic / scripts

To terminate the instance run below command in the same instance.

instanceid=`curl http://169.254.169.254/latest/meta-data/instance-id`
aws ec2 terminate-instances --instance-ids $instanceid

Approach 2.

Use python boto3 or kitchen chef ci.

Upvotes: 0

Related Questions