CODEWITHSUNDEEP
amazon-web-servicesamazon-cloudwatch
Philip
Philip

Reputation: 1901

CloudWatch agent per-instance streams

I'm using CloudWatch Agent (not CloudWatch Logs Agent) on a Windows instance. The configuration file amazon-cloudwatch-agent.toml includes a hard-coded instance id. If I create an AMI for an auto-scaling group, every instance launched uses that same stale instance id for the log stream. They all write to the same stream.

I'd like every instance to write to a stream with its own instance_id. Which seems like what you'd almost always want. How is this possible?

Upvotes: 5

Views: 3138

Answers (2)

Philip
Philip

Reputation: 1901

I believe that doing "amazon-cloudwatch-agent-ctl start" from your user-data is probably what they want you to do. They don't ever tell you anywhere that, not that I found, but that should create the TOML (if you make sure it doesn't already exist) and then start/restart the service. But "start" has a download part and I wasn't sure what to there given my JSON file is local.

So I did something different, but same idea:

  1. Set the AmazonCloudWatchAgent service to manual start
  2. Run the below batch file at boot with Task Scheduler

The below batch file will create the TOML then start the service:

SET CLOUD_WATCH="C:\Program Files\Amazon\AmazonCloudWatchAgent"
SET CLOUD_WATCH_DATA=C:\ProgramData\Amazon\AmazonCloudWatchAgent

SET JSON=%CLOUD_WATCH_DATA%\amazon-cloudwatch-agent.json
SET TOML=%CLOUD_WATCH_DATA%\amazon-cloudwatch-agent.toml
SET CONFIG=%CLOUD_WATCH_DATA%\common-config.toml

SET TRANSLATOR=%CLOUD_WATCH%\config-translator.exe

rem Translate JSON into TOML
%TRANSLATOR% --input %JSON% --output %TOML% --mode ec2 --config %CONFIG%

rem Start the service
sc start AmazonCloudWatchAgent

This seems to work for me. Also I make sure my AMI has empty log files, both the CloudWatch Agent's log and my own logs. So every instance starts fresh.

But I might switch to user-data "amazon-cloudwatch-agent-ctl start" at some point if I can make it work.

Upvotes: 3

RaGe
RaGe

Reputation: 23677

From documentation:

log_stream_name – Optional. Specifies what to use as the log stream name in CloudWatch Logs. As part of the name, you can use {instance_id}, {hostname}, {local_hostname}, and {ip_address} as variables within the name. {hostname} retrieves the hostname from the EC2 metadata, while {local_hostname} uses the hostname from the network configuration file.

If you omit this field, the default of {instance_id} is used. A log stream is created automatically if it does not already exist.

So the simplest thing to do is to not define log_stream_name at all.

It is wierd that the agent uses the .toml file for its config and not the json file and the toml generation is only done at install time. Maybe you can force a toml regeneration using:

$/opt/aws/amazon-cloudwatch-agent/bin/config-translator \
  --input /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json \
  --output /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml \
  --mode ec2 \
  --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml

You may even be able to stick this in the userdata script. agent might need a restart as well.

Upvotes: 3

Related Questions