Reputation: 171
i have been working on project, and i want to export metrics data from cloudwatch like CPU Utilization and Network Out data, is there any way to get that data? and convert it to csv?
Upvotes: 15
Views: 31229
Reputation: 99
You can now download the metrics directly from AWS Metrics dashboard in the .csv file format. As shown in the image in order to download the metrics from AWS, you need to go the CloudWatch dashboard
and click on the Metrics
in the left navigation bar and then select the metric you want to download. Now click on the Actions
drop-down button and select Download as .csv
Upvotes: 2
Reputation: 11
If the csv exporting is a routine work, you can try categraf cloudwatch plugin . It can easily export cloud-watch metrics to your Prometheus then you can visualize data using grafana and export to csv.
And further more, you can also try nightingale project which can be an alternative of prometheus&grafana.
Upvotes: 1
Reputation: 620
You can't do it directly but following is the step by step instructions:
Use the following CLI:
aws cloudwatch get-metric-statistics
--namespace AWS/EC2 --metric-name CPUUtilization
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx
--statistics Average
--start-time 2020-03-01T00:00:00
--end-time 2020-03-31T23:59:00
--period 3600
--region us-east-1
Valid options for --metric-name depend on the --name-space parameter. For AWS/EC2, the full list can be seen by running the following CLI command:
aws cloudwatch list-metrics --namespace "AWS/EC2"
Valid options for --statistics are:
SampleCount
Average
Sum
Minimum
Maximum
--start-time and --end-time specify the range.
--period The granularity, in seconds, of the returned data points.
--region The region where the CloudWatch metric is being meatured (us-east-1, us-west-2 etc.)
Data output will look something like the following:
{
"Label": "CPUUtilization",
"Datapoints": []
}
To convert it to CSV we will use jq. For this you have two options:
Pipe all the aws cli output to jq:
aws cloudwatch get-metric-statistics
--namespace AWS/EC2 --metric-name CPUUtilization
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx
--statistics Average
--start-time 2020-03-01T00:00:00
--end-time 2020-03-31T23:59:00
--period 3600
--region us-east-1
| jq -r '.Datapoints[] | [.Timestamp, .Minimum, .Unit] | @csv'
Export the data to JSON:
aws cloudwatch get-metric-statistics
--namespace AWS/EC2 --metric-name CPUUtilization
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx
--statistics Average
--start-time 2020-03-01T00:00:00
--end-time 2020-03-31T23:59:00
--period 3600
--region us-east-1 >> data.json
Use jq to read the json to csv:
jq -r '.Datapoints[] | [.Timestamp, .Minimum, .Unit] | @csv' data.json
Here is how the output will look like:
"2020-03-24T11:00:00Z",0.327868852454245,"Percent"
"2020-03-11T21:00:00Z",0.327868852454245,"Percent"
"2020-03-15T04:00:00Z",0.322580645156596,"Percent"
"2020-03-27T18:00:00Z",0.327868852478101,"Percent"
Upvotes: 30
Reputation: 270224
There is no in-built capability to export Amazon CloudWatch metrics to CSV.
There are API calls available to extract metrics, but you would need to write a program to call the API, receive the metrics and store it in a suitable format.
There are projects available that can assist with this, such as:
If you are looking for these types of tools, make sure they refer to CloudWatch metrics, not CloudWatch Logs (which is something different).
Upvotes: 6