How to enable AWS EMR CloudTrail logging?

Question

We have a team shared AWS account, that sometimes things are hard to debug. Especially, for EMR APIs, throttling happens regularly, that it'll be nice to have CloudTrail logs tell people who is not being nice when using EMR. I think our CloudTrail logging is enabled, that I can see these API events with EMR as event source--

AddJobFlowSteps
RunJobFlow
TerminateJobFlows

I'm pretty sure that I'm calling DescribeCluster for plenty times and caused some throttling, but not sure why they are not showing up in my CloudTrail logs...

Can someone help understand --

• Is there additional setting needed for DescribeCluster EMR API, in order to log events to CloudTrail?
• And what about other EMR APIs? Can they be configured to log events to CloudTrails, without using SDK explicitly writing to CloudTrails?

I have read these articles, feels like much can be done in CloudTrails...

• https://docs.aws.amazon.com/emr/latest/ManagementGuide/logging_emr_api_calls.html

• https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-management-and-data-events-with-cloudtrail.html#logging-management-events-with-the-cloudtrail-console.

• https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-supported-services.html

Appreciate any help!

Answer 1

A quick summary of AWS cloudtrail: The events recorded by AWS cloudtrail are of two types: Management events and Data events. Management events include actions like: stopping an instance, deleting a bucket etc. Data events are only available for two services (S3 and lambda), which include actions like: object 'abc.txt' was read from the S3 bucket.

Under management events, we again have 4 types:

1. Write-only

2. Read-only

3. All (both reads and writes)

4. None

The DescribeCluster event that you are looking for comes under the management event 'Read-only' type. DescribeCluster - cloudtrail image:

Please ensure that you have selected "All" or "ReadOnly" management event type in your cloudtrail trail. Selecting "WriteOnly" in management event type in your cloudtrail trail will not record 'DescribeCluster'. There is no other AWS service specific setting that you can enable in cloudtrail.

Also note that the 'Event history' tab in AWS Cloudtrail console records all types of logs (including ReadOnly) for a period of 90 days. You can see the DescribeCluster event there too.