CODEWITHSUNDEEP
amazon-web-servicesamazon-cloudwatchamazon-cloudwatchlogs
aceofbassgreg
aceofbassgreg

Reputation: 3947

Cloudwatch logs - No Event Data after time elapses

I've looked on the AWS forums and elsewhere but haven't found a solution. I have a lambda function that, when invoked, creates a log stream which populates with log events. After about 12 hours or so, the log stream is still present, but when I open it, I see the following:

No Event Data

The link explains how to start sending event data, but I already have this set up, and I am sending event data, it just disappears after a certain time period.

I'm guessing there is some setting somewhere (either for max storage allowed or for whether logs get purged), but if there is, I haven't found it.

Upvotes: 5

Views: 5949

Answers (2)

aceofbassgreg
aceofbassgreg

Reputation: 3947

TL;DR: Just use the CLI. See Update 2 below.

This is really bizarre but I can replicate it...

enter image description here

I un-checked the "Expire Events After" box, and lo and behold I was able to open older log streams. What seems REALLY odd is that if I choose to display the "Stored Bytes" data, many of the files are listed at 0 bytes even though they have log events:

7/29/2016 CloudWatch Log Streams

Update 1:

This solution no longer works as I can only view the log events in the first two log streams. What's more is that the Stored Bytes column displays different (and more accurate) data:

8/01/2016 CloudWatch Log Streams

This leads me to believe that AWS made some kind of update.

UPDATE 2:

Just use the CLI. I've verified that I can retrieve log events from the CLI that I cannot retrieve via the web console.

First install the CLI (if you haven't already) and use the following command:

aws logs get-log-events --log-group-name NAME-OF-LOGGROUP --log-stream-name LOG-STREAM-NAME // be sure to escape special characters such as /, [, $ etc

Upvotes: 1

Greg Holst
Greg Holst

Reputation: 974

Another reason for missing data in the log stream might be a corrupted agent-state file. First check your logs

vim /var/log/awslogs.log

If you find something like "Caught exception: An error occurred (InvalidSequenceTokenException) when calling the PutLogEvents operation: The given sequenceToken is invalid. The next expected sequenceToken is:" you can regenerate the agent-state file as follows:

sudo rm /var/lib/awslogs/agent-state
sudo service awslogs stop
sudo service awslogs start

Upvotes: 1

Related Questions