CODEWITHSUNDEEP
amazon-web-servicesamazon-cloudwatchamazon-cloudwatchlogs
Nikita Fedyashev
Nikita Fedyashev

Reputation: 19003

CloudWatch Metric Filter for checking JSON key exists

I'm trying to come up with a metric filter expression that filters CloudWatch Logs when a special JSON key attribute is present.

Use case is the following: the application does all kinds of logging(in JSON format) and whenever it has a special JSON key(nested JSON response from third-part service), I would like to filter it.

Example logs:

{"severity":"INFO","msg":"EVENT","event":{"key1":"value1"}}
{"severity":"INFO","msg":"FooService responded","response":{"response_code":800}}

Filter patterns that I've tried that don't work:

{ $.response }
{ $.response = *}
{ $.response = "*"}
{ $.response EXISTS }
{ $.response IS TRUE }
{ $.response NOT NULL }
{ $.response != NULL }

Expected filtering result:

{"severity":"INFO","msg":"FooService responded","response":{"response_code":800}}

{ $.response EXISTS } does the opposite of what I expect(returns the 1st line rather than then 2nd) but I'm not sure how to negate it.

Reference material: Filter and pattern syntax @ CloudWatch User Guide

Upvotes: 4

Views: 3249

Answers (2)

DataMacGyver
DataMacGyver

Reputation: 446

I don't have a solution to the task of finding records where a field exists. Indeed, the linked document in the question specifically calls this out as not supported.

but

If we simply reverse our logic this becomes a more tractable problem. Looking at your data, you want All records where there's a response key but that could also be stated as All records where there isn't an events key.

This means you could accomplish the task with {$.event NOT EXISTS}. Of course, this becomes more complicated the more types of log messages you get (I had to chain three different NOT EXISTS queries for my use case) but it does solve the problem.

Upvotes: 1

JohnRoux
JohnRoux

Reputation: 133

I haven't found a good solution.

But I did find one at least.

If you search for a key being != a specific value, it seems to do a null check on it.

So if you say:

{$.response != "something_no_one_should_have_ever_saved_this_response_as"}

Then you get all entries where response exists in your json, and where it's not your string (hopefully all of the valid entries)

Definitly not a clean solution, but it seems to be pretty functional

Upvotes: 7

Related Questions