CODEWITHSUNDEEP
amazon-cloudwatchamazon-cloudwatchlogs
Mark Hollas
Mark Hollas

Reputation: 1137

Accessing values in JSON array

I am following the instruction in the documentation for how to access JSON values in CloudWatch Insights where the recomendation is as follows 

JSON arrays are flattened into a list of field names and values. For example, to specify the value of instanceId for the first item in requestParameters.instancesSet, use requestParameters.instancesSet.items.0.instanceId.

ref https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_AnalyzeLogData-discoverable-fields.html

I am trying the following and getting nothing in return. The intellisense autofills up to processList.0 but no further

fields processList.0.vss
| sort @timestamp desc
| limit 1

The JSON I am woking with is 

"processList": [
        {
            "vss": xxxxx,
            "name": "aurora",
            "tgid": xxxx,
            "vmlimit": "unlimited",
            "parentID": 1,
            "memoryUsedPc": 16.01,
            "cpuUsedPc": 0.01,
            "id": xxxxx,
            "rss": xxxxx
        },
        {
            "vss": xxxx,
            "name": "aurora",
            "tgid": xxxxxx,
            "vmlimit": "unlimited",
            "parentID": 1,
            "memoryUsedPc": 16.01,
            "cpuUsedPc": 0.06,
            "id": xxxxx,
            "rss": xxxxx
        }]

Upvotes: 5

Views: 6689

Answers (2)

Dhwani Katagade
Dhwani Katagade

Reputation: 1240

The reference link that you have posted also states the following.

CloudWatch Logs Insights can extract a maximum of 100 log event fields from a JSON log. For extra fields that are not extracted, you can use the parse command to parse these fields from the raw unparsed log event in the message field.

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_AnalyzeLogData-discoverable-fields.html

For very large JSON messages, Insights intellisense may not be parsing all the fields into named fields. So, the solution is to use parse on the complete JSON string in the field where you expect your data field to be present. In your example and mine it is processList.

I was able to extract the value of specific cpuUsedPc under processList by using a query like the following.

fields @timestamp, cpuUtilization.total, processList
| parse processList /"name":"RDS processes","tgid":.*?,"parentID":.*?,"memoryUsedPc":.*?,"cpuUsedPc":(?<RDSProcessesCPUUsedPc>.*?),/
| sort @timestamp asc
| display @timestamp, cpuUtilization.total, RDSProcessesCPUUsedPc

Upvotes: 2

talt001
talt001

Reputation: 11

Have you tried the following?

fields @@timestamp, @processList.0.vss
| sort @@timestamp desc
| limit 5

It may be a syntax error. If not, please post a couple of records worth of the overall structure, with @timestamp included.

Upvotes: 1

Related Questions