CODEWITHSUNDEEP
amazon-web-servicesparsingaws-lambdalogparseraws-cloudwatch-log-insights
Emanuele
Emanuele

Reputation: 73

How to correctly parse lambda's whitespace separated log in Logs Insights

i have this type of log in @message:

2022-06-16T10:35:12.921Z 8984a0e4-0ff0-4cfd-ac5a-a312ec3f6157 DEBUG successfully retrieved 15758 object

How can i parse this log to have different columns with: timestamp (2022-06-16T10:35:12.921Z), requestID (8984a0e4-0ff0-4cfd-ac5a-a312ec3f6157), type(DEBUG), message (successfully retrieved 15758 object)?

I've tried with this query but the field type and message are not correctly recognized:

fields @timestamp, @message, @requestId
| parse @message "* * * *" as timestamp, requestId, type, message
| display @timestamp, @requestId, type, message

the result is that timestamp and requestID are correctly identified, while in type there is "15758" instead of DEBUG and in message there is "object"....

how can i modify this query to have the correct output fields?

Upvotes: 2

Views: 849

Answers (1)

Paolo
Paolo

Reputation: 26220

The parsing engine for CloudWatch Logs insights supports using regular expressions, so you can use a regex to obtain the desired result:

fields @timestamp, @message
| parse @message /^(?<timestamp>[^\s]+) (?<requestId>[^\s]+) (?<type>[^\s]+) (?<message>.+$)/
| display timestamp, requestId, type, message

will result in:

enter image description here

To see how this regex works in action you can use this regex101 link.

Upvotes: 3

Related Questions