CODEWITHSUNDEEP
herokusedawk
sarp
sarp

Reputation: 3750

How to parse/filter log line with awk or sed

I am trying to find out how to filter heroku log drains using awk, sed or some other tool. As you can see the examples below, the command filters the lines with grep related to the router only. But I could not figure out how to filter the rest.

heroku logs -t | grep router | ???

2013-07-20T20:44:40.855998+00:00 heroku[router]: at=info method=GET path=/oauth2/callback?error=access_denied host=app.herokuapp.com fwd="123.234.456.78" dyno=web.5 connect=1ms service=55ms status=302 bytes=5

2013-07-20T20:44:40.855998+00:00 heroku[router]: at=info method=GET path=/oauth2/callback/succesuser=%7B%22user%3Aexample%40example.com%22%2C%22force%22%3Afalse%7D host=app.herokuapp.com fwd="123.234.456.78" dyno=web.5 connect=1ms service=312ms status=302 bytes=5

Question 1) There is a part that gives detail about the service time in the log lines: (service=55ms and service=312ms from the examples) Let's say I want to see only the ones with service time more than 300ms. How can I achive that?

Optional Question) Is there any chance that I can format these lines comma separated like this: timestamp,path,service

Example outputs for the optional question:

2013-07-20 20:44:40,/oauth2/callback?error=access_denied,55`

2013-07-20 20:44:40,/oauth2/callback/succesuser=%7B%22user%3Aexample%40example.com%22%2C%22force%22%3Afalse%7D,312`

Upvotes: 2

Views: 1198

Answers (2)

Anil
Anil

Reputation: 3919

Use awk. Put this in a file awk0:

BEGIN       {OFS = ","}

            {split($10,arr,"\="); time = arr[2] + 0}

time > 300  {print $1, $5, time}

Then this should work:

heroku logs -t | grep router | awk -f awk0

Good luck!

Upvotes: 1

Alexej Magura
Alexej Magura

Reputation: 5119

See if this sed expression is of any use: 

sed '/[3-9][0-9][0-9][m][s]/!d'
sed '/[3-9][0-9][0-9][m][s]/!d

Here's an example: 

printf '301ms\n300ms\n302ms\n200ms\n40ms' | sed '/[3-9][0-9][0-9][m][s]/!d'

=> 301ms
=> 300ms
=> 302ms

To get only those lines with more than 300ms, use this instead:

sed '/[3-9][0-9][1-9]/!d'

Another example:

printf '301ms\n300ms\n302ms\n200ms\n40ms' | sed '/[3-9][0-9][1-9][m][s]/!d'

=> 301ms
=> 302ms

The reason why the above--this /[3-9][0-9][1-9]/!d--also filters out numbers smaller than 300, is because of the ! flaggy thingy--it tells sed to delete all lines that don't match the pattern [3-9][0-9][1-9][m][s].

Upvotes: 1

Related Questions