CODEWITHSUNDEEP
jsonlogginggrepjqtail
unmultimedio
unmultimedio

Reputation: 1244

Format and pretty print log via tail

I have this log file that I check on a frequent basis and because of the format of it, it's quite easier to read when pretty printed. I'd like to do so in a tail.

Logs in the file like:

2019-07-04T09:53:04-07:00   some.package.placeholder.stderr {"log": "The content", "foo": "bar", "baz": "blah"}
2019-07-04T10:15:37-07:00   some.package.placeholder.stderr {"log": "I'm actually", "foo": "bar", "baz": "blah"}
2019-07-04T10:15:37-07:00   some.package.placeholder.stderr {"log": "Interested on", "foo": "bar", "baz": "blah"}

And I want to do something similar to

tail -f myLogFile | grep [...?...] | jq '.log'

So when tailing I get:

The content
I'm actually
Interested on

Or even:

2019-07-04T09:53:04-07:00   The content
2019-07-04T10:15:37-07:00   I'm actually
2019-07-04T10:15:37-07:00   Interested on

Upvotes: 4

Views: 12651

Answers (3)

Bruce Edge
Bruce Edge

Reputation: 2189

Here's a thing I use that can be used in a pipe and with file args:

cat /usr/local/bin/j2t
#!/bin/bash

function usage {
  cat <<EOF
Usage:
        $0 <json filename>
    or
        tail -F <json filename> | $0
EOF
}

if (($# == 0)); then
    {
        sed "s/@\(timestamp\)/\1/" | jq -r '[.timestamp, .pri.facility, .pri.severity, .message] | join("\t")'
    } < /dev/stdin

else
    if [ -r "$1" ] ; then
        sed "s/@\(timestamp\)/\1/" $1 | jq -r '[.timestamp, .pri.facility, .pri.severity, .message] | join("\t")'
    else
        help
    fi
fi

eg: (if your daemon.log is json)

j2t /var/log/daemon.log
level: 63, builder: awillia2)
2021-08-14T00:00:06.820642+00:00        daemon  INFO     Starting Run Clamscan...
2021-08-14T00:00:06.846405+00:00        daemon  INFO     Started Run Clamscan.

Should probably reformat the time, it's a bit long.

Upvotes: 1

Jeff Mercado
Jeff Mercado

Reputation: 134881

If the log lines are tab delimited, you can read the lines in raw and split on tabs. Which you could then parse the json and filter to your hearts content, and recombine as necessary.

$ tail -f myLogFile | jq -Rr 'split("\t") | [.[0], (.[2] | fromjson.log)] | join("\t")'
2019-07-04T09:53:04-07:00   The content
2019-07-04T10:15:37-07:00   I'm actually
2019-07-04T10:15:37-07:00   Interested on

Upvotes: 2

Ed Morton
Ed Morton

Reputation: 203645

With GNU grep for -o:

$ tail file | grep -o '{[^}]*}' | jq -r '.log'
The content
I'm actually
Interested on

With any awk:

$ tail file | awk 'sub(/.*{/,"{")' | jq -r '.log'
The content
I'm actually
Interested on

$ tail file | awk '{d=$1} sub(/.*{/,""){$0="{\"date\": \""d"\", " $0} 1' | jq -r '.date + " " + .log'
2019-07-04T09:53:04-07:00 The content
2019-07-04T10:15:37-07:00 I'm actually
2019-07-04T10:15:37-07:00 Interested on

That last one works by merging the date field from the input into the json so then jq can just select and print it with the log field.

Upvotes: 5

Related Questions