CODEWITHSUNDEEP
regexfluentd
S Andrew
S Andrew

Reputation: 7198

How to create Regex pattern for fluentd

I am trying to parse daemon logs from my linux machine to elastic search using fluentd but having hard time creating regex pattern for it. Below are few of the logs from the daemon logs:

Jun  5 06:46:14 user avahi-daemon[309]: Registering new address record for fe80::a7c0:8b54:ee45:ea4 on wlan0.*.
Jun  5 06:46:14 user dhcpcd[337]: wlan0: deleting default route via fe80::1e56:feff:fe13:2da
Jun  5 06:46:14 user dhcpcd[337]: wlan0: deleting route to 2402:3a80:9db:48da::/64
Jun  5 06:46:14 user dhcpcd[337]: wlan0: deleting address fe80::a7c0:8b54:ee45:ea4
Jun  5 06:46:14 user avahi-daemon[309]: Withdrawing address record for fe80::a7c0:8b54:ee45:ea4 on wlan0.
Jun  5 06:46:14 user avahi-daemon[309]: Leaving mDNS multicast group on interface wlan0.IPv6 with address fe80::a7c0:8b54:ee45:ea4.

So as you can see from the above logs, first we have the time of the logs, then we have the username and the daemon name, followed by the message.

I want to create below json format for the above logs:

{
    "time": "Jun  5 06:46:14",
    "username": "user",
    "daemon": "avahi-daemon[309]",
    "msg": "Registering new address record for fe80::a7c0:8b54:ee45:ea4 on wlan0.*."
}

{
    "time": "Jun  5 06:46:14",
    "username": "user",
    "daemon": "dhcpcd[337]: wlan0",
    "msg": "deleting default route via fe80::1e56:feff:fe13:2da"
}

Can anyone please give me some help on this. Is there any tool which we can use to generate regex in fluentd.

Edit:

I have managed to get few things matched from the logs like:

^(?<time>^(.*?:.*?):\d\d) (?<username>[^ ]*) matches Jun  5 06:46:14 user

but when I am passing this in fluentular, its not showing any results.

Upvotes: 1

Views: 5216

Answers (1)

Matt.G
Matt.G

Reputation: 3609

Try Regex: ^(?<time>[A-Za-z]{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})\s(?<username>[^ ]+)\s+(?<daemon>[^:]+):\s+(?<message>.*)$

See Demo

Upvotes: 1

Related Questions