CODEWITHSUNDEEP
elasticsearchloggingfluentd
Anna Carolina
Anna Carolina

Reputation: 395

Pattern not match error in fluentd while tailing json file

I have installed fluentd logger and I want it to monitor the logs of my python code. The logs are the json logs and looks like below:

{
    "FileNo": 232,
    "FileClass": "timitry",
    "FileLevel": "24",
    "DataCount": 5,
    "Data": {
        "User1": <Username>,
        "User2": <Username>,
        "User3": <Username>,
        "User4": <Username>,
        "User5": <Username>"
    },
    "time": "2018-05-14T05:33:02.071793"
}

This is updated every 5mins. I need to write a fluentd input plugin for it so that it can read the new json data and the publish it to elastic search. I dont really know which input plugin to use here but I used tail which give me below errors:

2018-05-14 05:31:04 +0000 [warn]: #0 pattern not match: "    \"FileClass\": \"timitry\","

This is same for all the data. Can anyone please suggest me how can I resolve this issue. Below is the configuration file:

<source>
  @type tail
  format json
  path /home/user/Documents/logs/file_log.json
  tag first
</source>

<match first*>
  @type elasticsearch
  hosts 192.168.111.456:9200
  user <username>
  password <password>
</match>

I have seen others using regex and other formats. Do I also need to use it. How can I use the logs generated by python code to be used by fluentd and publish it to elastic search.

Thanks

Upvotes: 2

Views: 2668

Answers (1)

Nicolas Pepinster
Nicolas Pepinster

Reputation: 6239

Could you try too remove the wildcard after the first in your match directive ? Like :

<match first>
 @type elasticsearch
 hosts 192.168.111.456:9200
 user <username>
 password <password>
</match>

Upvotes: 0

Related Questions