CODEWITHSUNDEEP
regexelasticsearchtype-conversionfluentd
CC.
CC.

Reputation: 2928

fluentd convert field to integer

I need your help. I have fluentd configured in tail mode to send my logs to a elasticsearch cluster. I'm using a regex to parse each log line but for some fields I need to convert metricValue in integer. I've tried some things but is not working.

Here is my fluentd conf file:

    <source>
    @type tail
    path /tmp/application.log
    pos_file /tmp/access.log.pos
    format multi_format
    tag first.log

    <pattern>
      format /(?<app>.*)\((?<instance>.*)\) CID\((?<correlationId>.*)\) (?<level>.*) \[(?<timestamp>.*)\] \[(?<Thread>.*)\] (?<class>.*) - \[Provider:(?<providerName>.*),Curef:(?<cuRef>.*),Operation:(?<operation>.*),Reference:(?<reference>.*),Idx:(?<idx>.*)\]/
    </pattern>
    <pattern>
      format /(?<app>.*)\((?<instance>.*)\) CID\((?<correlationId>.*)\) (?<level>.*) \[(?<timestamp>.*)\] \[(?<Thread>.*)\] (?<class>.*) - (?<body>.*)/
    </pattern>
  </source>
  <match *.log>
    @type copy
    <store>
      @type stdout
    </store>
    <store>
      @type elasticsearch
      host localhost
      port 9200
      logstash_format true
      logstash_prefix fluentd-log
      flush_interval 10s
    </store>
  </match>

I found in the fluend documentation the syntax but is not working (and I don't know where to put these line)

types <field_name_1>:<type_name_1>,<field_name_2>:<type_name_2>,...

I want to convert the field metricValue to integer before sending it to elasticsearch.

Thanks for any idea.

C.C.

Upvotes: 1

Views: 5402

Answers (1)

kay
kay

Reputation: 753

You have to put these fields (types <field_name_1>:<type_name_1>,<field_name_2>:<type_name_2>,...) into the source section in your case. And if you use parser plugin, you have to insert it inside the filter section (here is an example)

Upvotes: 1

Related Questions