CODEWITHSUNDEEP
rubyelasticsearchfluentdtd-agent
Preyas
Preyas

Reputation: 254

Convert Number date to a String Date Field in Fluentd For Storing into Elasticsearch

I'm new to fluentd and using it to ingest data to elasticsearch. I have a field called request_time which has value : 110820120501 as a number of the format [ddmmyyHHMMSS]. How can I convert this as a date field.

This is what I've written but it doesn't work. Any help is appreciated.

Example request_time value : 100820015642

    <filter>
        @type record_transformer
        enable_ruby true
        <record>
        request_time ${ require 'date'; DateTime.strptime('request_time', '%d%m%y%H%M%S')}
        </record>
    </filter>

I have tried the below as well:

    <filter>
        @type record_transformer
        enable_ruby true
        <record>
            time ${require 'date'; DateTime.parse('request_time').strftime('%d%m%y%H%M%S')}
        </record>
    </filter>

How can I store request_time as a date field coverted to a value as eg. "request_time": "2020-08-11T11:24:23.000+0100" ?

Upvotes: 1

Views: 2090

Answers (1)

Arun M
Arun M

Reputation: 1

Pleas try the below snippet. I have given milli second precision, but feel free to cut it down to your need.

<filter>
    @type record_transformer
    enable_ruby true
    <record>
    date ${Time.at(record['request_time'].to_i/1000, record['request_time'].to_i%1000*1000).utc.strftime('%Y-%m-%dT%H:%M:%S.%LZ')}
    </record>
</filter>

Upvotes: 0

Related Questions