CODEWITHSUNDEEP
apachemongodbloggingfluentd
user1403863
user1403863

Reputation: 11

How to write Apache Combined Log into mongoDB with custom attributes using fluentd

I'm using tail and fluent-plugin-mongo to write Apache Combined Log Format into mongoDB but I want to custom some attribute format to write into mongoDB such as

{

ip : 21.18.104.8
host : www.sample.com
message : buildtime=2012_02_14_16_25;version=01.03.3752;totalTimeMin=0;

}

to

{

ip : 21.18.104.8
host : www.sample.com 
message : buildtime=2012_02_14_16_25;version=01.03.3752;totalTimeMin=0;
buildtime : 2012_02_14_16_25
version : 01.03.3752
totalTimeNin : 0

}

{

ip : 69.249.69.211
host : www.sample.com
message : buildtime=2012_02_14_16_25;version=01.03.3752;totalTimeMin=0;

}

to

{

ip :  69.249.69.211
country : US
host : www.sample.com
message : buildtime=2012_02_14_16_25;version=01.03.3752;totalTimeMin=0;

}

Is that possible to do? I don't have any ideas which plugins I should use. I've already read docs but I did't get it since it doesn't have many examples.

Upvotes: 1

Views: 967

Answers (1)

Kazuki Ohta
Kazuki Ohta

Reputation: 1441

You can extend in_tail plugin to achieve your goals. The URL below describes how to do it.

http://docs.fluentd.org/articles/plugin-development#custom-parser-for-tail-input-plugin

If you have any question, please ask me here or Google Groups. Thanks -K

Upvotes: 1

Related Questions