CODEWITHSUNDEEP
logginglogstashlog-shippingfluentd
user3195649
user3195649

Reputation: 437

Fluentd to Logstash output plugin

I am trying to read from the scribe server using flunetd and output those logs to be stored in logstash for now. I know it's very stupid to log the scribe_central logs to another central logger, but we need this to be done in our current architecture.

Does anyone know if there is any plugin to do that? I searched Google but could not find any.

Upvotes: 7

Views: 16557

Answers (3)

dorony
dorony

Reputation: 1072

You can forward it directly to your logstash tcp input :)

I wrote a flunetd output plugin for forwarding fluentd events to a generic receiver through a secured tcp connection (can be configured for non-secured as well).

To add the plugin to your fluentd agent, use the following command: 

gem install fluent-plugin-loomsystems

Also see full original answer, good luck.

Upvotes: 4

Tiago Lopo
Tiago Lopo

Reputation: 7959

I got it working with http output from fluentd and http input for logstash, below the configuration snippets:

fluentd: 

<match **>
    @type http
    endpoint_url http://logstash-box:8080/
    http_method put
    serializer json
    raise_on_error false
</match>

Logstash: 

input {
    http {
        port => 8080 
    }
}

Obs: You may have to install the plugin gem install fluent-plugin-out-http

Upvotes: 1

Kiyoto Tamura
Kiyoto Tamura

Reputation: 730

For Scribe <-> Fluentd, there is fluent-plugin-scribe:

For Fluentd <-> Logstash, a couple of options:

  1. Use Redis in the middle, and use fluent-plugin-redis and input_redis on Logstash's side. So it would be Fluentd -> Redis -> Logstash. This is what Logstash recommends anyway with log shippers + Logstash.
  2. Alternatively, you can use Fluentd's out_forward plugin with Logstash's TCP input. Logstash has Fluentd codec to handle the input coming from Fluentd.

Upvotes: 5

Related Questions