CODEWITHSUNDEEP
elasticsearchrabbitmqelasticsearch-river
Ap100
Ap100

Reputation: 231

Use Elastic Search with RabbitMQ

I have a RabbitMQ Queue that ingest some docs that needs to be transferred to Elastic Search.

How do I create the Elastic Search as a consumer, so that ES acts a consumer to the docs Queue.

Possible options :

  1. Using RabbitMQ River ?
  2. Using RabbitMQ Plugin ? (How to do this )
  3. Other ?

Can someone post an example ?

Upvotes: 3

Views: 13556

Answers (2)

Val
Val

Reputation: 217514

Since you're asking for a RabbitMQ river, here is an example, although you should note that rivers will be deprecated soon (i.e. as of ES 1.5) and the other solution by Prameswar Lal using Logstash will be preferred.

curl -XPOST localhost:9200/_river/custom_river_name/_meta -d '{
    "type" : "rabbitmq",
    "rabbitmq" : {
        "host" : "localhost",
        "port" : 5672,
        "user" : "guest",
        "pass" : "guest",
        "vhost" : "/",
        "queue" : "elasticsearch",
        "exchange" : "elasticsearch",
        "routing_key" : "elasticsearch",
        "exchange_declare" : true,
        "exchange_type" : "direct",
        "exchange_durable" : true,
        "queue_declare" : true,
        "queue_bind" : true,
        "queue_durable" : true,
        "queue_auto_delete" : false,
        "heartbeat" : "30m",
        "qos_prefetch_size" : 0,
        "qos_prefetch_count" : 10,
        "nack_errors" : true
    },
    "index" : {
        "bulk_size" : 100,
        "bulk_timeout" : "10ms",
        "ordered" : false,
        "replication" : "default"
    }
}

Upvotes: 1

pl_rock
pl_rock

Reputation: 15822

at your ELK create config file /etc/logstash/conf.d/anyfile.conf

input {
   rabbitmq {
      host => 'rabbit.example.com'
      queue => 'my_queue_name'
      exchange => "my_exchange_name"
      key => 'my_logs'
      durable => true
   }
}
output {
   elasticsearch {
      host => "elk.example.com"
   }
}

Upvotes: 4

Related Questions