CODEWITHSUNDEEP
pythonjsonelasticsearchamazon-sqs
Nishant Singh
Nishant Singh

Reputation: 3199

How to configure Elasticsearch to pull messages from Amazon SQS

In my current scenario , i am having ElasticSearch on a ubuntu 14.04 machine and a amazon SQS queue holding some ".json" messages in the queue. I want to pull the messages from amazon SQS via ES so that it can use the .json messages to index it.

What should i be looking for, how exactly should i be proceeding? I am looking for something in lines of python's boto code.

NOTE: I am not interested in using logstash for this

Upvotes: 0

Views: 4291

Answers (1)

aayoubi
aayoubi

Reputation: 12059

Amazon SQS provides an API to receive messages: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSGettingStartedGuide/ReceiveMessage.html

ElasticSearch provides an API to index messages: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/index-doc.html

I am not sure if ES can automatically receive messages from SQS, but one feasible solution is a simple broker between SQS and ES to receive and index.

Upvotes: 1

Related Questions