CODEWITHSUNDEEP
elasticsearch
Arjun Menon
Arjun Menon

Reputation: 703

Bulk indexing using elastic search

Till now i was indexing data to elastic document by document and now as the data started increasing it has become very slow and not an optimized approach. So i was searching for a bulk insert thing and found Elastic Bulk API. From the documents in their official site i got confused. The approach i am using is by passing the data as WebRequest and executing them in the elastic server. So while creating a batch/bulk insert request the API wants us to form a template like

localhost:9200/_bulk as URL and 
{ "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }

to index a document with id 1 and field1 values as value 1. Also the API suggests to send the data as JSON (unpretty, to maintain a non escaping character or so). So to pass multiple document with multiple properties how can i structure my data.

I tried like this in FF RestClient , with POST and header as JSON , but RestClient is throwing some error and i know its not a valid JSON

{ "index" : { "_index" : "indexName", "_type" : "type1", "_id" : "111" },
{ "Name" : "CHRIS","Age" : "23" },"Gender" : "M"}

Upvotes: 4

Views: 16421

Answers (2)

Val
Val

Reputation: 217254

Your data is not well-formed:

  1. You don't need the comma after the first line
  2. You're missing a closing } on the first line
  3. You have a closing } in the middle of your second line you need to remove it as well.

The correct way of formatting your data for a bulk insert look like this:

curl -XPOST localhost:9200/_bulk -d '
{ "index" : { "_index" : "indexName", "_type" : "type1", "_id" : "111" }}
{ "Name" : "CHRIS","Age" : "23" ,"Gender" : "M"}
-H 'Content-Type: application/x-ndjson'

This will work.

UPDATE

Using Postman on Chrome it looks like this. Make sure to add a new line after line 2:

enter image description here

Upvotes: 15

Aman Tandon
Aman Tandon

Reputation: 1509

Using the elasticsearch 7.9.2

In order to send the bulk update I was getting the error of new line as below

Failed update without new line

enter image description here

This is wierd but after adding the new line in the last of the all the operations it is working fine with postman, notice line number 5 in below screenshot

bulk update success after adding newline in last of all the commands in postman

enter image description here

Upvotes: 10

Related Questions