CODEWITHSUNDEEP
elasticsearchelasticsearch-2.0
AbtPst
AbtPst

Reputation: 8018

Elasticsearch Bulk API : Cannot post more than one record

I am trying to post the following using the bulk api. I have ES 2.2.0

{"index":{"_index":"junktest","_type":"test"}}
{"DocumentID":"555662","Tags":["B","C","D"],"Summary":"Summary Text","Status":"Review","Location":"HDFS","Error":"None","Author":"Abc Mnb","Sector":"Energy","Created Date":"2013-05-23"},
{"DocumentID":"555663","Tags":["A","B","C"],"Summary":"Summary Text","Status":"Review","Location":"HDFS","Error":"None","Author":"Abc Mnb","Sector":"Energy","Created Date":"2013-04-25"}

as

curl -XPOST "http://localhost:9200/_bulk" --data-binary @post.json

but i get

  {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Malformed
     action/metadata line [3], expected START_OBJECT or END_OBJECT but found [VALUE_
STRING]"}],"type":"illegal_argument_exception","reason":"Malformed action/metadata line [3], expected START_OBJECT or END_OBJECT but found [VALUE_STRING]"},"status":400}

why is }, invalid? i have even tried it without the comma but i still get the error , even though i do not have a , !

What is wrong with my syntax?

Edit

I was able to get it to work by

{"index":{"_index":"junktest","_type":"test"}}
{"DocumentID":"555662","Tags":["B","C","D"],"Summary":"Summary Text","Status":"Review","Location":"HDFS","Error":"None","Author":"Abc Mnb","Sector":"Energy","Created Date":"2013-05-23"}
{"index":{"_index":"junktest","_type":"test"}}
{"DocumentID":"555663","Tags":["A","B","C"],"Summary":"Summary Text","Status":"Review","Location":"HDFS","Error":"None","Author":"Abc Mnb","Sector":"Energy","Created Date":"2013-04-25"}

is this the only way to index multiple records using the bulk api?

Upvotes: 1

Views: 3872

Answers (3)

Piyush
Piyush

Reputation: 66

a line break after the last record is important to get it working. I solved a similar problem by adding \n (line break) at the end of last record.

e.g. This will not work :

"{ "name":"Central School", "description":"CBSE Affiliation", "street":"Nagan"}"

but this will

"{ "name":"Central School", "description":"CBSE Affiliation", "street":"Nagan"} \n "

Upvotes: 1

Ali Abbas
Ali Abbas

Reputation: 488

There should be no line break in JSON structure , previously I was doing with this input, It was producing the above error,

{ 
"index" : { "_index" : "ecommerce", "_type" : "product", "_id" : "1002" 
} 
}
{ "id": 2}

Now it is working fine with the following input

{ "index" : { "_index" : "ecommerce", "_type" : "product", "_id" : "1002" } }
{ "id": 2}
{"index":{ "_index" : "ecommerce", "_type" : "product","_id":"1003"}}
{ "id": 3,"name":"Dot net"}

Upvotes: 0

keety
keety

Reputation: 17461

From the documentation

The REST API endpoint is /_bulk, and it expects the following JSON structure:

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n

the document source is optional but the action_meta_data is mandatory and the two are separated by new lines. Given these constraints you can only speicify one record per action.

Also in the example you provided you are not passing "_id" in the meta data which would mean the "_id" is auto-generated . Probably it is intentional but just remember in case you intend to update a document you would not be able to use the DocumentId.

Upvotes: 3

Related Questions