CODEWITHSUNDEEP
elasticsearchapache-nifi
user2297083
user2297083

Reputation: 188

Apache NIFI - Putelasticsearch - Malformed content, found extra data after parsing: START_OBJECT

I am using Putelasticsearch5 processor and i have few more processors upstream.

The resulting json from upstream processor loks like below: {"id":"1234","name":"apple"} {"id":"5678","name":"Orange"}

I am expecting this to do an bulk index. But it is throwing error saying "Caused by: java.lang.IllegalArgumentException: Malformed content, found extra data after parsing: START_OBJECT"

What am i doing wrong here?

Upvotes: 0

Views: 596

Answers (1)

mattyb
mattyb

Reputation: 12093

The PutElasticsearch processors do the bundling of individual documents into the Bulk API automatically. This means (with the exception of PutElasticsearchHttpRecord) they expect one document per flow file, and you can try to batch them up by setting the Batch Size property. If you set Batch Size to 100 for example, the processor will get up to the next 100 flow files (i.e. however many flow files are in the queue -- up to 100 -- when the processor is triggered).

If you have a MergeContent processor or something bundling all your documents up, try removing that part of the flow. If you are seeing performance issues, try scheduling the PutElasticsearch processor less often or setting the Run Duration to something like 25 ms or more.

Upvotes: 1

Related Questions