CODEWITHSUNDEEP
javarestelasticsearch
Muatik
Muatik

Reputation: 4171

Elasticsearch bulk insert using rest client

To improve performance, I want to send documents to Elasticsearch in bulk instead sending one by one. I've read about elastic bulk API at https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-docs-bulk.html

However, I am using Elasticsearch rest-client (https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html) and couldn't find any example or documentation about how to make a bulk insert. All I could find was about bulk requests through transport client.

I guess I have to prepare request body as described here (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html) and pass it to restclient's performRequest method? Is there another way, for instance, a builder mechanism in the ES java rest-client library, to make bulk insert using rest?

Upvotes: 6

Views: 16022

Answers (2)

chomp
chomp

Reputation: 1342

Another example in addition to Val answer: http://web.archive.org/web/20180813044955/http://cscengineer.net/2016/10/22/elastic-search-bulk-api/

Just use POST instead of PUT (be careful with the .exchange in the use of rest template)

Upvotes: -1

Val
Val

Reputation: 217304

Yes, that's correct, for now the REST client only allows to send raw REST queries to ES but nothing too sophisticated. Elastic is working on a high-level client next that will work on top of the REST client and allow you to send DSL queries, etc.

For now, here is a sample code that you can use to send documents in bulk to your ES server:

RestClient client = ...;
String actionMetaData = String.format("{ \"index\" : { \"_index\" : \"%s\", \"_type\" : \"%s\" } }%n", index, type);

List<String> bulkData = ...; // a list of your documents in JSON strings    
StringBuilder bulkRequestBody = new StringBuilder();
for (String bulkItem : bulkData) {
    bulkRequestBody.append(actionMetaData);
    bulkRequestBody.append(bulkItem);
    bulkRequestBody.append("\n");
}
HttpEntity entity = new NStringEntity(bulkRequestBody.toString(), ContentType.APPLICATION_JSON);
try {
    Response response = client.performRequest("POST", "/your_index/your_type/_bulk", Collections.emptyMap(), entity);
    return response.getStatusLine().getStatusCode() == HttpStatus.SC_OK;
} catch (Exception e) {
    // do something
}

Upvotes: 19

Related Questions