How can I create an elastic search index with the JAVA high level REST client?

Question

I was going through these docs to create an elastic search index from Elastic's high level JAVA REST client. It seems to skip over steps for authenticating with my elastic cloud account. Can someone please point me toward the relevant documentation?

I launched my elastic search instance and copied the endpoint URL into my client code.

I initially had connection errors and now there are none. Only authentication errors. So, I'm pretty sure I'm connecting with the correct endpoint URL and need to authenticate somehow - perhaps with a header.

Now, I am seeing this error:

Elasticsearch exception [type=security_exception, reason=action [indices:data/write/index] requires authentication]

I can view the endpoint of my Elastic Search deployment with no problems from Postman with this command: GET https://:@d97215aee2.us-east-1.aws.found.io:9243

I can also create an index using this command from Postman... PUT https://elastic:4YQIMXfoSZ9mXPgY1fj7T5BU@d97218f74f6d48489b355dd7d665aee2.us-east-1.aws.found.io:9243/. Yet, I cannot do the same from the Java code.

Here is the state of my Java code. It is pretty much the code from these tutorial pages.

https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-getting-started-initialization.html

https://www.elastic.co/guide/en/elasticsearch/client/java-rest/6.4/java-rest-high-document-index.html

import java.io.IOException;

import javax.ws.rs.POST;
import javax.ws.rs.Path;

import org.apache.http.HttpHost;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.xcontent.XContentType;

@Path("/elasticsearch")
public class ElasticSearchService {

    @POST
    public void createElasticIndex() throws IOException {
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(
                    new HttpHost("d9<deleted a bunch of characters for privacy>7d665aee2.us-east-1.aws.found.io", 9243, "https")));


        IndexRequest request = new IndexRequest(
                "posts",
                "doc",
                "1");
        String jsonString = "{" +
                "\"user\":\"kimchy\"," +
                "\"postDate\":\"2013-01-30\"," +
                "\"message\":\"trying out Elasticsearch\"" +
                "}";
        request.source(jsonString, XContentType.JSON);

        client.close();
    }    
}

I have also tried updating the URL address with our username and password as suggested by this post: ElasticSearch authentication error with ElasticCloud?

Essentially, I updated my URL like this...

        RestClient.builder(
                new HttpHost(
                        "<my user name>:<my password>@d97218<hidden characters>d665aee2.us-east-1.aws.found.io",
                        9243, "https")));

This did not work for me. I am guessing this person wasn't using the new Elastic High Level REST client. I received this error:

org.glassfish.jersey.server.internal.process.MappableException: java.io.IOException: :@d97265aee2.us-east-1.aws.found.io: invalid IPv6 address

Answer 1

If you know how to insert documents through API then this way will much more easy for you to do anything similar API (DELETE,POST,PUT...) First, you will need RestHighLevelClient and all you have to do

String index = "/indexName/_doc";
Request request = new Request("POST", index);
request.setJsonEntity(
      "{ \"message\": \" example add insert\" }"
);

client.getLowLevelClient().performRequest(request); 

This will execute like how API does.

Answer 2

You can use both synchronous and asynchronous API of elastic search to create index. But it depends on requirement.

Find the below link of elastic search documentation which explain both synchronous and asynchronous API use. https://www.elastic.co/guide/en/elasticsearch/client/java-rest/master/java-rest-high-create-index.html

Sample code:- Synchronous API :-

    CreateIndexRequest request = new CreateIndexRequest("twitter");
    request.settings(Settings.builder() 
        .put("index.number_of_shards", 3)
        .put("index.number_of_replicas", 2)
    );

CreateIndexResponse createIndexResponse = client.indices().create(request, RequestOptions.DEFAULT);

Asynchronous API:-

client.indices().createAsync(request, RequestOptions.DEFAULT, listener);

Asynchronous API adds advantages of thread and makes API to work better way. Concern in asynchronous API is to receive response. Below is the snippet how can you receive response.

PlainActionFuture<CreateIndexResponse > future = new PlainActionFuture<>();
client.indices().createAsync(request, RequestOptions.DEFAULT, future);
CreateIndexResponse response = future.actionGet(); 

Answer 3

Found the answer here: enter link description here

Updated code that works:

import java.io.IOException;

import javax.ws.rs.POST;
import javax.ws.rs.Path;

import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.rest.RestStatus;

@Path("/elasticsearch")
public class ElasticSearchService {
    private static final String ELASTIC_SEARCH_USER_NAME = <my elastic search username>;
    private static final String ELASTIC_SEARCH_PASSWORD = <my elastic search password>;
    private static final String ELASTIC_SEARCH_ENDPOINT_URL = <my elastic search endpoint url>
    private static final Integer ELASTIC_SEARCH_PORT = 9243;

    @POST
    public void createElasticIndex() throws IOException {

        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        credentialsProvider.setCredentials(AuthScope.ANY,
                new UsernamePasswordCredentials(ELASTIC_SEARCH_USER_NAME, ELASTIC_SEARCH_PASSWORD));

        RestClientBuilder builder = RestClient
                .builder(new HttpHost(
                        ELASTIC_SEARCH_ENDPOINT_URL,
                        ELASTIC_SEARCH_PORT, "https"))
                .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                    @Override
                    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                        return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                    }
                });

        RestHighLevelClient client = new RestHighLevelClient(builder);

        IndexRequest request = new IndexRequest(
                "contacts",
                "doc",
                "1");
        String jsonString = "{" +
                "\"user\":\"frank\"," +
                "\"postDate\":\"2020-03-02\"," +
                "\"message\":\"created this document from Java\"" +
                "}";
        request.source(jsonString, XContentType.JSON);

        try {
            IndexResponse response = client.index(request, RequestOptions.DEFAULT);
            System.out.println(response);

        } catch (ElasticsearchException e) {
            if (e.status() == RestStatus.CONFLICT) {
            }
        }

        client.close();
    }

}

This code creates an index called contacts and adds a document to that index.