CODEWITHSUNDEEP
elasticsearch
Sumit G
Sumit G

Reputation: 456

ConnectionClosedException while using Elastic RestHighLevelClient bulk request

I am using the following code for RestHighLevelClient in Elastic Search.

val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(conf.value.getString("elkUserName"), conf.value.getString("elkPassword")))
val builder = RestClient.builder(new HttpHost(conf.value.getString("elkIp"), Integer.valueOf(conf.value.getString("elkPort"))))
    .setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
        //set timeout
        override def customizeRequestConfig(requestConfigBuilder: RequestConfig.Builder): RequestConfig.Builder = requestConfigBuilder.setConnectTimeout(Integer.valueOf(conf.value.getString("elkWriteTimeOut"))).setSocketTimeout(Integer.valueOf(conf.value.getString("elkWriteTimeOut")))
    }).setHttpClientConfigCallback(new HttpClientConfigCallback() {
        override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = {
            httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
        }
    })
client = new RestHighLevelClient(builder)
val requestBuilder = RequestOptions.DEFAULT.toBuilder
requestBuilder.setHttpAsyncResponseConsumerFactory(
    new HttpAsyncResponseConsumerFactory.HeapBufferedResponseConsumerFactory(1024 * 1024 * 1024))


var request = new BulkRequest()
request.setRefreshPolicy("wait_for")
var sizeOfRequest = 1 L
newListOfMap.foreach {
    vals =>
        val newMap = vals.asJava
    request.add(new IndexRequest(indexName).source(newMap))

}
client.bulk(request, requestBuilder.build)

But I am getting the following Exception

java.lang.NoSuchMethodError:
org.apache.http.ConnectionClosedException: method <init>()V not found
  at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.endOfInput(HttpAsyncRequestExecutor.java:356)
  at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:261)
  at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
  at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
  at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
  at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
  at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
  at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
  at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
  at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
  at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
  at java.lang.Thread.run(Thread.java:748)
org.apache.http.ConnectionClosedException: Connection closed
unexpectedly at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:778)
  at org.elasticsearch.client.RestClient.performRequest(RestClient.java:218)
  at org.elasticsearch.client.RestClient.performRequest(RestClient.java:205)
  at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1454)
  at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1424)
  at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1394)
  at org.elasticsearch.client.RestHighLevelClient.bulk(RestHighLevelClient.java:492)
  at Utils.ELKUtil$.postDataToELK(ELKUtil.scala:59)

NOTE: The above code is working for smaller size of request but getting above error while posting bigger size of request. Please suggest.

Upvotes: 2

Views: 5062

Answers (2)

Tabish Mir
Tabish Mir

Reputation: 799

If your project is using both httpcore and httpcore-nio dependencies, ensure that both of their versions are either simultaneously <= 4.4.10 or > 4.4.10.

The suggestion by Harshit is partially what caused this issue in my case. On closer inspection of the logs, it is evident that HttpAsyncRequestExecutor is calling a default constructor called ConnectionClosedException, however, this default constructor does not exist in the destination

The HttpAsyncRequestExecutor is a class of httpcore-nio package. ConnectionClosedException is a class of the httpcore package. This issue starts occurring after v4.4.10, beyond which point the ConnectionClosedException class has a default constructor included which HttpAsyncRequestExecutor calls. For versions <= 4.4.10, the default constructor is not present, however, it is called with a parameter in HttpAsyncRequestExecutor. Thus the versions for both libraries should be either above or below v4.4.10 when being used together.

Upvotes: 2

harshit
harshit

Reputation: 7951

we had a similar issue and the fix we that the service has httpcore-4.4.3 where as the elastic client requires httpcore-4.4.12. So we updated all HTTP dependencies to map the version needed for elastic-rest-high-level-client

https://mvnrepository.com/artifact/org.elasticsearch.client/elasticsearch-rest-client/7.5.2

Upvotes: 0

Related Questions