CODEWITHSUNDEEP
javaelasticsearch
sunsets
sunsets

Reputation: 411

NullPointerException when I'm executing jestClient to index into Elasticsearch

I've been struggling with NullPointerException when I'm executing jestClient to index into Elasticsearch for two days.

java.lang.NullPointerException
at java.util.Calendar.setTime(Calendar.java:1770)
at java.text.SimpleDateFormat.format(SimpleDateFormat.java:943)
at java.text.SimpleDateFormat.format(SimpleDateFormat.java:936)
at java.text.DateFormat.format(DateFormat.java:345)
at com.google.gson.DefaultDateTypeAdapter.write(DefaultDateTypeAdapter.java:88)
at com.google.gson.DefaultDateTypeAdapter.write(DefaultDateTypeAdapter.java:40)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:125)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:243)
at com.google.gson.internal.bind.ObjectTypeAdapter.write(ObjectTypeAdapter.java:107)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:97)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:61)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:69)
at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:208)
at com.google.gson.internal.bind.MapTypeAdapterFactory$Adapter.write(MapTypeAdapterFactory.java:145)
at com.google.gson.Gson.toJson(Gson.java:669)
at com.google.gson.Gson.toJson(Gson.java:648)
at com.google.gson.Gson.toJson(Gson.java:603)
at com.google.gson.Gson.toJson(Gson.java:583)
at io.searchbox.action.AbstractAction.getData(AbstractAction.java:167)
at io.searchbox.core.Bulk.getData(Bulk.java:102)
at io.searchbox.client.http.JestHttpClient.prepareRequest(JestHttpClient.java:115)
at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:64)
at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:60)
at com.example.testcb.databus.RealTimeDataSink.saveToElasticSearch(RealTimeDataSink.java:154)
at com.example.testcb.databus.RealTimeDataSink.productionMessageSink(RealTimeDataSink.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:180)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:112)
at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:48)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:109)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
at org.springframework.cloud.stream.binding.DispatchingStreamListenerMessageHandler$ConditionalStreamListenerHandler.handleMessage(DispatchingStreamListenerMessageHandler.java:122)
at org.springframework.cloud.stream.binding.DispatchingStreamListenerMessageHandler.handleRequestMessage(DispatchingStreamListenerMessageHandler.java:75)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:109)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:148)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:121)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:89)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:425)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:375)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:360)
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:271)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:188)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:115)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
at org.springframework.integration.channel.FixedSubscriberChannel.send(FixedSubscriberChannel.java:70)
at org.springframework.integration.channel.FixedSubscriberChannel.send(FixedSubscriberChannel.java:64)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:188)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.access$000(KafkaMessageDrivenChannelAdapter.java:54)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:288)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:279)
at org.springframework.kafka.listener.adapter.RetryingAcknowledgingMessageListenerAdapter$1.doWithRetry(RetryingAcknowledgingMessageListenerAdapter.java:79)
at org.springframework.kafka.listener.adapter.RetryingAcknowledgingMessageListenerAdapter$1.doWithRetry(RetryingAcknowledgingMessageListenerAdapter.java:73)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:180)
at org.springframework.kafka.listener.adapter.RetryingAcknowledgingMessageListenerAdapter.onMessage(RetryingAcknowledgingMessageListenerAdapter.java:73)
at org.springframework.kafka.listener.adapter.RetryingAcknowledgingMessageListenerAdapter.onMessage(RetryingAcknowledgingMessageListenerAdapter.java:39)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:794)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:738)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.access$2200(KafkaMessageListenerContainer.java:245)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ListenerInvoker.run(KafkaMessageListenerContainer.java:1031)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)

Let me add my code to understand the situation.

public void saveToElasticSearch(String type, JSONArray outputs) throws Exception{

    if (outputs != null) {

        boolean indexExists = jestClient.execute(new IndicesExists.Builder(INDEX_NAME).build()).isSucceeded();

        try {
            Bulk.Builder bulkIndexBuilder = new Bulk.Builder();

            for(Object object : outputs) {
                bulkIndexBuilder.addAction(new Index.Builder(object).index(INDEX_NAME).type(type).build());
                System.out.println(object);
            }
            jestClient.execute(bulkIndexBuilder.build());

        } catch(NullPointerException e) {
            e.printStackTrace();
        }
    }
}

To be honest, I have no idea how to figure this out. Here are objects printed in a code.

{"timeStamp":1509496260000,"data":[com.example.testcb.domain.RealTimeProduction@1ab971fb],"equipment":"LA-081","unitTime":Wed Nov 01 09:31:00 KST 2017}
{"timeStamp":1509496320000,"data":[com.example.testcb.domain.RealTimeProduction@8e0a2c3],"equipment":"LA-081","unitTime":Wed Nov 01 09:32:00 KST 2017}
{"timeStamp":1509496200000,"data":[com.example.testcb.domain.RealTimeProduction@426b9507],"equipment":"LA-091","unitTime":Wed Nov 01 09:30:00 KST 2017}

Maybe I can be related to an object which has java address above? Any idea of this?

Upvotes: 4

Views: 627

Answers (1)

jccampanero
jccampanero

Reputation: 53461

As the stack trace you provided in the question indicates, the error is related to the fact that the Gson library, used internally by Jest for JSON processing, is not handling appropriately null values when processing some date values contained in your data.

The problem was reported in this and especially this other Github issues, and fixed in Gson 2.8.2.

The aforementioned issues indicate that everything should work properly with Gson 2.8.0, the version bounded with Jest, but please, in any case, due to the fact that AFAIK is not possible to define your own JSON (de)serializers or tweak the Gson existing ones when configuring Jest, try providing a version of Gson in which the problem is solved in your Maven or Gradle as a first level dependency, forcing your project to use it. For example:

<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.8.2</version>
</dependency>

Upvotes: 3

Related Questions