CODEWITHSUNDEEP
python-2.7elasticsearch
RCP
RCP

Reputation: 374

elasticsearch exception SerializationError

From python script i am sending data to elasticsearch server

this will help me to connect to ES

es = Elasticsearch('localhost:9200',use_ssl=False,verify_certs=True)

and by using the bellow code i am able to send all data to my local ES server

es.index(index='alertnagios', doc_type='nagios', body=jsonvalue)

But when i am trying to send data to cloud ES server,the script is executing fine and it is indexing few documents after indexing few documents i am getting following error 

Traceback (most recent call last):
  File "scriptfile.py", line 78, in <module>
    es.index(index='test', doc_type='test123', body=jsonvalue) 
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/__init__.py", line 298, in index
    _make_path(index, doc_type, id), params=params, body=body)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 342, in perform_request
    data = self.deserializer.loads(data, headers.get('content-type'))
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/serializer.py", line 76, in loads
    return deserializer.loads(s)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/serializer.py", line 40, in loads
    raise SerializationError(s, e)
elasticsearch.exceptions.SerializationError: (u'order=0></iframe>', JSONDecodeError('No JSON object could be decoded: line 1 column 0 (char 0)',))

The same script is working fine when i am sending data to my localhost ES server , I don't know why it is not working when i am sending data to cloud instance

Please help me

Upvotes: 2

Views: 7806

Answers (1)

RCP
RCP

Reputation: 374

The problem is resolved by using Bulk indexing method ,when we are indexing to local server it won't be a matter if we index documents one after the other ,but while indexing to cloud instance we have to follow bulk indexing method to overcome from memory issues and connection issues

if we follow bulk indexing method it will index all the documnets once into a elasticsearch and no need to open connection again and again,it won't take much time.

here is my code

from elasticsearch import Elasticsearch, helpers    

jsonobject= {
    '_index': 'index',
    '_type': 'index123',
    '_source':jsonData
} 
actions = [jsonobject]
helpers.bulk(es, actions, chunk_size=1000, request_timeout=200)

Upvotes: 5

Related Questions