CODEWITHSUNDEEP
pythonpython-2.7elasticsearch
Johann Gomes
Johann Gomes

Reputation: 4163

Connection Timeout with Elasticsearch

from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch()

doc = {
    'author': 'kimchy',
    'text': 'Elasticsearch: cool. bonsai cool.',
    'timestamp': datetime(2010, 10, 10, 10, 10, 10)
}
res = es.index(index="test-index", doc_type='tweet', id=1, body=doc)
print(res['created'])

This simples code is returning the following error:

elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10))

Very strange, because the server is ready and set (http://localhost:9200/ is returning some json).

Upvotes: 125

Views: 209875

Answers (15)

bereket gebredingle
bereket gebredingle

Reputation: 12996

extend the requestTimeout:

client = new elasticsearch.Client({
        host          : 'http://localhost:9200',
        requestTimeout: 60000
    });

Upvotes: -1

d-dutchview
d-dutchview

Reputation: 98

so there are 2 different timeouts in ES. There is a client timeout when you connect to the instance and when you search for something.

for example

def get(self, json):
    return self.es.search(
        index="data",
        body=json
        , request_timeout=999
    )

I had this set as request_timeout=None but this does not mean that there is no timeout no it just reverts back to the default 10 seconds.

Upvotes: 2

Vishal D.
Vishal D.

Reputation: 31

I faced the same error while running python elasticsearch client's es.index() method in a docker container.

The reason was that the disk space was >90% full. Elasticsearch cluster detects the disk usage and if its >90% full, it's unable to start the shard.

Once I freed up some space, connection timeout error disappeared.

Upvotes: 3

Morten Jensen
Morten Jensen

Reputation: 5936

I had a similar problem.

> client.ping()
True
> client.info()
ObjectApiResponse({'name': .... })

Searching also worked, but indexing failed with an exception: ConnectionTimeout: Connection timed out.

The reason was that my disk was almost full. I freed up some space and then I could index without errors...

Upvotes: 0

MadeOfAir
MadeOfAir

Reputation: 3183

I got this error when specifying the fields option in my query for some reason. Removing the option made the query work (in OpenSearch at least):

Before:

elasticsearch.helpers.scan(es, index="my_index", query={"query": {"match_all": {}}, "fields": ["zip_code", "polygon"]}, size=5000, scroll="15m")

After:

elasticsearch.helpers.scan(es, index="my_index", query={"query": {"match_all": {}}}, size=5000, scroll="15m")

Upvotes: 0

mikey
mikey

Reputation: 2644

I got this timeout problem when trying out elasticsearch 7.11.1 (installed using apt install) at port 9202 (9200 is in use by an elasticsearch 6) in ubuntu.

I googled around for nearly two hours before finally fixing it. It turns out I nneed to set (comment out) one line in /etc/elasticsearch/elasticserch.yml

cluster.initial_master_nodes: ["node-1"]  # earlier in the same file: node.name: node-1

I hope this may help someone.

Upvotes: 0

Anupam
Anupam

Reputation: 15610

The reasons for the timeout could be many and it seems worth checking the logs on elasticsearch side (logs/elasticsearch.log) to see the detailed error. In our case, the error on ES was:

primary shard is not active Timeout: [1m]

As described in this post, this was because our disk was full. We had resized it (and the partition) a day ago to take care of that but ES needs to be restarted if the high/low watermark has been hit once (we are on 5.5.x) which we had not done.

Simply restarting the ES on production resolved the issue for us.

Upvotes: 7

Ghasem
Ghasem

Reputation: 15573

Try setting timeout in Elasticsearch initialization:

es = Elasticsearch([{'host': HOST_ADDRESS, 'port': THE_PORT}], timeout=30)

You can even set retry_on_timeout to True and give the max_retries an optional number:

es = Elasticsearch([{'host': HOST_ADDRESS, 'port': THE_PORT}], timeout=30, max_retries=10, retry_on_timeout=True)

Upvotes: 45

Karl Lorey
Karl Lorey

Reputation: 1616

Two options that help:

1: increase the timeout

Setting a timeout solved this problem for me. Note that newer versions need a unit, e.g. timeout="60s":

es.index(index=index_name, doc_type="domains", id=domain.id, body=body, timeout="60s")

Without a unit, for example by setting timeout=60, you'll get

elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'failed to parse setting [timeout] with value [60] as a time value: unit is missing or unrecognized')

2: reduce text length

It also helps to reduce the text length, e.g. by cutting of long texts, so elastic can store the text faster which will avoid timeouts, too:

es.index(index=index_name, doc_type="domains", id=domain.id, body=text[:5000], timeout="60s")

Upvotes: 3

Mukund
Mukund

Reputation: 946

The connection timed out problem could occur if you are using Amazon Elastic Search service.

es = Elasticsearch([{'host': 'xxxxxx.us-east-1.es.amazonaws.com', 'port': 443,  'use_ssl': True}])

The above python code where you override the default port from 9200 to 443 and setting the SSL to true will resolve the issue.

If no port is specified, it is trying to connect to the port 9200 in the specified host and fails after time out

Upvotes: 22

vlyubin
vlyubin

Reputation: 648

Note that one of the common reasons for timeouts when doing es.search (or es.index) is large query size. For example, in my case of a pretty large ES index size (> 3M documents), doing a search for a query with 30 words took around 2 seconds, while doing a search for a query with 400 words took over 18 seconds. So for a sufficiently large query even timeout=30 won't save you. An easy solution is to crop the query to the size that can be answered below the timeout.

Increasing timeout or doing retries on timeout will help you if the cause was in traffic, otherwise this might be your culprit.

Upvotes: 7

GGEv
GGEv

Reputation: 933

my personal problem was solved with (timeout = 10000) which was practically never reached because the entries on server were only 7.000 but it had heavy traffic and its resources were being hogged and that was why the connection was dropping

Upvotes: 3

whoopididoo
whoopididoo

Reputation: 431

This is nothing to do with increasing your timeout to 30 seconds. Do people actually think that elastic search should need up to 30 seconds to return one tiny hit?

The way I fixed this problem was go to config/elasticsearch.yml uncomment the following

http.port: 9200
network.host: 'localhost'

Network.host might be set to 192.168.0.1 which might work But I just changed it to 'localhost'

Upvotes: 9

Rahul
Rahul

Reputation: 2635

By default, the timeout value is set to 10 secs. If one wants to change the global timeout value, this can be achieved by setting the flag timeout=your-time while creating the object.

If you have already created the object without specifying the timeout value, then you can set the timeout value for particular request by using request_timeout=your-time flag in the query.

es.search(index="my_index",
          doc_type="document",
          body=get_req_body(),
          request_timeout=30)

Upvotes: 127

Mir Ilias
Mir Ilias

Reputation: 515

elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)) mean the request didn't end in the specified time (by default, timeout=10).

This will work with 30 seconds :

res = es.index(index="test-index", doc_type='tweet', id=1, body=doc, timeout=30)

Upvotes: 6

Related Questions