CODEWITHSUNDEEP
elasticsearch
Rolando
Rolando

Reputation: 62596

How to get elasticsearch to return more than 10000 results at a time? ES appears to not be honoring the index.max_result_window setting

I tried based on the documentation to create an index then use:

PUT indexname/_settings
{
      "index.max_result_window": 20000
}

When I get the settings, I see the setting is set, but whenever I do a query against it, I keep getting the 10,000 results. I tried creating the index with the setting set, but that didn't work either.

I also tried when making the search query, to include a size parameter of 11,000 and it still did not return.

What is it do I have to do to get the results to return greater than 10,000?

Is there some setting I have to apply to the node, or some other setting applied to the index to get it to work?

I am using the latest version 7.3.1.

Upvotes: 3

Views: 15578

Answers (3)

secana
secana

Reputation: 681

The scroll API is no longer recommended. The current way for pagination on more than 10k results is the search-after API of ElasticSearch. From the docs: this lets you retrieve the next page of hits using a set of sort values from the previous page.

Upvotes: 1

Shivam
Shivam

Reputation: 197

if you are just looking for hits in the JSON response and not the actual documents then add "track_total_hits": true in the search request to get the actual total hits.

POST indexname/_search { "from": 0, "size": 0, "track_total_hits": true }

Upvotes: 3

Tarek Essam
Tarek Essam

Reputation: 4010

You can use scroll API to retrieve more than 10000 records in elastic search as by default, 10000 is the upper cap for the number of documents returned.

What Scroll API basically does is it fetches documents in chunks whose size can be customized by us. We can control the size of document-set returned by using size and a time value. The actual calls take the following forms:

1st Call

In the first call to fetch the documents, you can give the size ( say 5000 docs) and scroll parameter specifying the time in minutes after which search context times out. 

 POST /index/type/_search?scroll=1m
{
    "size": 5000,
    "query": {
        "match" : {
            "title" : "Something"
        }
    }
}

2nd Call ( and every other subsequent call)

In the first call's response, we get a _scroll_id which can be used to retrieve the next chunk of documents.

    POST /_search/scroll 
{
    "scroll" : "1m", 
    "scroll_id" : "XGSKJKSVNNNNNDDDD1233BBNMBBNNN===" 
}

Also, check this answer.

Upvotes: 7

Related Questions