CODEWITHSUNDEEP
node.jselasticsearchelasticsearch-pluginelasticsearch-5
Anouar Kacem
Anouar Kacem

Reputation: 645

Elasticsearch Pagination From - Size Result window is too large

I would like to have pages for my documents, each Page should have 1000 results.

Documents size: 95.000 docs

So starting from product number 10k, I would like to have 1K results from that point, however, using this query, I got the error mentioned below

Query:

this.es.search({
  index: indexName,
  type: type,
  from: 10000,
  size: 1000,
  body: {}
});

Error:

{
    "msg": "[query_phase_execution_exception] Result window is too large, from + size must be less than or equal to: [10000] but was [16000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.",
    "path": "/products/Product/_search",
    "query": {},
    "body": "{\"from\":10000,\"size\":1000}",
    "statusCode": 500,
    "response": "{\"error\":{\"root_cause\":[{\"type\":\"query_phase_execution_exception\",\"reason\":\"Result window is too large, from + size must be less than or equal to: [10000] but was [16000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":0,\"index\":\"products\",\"node\":\"bTGkra6dTB-0Z_8jVUNByQ\",\"reason\":{\"type\":\"query_phase_execution_exception\",\"reason\":\"Result window is too large, from + size must be less than or equal to: [10000] but was [16000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.\"}}]},\"status\":500}"
}

How can I increase the paging result/ limit?

Also any other method is welcome, thank you.

Upvotes: 1

Views: 9843

Answers (4)

Sharanabasu Angadi
Sharanabasu Angadi

Reputation: 4382

Increase max_result_window as below, it works

PUT indexName/_settings
{
  "index": {
    "max_result_window": 10000000
  }
}

Upvotes: 2

Anouar Kacem
Anouar Kacem

Reputation: 645

Well, I could increase the limit of pagination using this CURL

curl -XPUT "http://localhost:9200/my_index/_settings" -d '{ "index" : { "max_result_window" : 500000 } }'

Upvotes: 1

deerawan
deerawan

Reputation: 8443

Based on elasticsearch documentation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-from-size.html

Note that from + size can not be more than the index.max_result_window index setting which defaults to 10,000

As alternative, you can try to use scroll or searchAfter. For more real-time query, searchAfter is more suitable.

Scroll documentation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html

SearchAfter documentation https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-search-after.html

Upvotes: 2

H30
H30

Reputation: 124

As the error shows Elastic will only display 10000 rows.

so to display more than that number you need to use the scroll function

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html

An example of a js search with scroll can be found here.

async await with elasticsearch search/scroll

Upvotes: 1

Related Questions