CODEWITHSUNDEEP
pythondjangoelasticsearchdjango-haystack
eLRuLL
eLRuLL

Reputation: 18799

django haystack: how to iterate over all indexed elements

I am trying to iterate over a Search Queryset with haystack, but it throws me this error:

Result window is too large, from + size must be less than or equal to: [10000] but was [11010]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level parameter.

Is there a way to iterate over all indexed elements? (let's say I have several million records).

Upvotes: 2

Views: 467

Answers (1)

WisdomPill
WisdomPill

Reputation: 800

max_result_window is an index setting that you can change if you want but most of the time you don't have to, because if you'd like to iterate on all your documents using the search api is not the way you should go. Try with a scan and scroll api.

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html

And a personal note: I use elasticsearch with django and I found haystack difficult to use as opposed to elasticsearch-dsl. Try to have a look to elasticsearch-dsl-py. https://github.com/elastic/elasticsearch-dsl-py

Upvotes: 1

Related Questions