David
David

Reputation: 87

RediSearch - searching for particular word which occurs in many records take long time. How to improve it?

I have addresses database (as hashes) with about 30 millions records. I was adding text index to all addresses fields. Searching looks ok until I want to search word which occur in many records. For example searchin word "London" which occur in about 2500000 records took 4,5 seconds (FT.SEARCH idx:a4 london LIMIT 0 2). Is it any possibility to improve this result, any changes to make? Thank you for help.

Upvotes: 1

Views: 803

Answers (1)

Meir Shpilraien
Meir Shpilraien

Reputation: 524

If you do not care about getting the first 2 results sorted by scoring (calculated by tfidf), you can use FT.AGGREGATE which will just return after finding the first 2 results (without getting all the results, calculating the score, sort them, and get the first 2 results). It should look like this:

FT.AGGREGATE idx:a4 london LIMIT 0 2

Notice that you should use LOAD to decide which fields to return from the hash. Please refer here for the full FT.AGGREGATE documentation: https://oss.redislabs.com/redisearch/Aggregations/

Again if you chose to use it, know that you are losing sorting by results score.

Upvotes: 2

Related Questions