CODEWITHSUNDEEP
redisnode-redisredis-clusterredisearchredisjson
peterhanania
peterhanania

Reputation: 23

Unable to get all data from redisearch when sharding redis

When I try to get some data from redis using the redisearch command; ft.search index:something * I only get a small portion of what exists across all my redis nodes.

In my case I got 5 entries out of 19. I have 6 nodes, using redisjson + redisearch.

Does anyone how to solve this issue? Using the node-redis npm package; I create my indexes and use redisearch commands.

I tried looking through PARTITIONS and RSCoordinator, nothing seemed to help

Upvotes: 2

Views: 577

Answers (2)

feder
feder

Reputation: 2058

No. It is not a timeout problem. At least until Version 7.2 it is a deliberate choice of Redis to not offer the full version of the RSCoordinator, which is now part of RediSearch - for the enterprise version.

RediSearch has a distributed cluster version that scales to billions of documents across hundreds of servers. At the moment, distributed RediSearch is available as part of Redis Cloud and Redis Enterprise Software.

The open source version merely gets the RSCoordinatorLight. This means, indices are NOT synced. Thus, despite you see your records being inserted into the database and sharded, the query running against the cluster would not have a chance to pickup the records, which reside on the other 2 nodes (assuming you have a cluster of 3 nodes).

Thus, forget to use Redis Cluster as open source if you intend to use any other search functionality than get-by-key. That is true for:

  • searching json attributes
  • searching hashset attributes
  • searching fulltext/stemming
  • sorting queries
  • etc.

Basically all the standard functions one would need.

Workaround To get around this problem, there are 3 options:

  • buying the enterprise license
  • one needs to managed the assignments/creation of the indices manually. Can be done. I find that a recipe for disaster in most cases.
  • one implements the RSCoordinator by building it manually. I personally never managed to build it and then have successful results. Would be nice if REDIS or a member would deliver a manual. https://redis.io/docs/interact/search-and-query/deprecated/development/

Upvotes: 0

oshadmi
oshadmi

Reputation: 171

Partial results could be due to timeout. Please try adding TIMEOUT 0 to FT.SEARCH command, to see you get all results. See also https://redis.io/docs/stack/search/configuring/#timeout

Upvotes: 1

Related Questions