CODEWITHSUNDEEP
indexingredisstackexchange.redis
JavaMan
JavaMan

Reputation: 534

REDIS search does not return results consistently

I am reading events from Kafka and depositing into REDIS. Then, we read events using Python and in case we don’t find events we drop/re-create the index.

However, I noticed at times even after re-creating the index I still don’t find events.

I have couple of questions -

[Q1] Is re-indexing a good approach where we are continuously getting a huge flow of events?

[Q2] Also, I noticed during REDIS search at times I do get events and then at another instance query does not return results, can this be related to dropping / re-creating index?

[Q3] Is there a better standard approach to ensure JSON are deposited / retrieved consistenly.

[Q4] Is there an explanation as to why at times everything just seems to work fine continuously for several hours and then does not work at all for few hours.

I would appreciate alternate approaches for this simple use case as I am fairly new to REDIS

Upvotes: 0

Views: 1176

Answers (1)

Sparko Sol
Sparko Sol

Reputation: 717

Here are some points. I hope it would be helpful.

[Q1]

Re-indexing can be a good approach if you are getting a huge flow of events, as it will help keep your data up-to-date. However, you also need to make sure that your Redis instance is able to handle the increased load otherwise Redis can become very slow when it is re-indexing and may not be able to keep up with the flow of event. And in that case, you may need to scale your Redis instance to handle the load or incremental indexing may be a better option.

[Q2]

There could be many reasons why search results vary, but it is possible that the index is being dropped and re-created, which would cause the event to not be found. There are a few other things that could be happening:

  1. There could be an issue with the search algorithm, which would cause search results to not be returned.
  2. The data in the database could be changing, which would cause search results to not be returned.

[Q3]

There is no one standard approach to ensure JSON are deposited or retrieved consistently. However, some best practices you may consider include using a library or tool that supports serialization and deserialization of JSON data, validating input and output data, and using an editor that highlights errors in JSON syntax. You can also use locking mechanism to ensure that only one process can write to the Redis instance at a time, or using a queue to buffer writes to Redis. Also, different developers may have different preferences and opinions on the best way to handle JSON in Redis. Some possible methods include using commands such as JSET or JGET to manage JSON objects, or using a library such as JRedis to simplify the process.

[Q4]

Redis can be temperamental, and its behavior can vary depending on the specific configuration and usage scenarios. There is no specific explanation for this behavior, but it could be due to various factors such as load on the Redis server, network conditions, or other applications using the same Redis instance. In that case, server will not be able to handle requests properly and will stop responding. If everything is working fine for a few hours and then suddenly stops working, you can try restarting Redis or checking the logs for any errors that may have occurred.

Upvotes: 2

Related Questions