Adil Asif
Adil Asif

Reputation: 11

Redis Enterprise Cluster Mode - Pagination Error "StackExchange.Redis.RedisServerException: 'Could not send query to cluster'" and Ops/Sec Discrepancy

I have set up three Redis Enterprise Docker Containers in Clustered Mode. Currently, I’m trying to check the performance of Redis Enterprise in cluster mode after pushing 5 million records. I'm facing two issues:

When I try to search with a paginated query, I get this error:

StackExchange.Redis.RedisServerException: 'Could not send query to cluster'

However, it works fine when querying the to get count of entire dataset.

In the Redis Enterprise Console, only node 1 shows ops/sec, while nodes 2 and 3 show zero ops/sec, even though the database is in clustered mode.

Here’s the clustered setting status:

enter image description here

Docker Containers image attached

enter image description here

i have setup the endpoints in this way

var config = new ConfigurationOptions
{
    EndPoints = {
        { "localhost", 12000 }, { "localhost", 12001 }, { "localhost", 12002 },
    },
};

static void InspectSampleData(IDatabase db)
{
    var ftClient = new Client("orders", db);

    Console.WriteLine("Starting comprehensive database report...");
    Console.WriteLine("------------------------------");

    // Define the query to match both conditions without spaces around '|'
    var baseQuery = "@isfalse:{false} @acco:{10001|10002|10003|10004}";

    // 1. Measure time to get the total count of matching records
    Stopwatch countStopwatch = new Stopwatch();
    countStopwatch.Start();

    var countQuery = new Query(baseQuery)
        .SetNoContent()
        .SetVerbatim()
        .Limit(0, 0); // No records returned, only count it is working fine 

    var countResult = ftClient.Search(countQuery);
    int totalResults = (int)countResult.TotalResults;

    countStopwatch.Stop();
    Console.WriteLine($"Total documents matching both conditions: {totalResults}");
    Console.WriteLine($"Time taken to retrieve count: {countStopwatch.ElapsedMilliseconds} ms");
    Console.WriteLine("------------------------------");

    // 2. Paginate through the dataset in 1-million record offsets, fetching 100 records each
    Console.WriteLine("Starting pagination with offset and limit...");
    Console.WriteLine("------------------------------");

    int pageSize = 100;          // Number of records to fetch per iteration
    int maxOffset = 50_000_00;  // Total records (5 million)
    int increment = 1_000_000;   // Offset increment (1 million)
    int iterations = maxOffset / increment; // Total iterations (5)

    for (int i = 1; i <= iterations; i++)
    {
        int offset = i * increment;
        Console.WriteLine($"Fetching records with offset: {offset}, limit: {pageSize}");

        // Initialize a stopwatch to measure query execution time
        Stopwatch stopwatch = new Stopwatch();
        stopwatch.Start();

        // Perform the paginated query
        var paginatedQuery = new Query(baseQuery)
            .SetVerbatim()
            .Limit(offset, pageSize); // Apply offset and limit

        try
        {
            var paginatedResult = ftClient.Search(paginatedQuery); //error on this part

            // Stop the stopwatch after the query execution
            stopwatch.Stop();

            // Display the time taken and number of documents retrieved
            Console.WriteLine($"Query execution time: {stopwatch.ElapsedMilliseconds} ms");
            Console.WriteLine($"Documents retrieved: {paginatedResult.Documents.Count}");
            Console.WriteLine("------------------------------");

            // Optionally, display the document IDs
            //foreach (var doc in paginatedResult.Documents)
            //{
            //    Console.WriteLine($"Document ID: {doc.Id}");
            //    // Access other fields as needed, e.g., doc["fieldname"]
            //}

            Console.WriteLine("------------------------------");
        }
        catch (RedisTimeoutException ex)
        {
            stopwatch.Stop();
            Console.WriteLine($"Timeout on offset {offset}: {ex.Message}");
            Console.WriteLine("Consider reducing the offset or increasing the timeout settings.");
            Console.WriteLine("------------------------------");
        }
        catch (Exception ex)
        {
            stopwatch.Stop();
            Console.WriteLine($"Error on offset {offset}: {ex.Message}");
            Console.WriteLine("------------------------------");
        }
    }

I am new to redis looked for the solution of this error but unable to find any i believe it can be due to memory issue but on 100k records it is giving correct output.

Upvotes: 0

Views: 30

Answers (0)

Related Questions