Random Data Loss in Redis Cluster During Bulk Operations

Question

Random Data Loss in Redis Cluster During Bulk Operations

Issue Description

I'm experiencing random data loss in my Redis cluster setup. Here's a detailed breakdown of the scenario:

Setup

• Local Redis cluster environment
• NestJS application
• Main async function containing 13 sub-functions
• Each function handles setting ~100,000 records to Redis in a loop

Observed Behavior

• All data successfully passes through the application logic and reaches the Redis SET operation
• Random loss of approximately 100-1,000 records
• No pattern in which records are lost

Environment Details

• Each cluster node allocated:
  • 1 core CPU
  • 600MB memory
• Current memory usage: 100-200MB per node (well within limits)
• No network connectivity issues observed

Current Configuration

environment.clusterMode
    ? new Redis.Cluster(
          [
              {
                  host: environment.redisCluster.clusterHost,
                  port: parseInt(environment.redisCluster.clusterPort),
              },
          ],
          {
              redisOptions: {
                  username: environment.redisCluster.clusterUsername,
                  password: environment.redisCluster.clusterPassword,
              },
              maxRedirections: 300,
              retryDelayOnFailover: 300,
          },
      )
    : new Redis({
          host: environment.redisHost,
          port: parseInt(environment.redisPort),
      })

Troubleshooting Done

1. Verified all data reaches the SET operation (no logical errors in application code)
2. Confirmed adequate memory allocation
3. Monitored cluster performance during operations
4. Checked network stability

Any insights or suggestions would be greatly appreciated!