CODEWITHSUNDEEP
redisignitestackexchange.redisdistributed-caching
Gopu_Tunas
Gopu_Tunas

Reputation: 194

Data Partition and data persistence in in-memory distributed cache

I am exploring different distributed in-memory databases for use in our applications. I have come across one scenario and want to know if it is possible in Redis Cache and Apache Ignite Cache?

  1. Let's say my In-memory distributed cache size is 15GB
  2. I am using partitioning my data (for e.g. in Redis - Redis Cluster and similar way in Ignite)
  3. Once any of node in a cluster reaches threshold level in terms of memory Does it distribute data automatically across of node in Redis? Does it distribute data automatically across of node in Ignite? When eviction policy is applied in both Redis and Ignite? Only after all nodes reach to there memory capacity?
  4. What if one or more nodes go down in cluster or new nodes added in cluster?
  5. What about data backup? (I know concept of Redis Sentinel for Redis)

I am looking for this kind of scenario (is available in Redis Cache?) explained in this link of overview section https://apacheignite.readme.io/docs/distributed-persistent-store

Can below thing achievable in Redis Cache?

Let's say I have 100 records. Cache can hold only 40 records (most frequently used) and 100 records on disk file (Not in any other database).

  1. So if anything from these 100 records are requested I never have to go actual database (for e.g. Sybase db)?
  2. If key found in 100 records but its not present in in-memory cache (40 records) then get that key, put in in-memory cache and swap out other key using eviction policy to a disk file (But here on disk i have 100 records always)
  3. If key not present in Cache and disk file then only we have to go database (Sybase) and fetch it.

Upvotes: 1

Views: 546

Answers (2)

dmagda
dmagda

Reputation: 1785

With Ignite (and an absolute majority of other distributed systems including Redis) you just scale out your cluster once the monitoring shows that a node is about to go beyond allocated RAM space. Just add one extra node to your Ignite cluster and data will be automatically redistributed.

Moreover, with Ignite you have an option to enable its native persistence to store more data than fits in RAM and to have instantaneous restarts. There is no need to preload anything in RAM on restarts, Ignite reads from disk.

Hope it helps. Based on the provided details, Ignite is a perfect fit for you.

Upvotes: 1

alamar
alamar

Reputation: 19313

When using Ignite, you should make sure you never run out of data region memory. There's no data redistribution and the node will just become unavailable.

When node is down its data will be distributed to other nodes (if there are enough backups configured), but above-mentioned restiction still applies - no nodes should run out of memory.

If you add new nodes some data will be redistributed to them.

If you use persistent mode, you won't run out of memory since your data will be just written to disk. Make sure to not run out of disk space :) When a node is down or added, the behavior will be governed by Baseline Topology (usually, data will not be moved immediately).

Upvotes: 0

Related Questions