CODEWITHSUNDEEP
cassandradatastax-enterprise
Hanson
Hanson

Reputation: 33

Cassandra In-Memory option

There is an In-Memory option introduced in the Cassandra by DataStax Enterprise 4.0: http://www.datastax.com/documentation/datastax_enterprise/4.0/datastax_enterprise/inMemory.html But with 1GB size limited for an in-memory table.

Anyone know the consideration why limited it as 1GB? And possible extend to a large size of in-memory table, such as 64GB?

Upvotes: 3

Views: 2602

Answers (1)

Carlo Bertuccini
Carlo Bertuccini

Reputation: 20021

To answer your question: today it's not possible to bypass this limitation. In-Memory tables are stored within the JVM Heap, regardless the amount of memory available on single node allocating more than 8GB to JVM Heap is not recommended. The main reason of this limitation is that Java Garbage Collector slow down when dealing with huge memory amount.

However if you consider Cassandra as a distributed system 1GB is not the real limitation.

(nodes*allocated_memory)/ReplicationFactor

allocated_memory is max 1GB -- So your table may contains many GB in memory allocated in different nodes.

I think that in future something will improve but dealing with 64GB in memory it could be a real problem when you need to flush data on disk. One more consideration that creates limitation: avoid TTL when working with In-Memory tables. TTL creates tombstones, a tombstone is not deallocated until the GCGraceSeconds period passes -- so considering a default value of 10 days each tombstone will keep the portion of memory busy and unavailable, possibly for long time.

HTH, Carlo

Upvotes: 1

Related Questions