Reputation: 695
I'm testing Cassandra performance in the case of simultaneous read and writes operations.
Latency for READ
operations is important for me, WRITES usually are done once per day, so I'm investigating if the WRITE
operations would affect READ
operations latency.
I have a table of 10_000
records which I READ
from application at 50 RPS. For example, the latency is 10ms, REQUEST_SERIAL_CONSISTENCY
and REQUEST_CONSISTENCY
is QUORUM
for READ
operations.
Query for READs
is: SELECT * from table where id = X
Simultaneously I run another application which insert same records to table (with same ids, so in fact it is UPDATE
operations), the REQUEST_SERIAL_CONSISTENCY
and REQUEST_CONSISTENCY
is ALL
for WRITE
operations. In several threads, I load my Cassandra cluster with batches of INSERT queries size of 60 queries in batch.
Query for WRITEs
is: INSERT into table values(id,..) where id = X;
What I expect it is the latency increasing for READ operations when WRITES is enabled. But latency is stable in general besides random spikes like:
Why hasn't latency been changed, was it wrong to expect uniform latency increase? What about these spikes, can it be connected with MemTable flushes to disk? I can see them in statistic:
nodetool tpstats
MemtableFlushWriter 0 0 4272 0 0
Info of my cluster:
cqlsh> SELECT * FROM system_schema.keyspaces;
keyspace_name | durable_writes | replication
--------------------+----------------+-------------------------------------------------------------------------------------
system_auth | True | {'class': 'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': '1'}
system_schema | True | {'class': 'org.apache.cassandra.locator.LocalStrategy'}
system_distributed | True | {'class': 'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': '3'}
system | True | {'class': 'org.apache.cassandra.locator.LocalStrategy'}
search | True | {'class': 'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': '1'}
system_traces | True | {'class': 'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': '2'}
Upvotes: 0
Views: 890
Reputation: 2310
With 10000 records, I do not expect much change in latency. Latency can get hit due to lot of factor. It could be that Cassandra is doing I/O for compactions, repairs and that could hit your latency even when no write is done. So I assume random spikes you are seeing is because of some compactions running in background. One more thing, when you say you are inserting records in batch of 60, please dont use Cassandra batch for inserting records as they will be slow.
Upvotes: 1