CODEWITHSUNDEEP
performancecassandrascyllacassandra-stress-tool
Major Payne
Major Payne

Reputation: 1

interpret results of cassandra stress test

We have scylla cluster with tree node. Each node is in its own datacenter. Тo evaluate cluster performance i use cassandra stress test with this parameters:

cassandra-stress write "no-warmup  cl=QUORUM" -rate threads=1 -schema "replication(strategy=NetworkTopologyStrategy, DC1=1, DC2=1, DC3=1)" -mode cql3 native connectionsPerHost=80 protocolVersion=4 user=cassandra password=cassandra -node 10.0.0.3

Result:

type       total ops,    op/s,    pk/s,   row/s,    mean,     med,     .95,     .99,    .999,     max,   time,   stderr, errors,  gc: #,  max ms,  sum ms,  sdv ms,      mb
total,            84,      84,      84,      84,    10.3,     5.0,    45.8,    59.3,    65.9,    65.9,    1.0,  0.00000,      0,      0,       0,       0,       0,       0
total,           145,      61,      61,      61,    16.3,     5.5,    45.1,    46.2,    47.2,    47.2,    2.0,  0.11899,      0,      0,       0,       0,       0,       0
total,           197,      52,      52,      52,    19.3,     5.4,    44.5,    46.0,    46.7,    46.7,    3.0,  0.12423,      0,      0,       0,       0,       0,       0
total,           258,      61,      61,      61,    15.9,     5.3,    44.2,    44.4,    46.0,    46.0,    4.0,  0.09670,      0,      0,       0,       0,       0,       0
total,           325,      67,      67,      67,    15.3,     5.1,    45.4,    46.7,    47.0,    47.0,    5.0,  0.07708,      0,      0,       0,       0,       0,       0
total,           388,      63,      63,      63,    15.9,     5.2,    44.3,    46.2,    47.9,    47.9,    6.0,  0.06482,      0,      0,       0,       0,       0,       0
total,           455,      67,      67,      67,    14.8,     5.3,    46.0,    46.3,    46.5,    46.5,    7.0,  0.05547,      0,      0,       0,       0,       0,       0
total,           518,      63,      63,      63,    15.5,     5.3,    44.0,    46.2,    46.9,    46.9,    8.0,  0.04891,      0,      0,       0,       0,       0,       0
total,           576,      58,      58,      58,    17.2,     5.3,    45.3,    45.8,    48.0,    48.0,    9.0,  0.04541,      0,      0,       0,       0,       0,       0
total,           634,      58,      58,      58,    17.0,     5.3,    44.4,    46.1,    46.2,    46.2,   10.0,  0.04231,      0,      0,       0,       0,       0,       0
total,           704,      70,      70,      70,    14.3,     4.8,    44.1,    46.6,    46.6,    46.6,   11.0,  0.03908,      0,      0,       0,       0,       0,       0
total,           775,      71,      71,      71,    14.4,     4.8,    43.8,    44.1,    46.7,    46.7,   12.0,  0.03650,      0,      0,       0,       0,       0,       0
total,           847,      72,      72,      72,    13.5,     5.0,    41.7,    44.2,    45.7,    45.7,   13.0,  0.03438,      0,      0,       0,       0,       0,       0
total,           914,      67,      67,      67,    15.1,     5.1,    43.8,    43.9,    44.1,    44.1,   14.0,  0.03191,      0,      0,       0,       0,       0,       0
total,           972,      58,      58,      58,    17.0,     5.1,    45.7,    46.6,    47.1,    47.1,   15.0,  0.03089,      0,      0,       0,       0,       0,       0
total,          1031,      59,      59,      59,    17.0,     4.9,    44.2,    44.8,    45.9,    45.9,   16.0,  0.02966,      0,      0,       0,       0,       0,       0
total,          1085,      54,      54,      54,    18.7,     4.8,    44.0,    44.1,    44.7,    44.7,   17.0,  0.02975,      0,      0,       0,       0,       0,       0
total,          1165,      80,      80,      80,    12.1,     4.7,    43.8,    44.3,    45.6,    45.6,   18.0,  0.03075,      0,      0,       0,       0,       0,       0
total,          1224,      59,      59,      59,    17.3,     5.2,    43.9,    46.9,    46.9,    46.9,   19.0,  0.02963,      0,      0,       0,       0,       0,       0
total,          1294,      70,      70,      70,    14.2,     4.8,    43.9,    44.3,    46.4,    46.4,   20.0,  0.02833,      0,      0,       0,       0,       0,       0
total,          1363,      69,      69,      69,    14.2,     4.7,    44.2,    46.7,    46.8,    46.8,   21.0,  0.02707,      0,      0,       0,       0,       0,       0
total,          1432,      69,      69,      69,    14.3,     4.9,    44.0,    44.3,    48.2,    48.2,   22.0,  0.02591,      0,      0,       0,       0,       0,       0
total,          1494,      62,      62,      62,    16.3,     4.9,    44.3,    46.8,    47.3,    47.3,   23.0,  0.02492,      0,      0,       0,       0,       0,       0
total,          1557,      63,      63,      63,    15.7,     5.5,    45.4,    47.3,    47.4,    47.4,   24.0,  0.02395,      0,      0,       0,       0,       0,       0
total,          1620,      63,      63,      63,    16.2,     5.0,    45.2,    46.6,    46.9,    46.9,   25.0,  0.02305,      0,      0,       0,       0,       0,       0
total,          1687,      67,      67,      67,    14.9,     4.8,    44.1,    45.5,    45.6,    45.6,   26.0,  0.02216,      0,      0,       0,       0,       0,       0
total,          1743,      56,      56,      56,    17.8,     4.8,    44.0,    46.3,    47.3,    47.3,   27.0,  0.02203,      0,      0,       0,       0,       0,       0
total,          1804,      61,      61,      61,    16.0,     4.8,    43.8,    44.4,    45.4,    45.4,   28.0,  0.02138,      0,      0,       0,       0,       0,       0
total,          1877,      73,      73,      73,    13.5,     4.8,    44.1,    44.5,    44.8,    44.8,   29.0,  0.02101,      0,      0,       0,       0,       0,       0
total,          1941,      64,      64,      64,    15.9,     5.0,    44.0,    46.7,    47.3,    47.3,   30.0,  0.02032,      0,      0,       0,       0,       0,       0
total,          2003,      62,      62,      62,    16.0,     4.8,    44.0,    44.1,    44.2,    44.2,   31.0,  0.01975,      0,      0,       0,       0,       0,       0
total,          2004,     215,     215,     215,    16.9,    16.9,    16.9,    16.9,    16.9,    16.9,   31.0,  0.07564,      0,      0,       0,       0,       0,       0

As we see, we have a relatively small mead and mean latency, but catastrophic .95 and .99 latency. And main question: why? And why this values so different?

Upvotes: 0

Views: 157

Answers (1)

dor laor
dor laor

Reputation: 870

It's not a goo practice to have a single node per data center. It could be the source of latency. You also run a single thread in the client but that's more of a throughput issue. There are no details about the machines you run. Try to replicate one of the published benchmarks of Scylla

Upvotes: 1

Related Questions