CODEWITHSUNDEEP
cassandracassandra-2.2
Pratik Agrawal
Pratik Agrawal

Reputation: 11

Fail fast Cassandra NTR blocked tasks

We ran into an issue where a Cassandra node goes down in a cluster of 18 nodes and the overall cluster read/write latencies spike up due to which the Native Transport requests threads reach maximum capacity of 128 (default) and the NTR max queued capcity is reached (128 default) and the native-transport requests starts getting blocked.

I am not sure what blocked requests mean here? does cassandra starts failing the incoming requests until the queue is full? or the requests are blocked on the server side until they time out.

If it's the latter, is it possible to fail fast these requests from the Cassandra server side?

we are using Apache Cassandra version 2.2.8 with Datastax Cassandra java driver 3.0.0

Upvotes: 0

Views: 379

Answers (1)

Chris Lohfink
Chris Lohfink

Reputation: 16410

You can increase the number of concurrent requests to coordinate which is a common enough configuration with many tiny requests with -Dcassandra.max_queued_native_transport_requests=4096 with 2.2.8+. There is no feature to have it return an error instead of blocking but the back pressure will be noticed on the client and queued there until you get busy pool exceptions.

Upvotes: 1

Related Questions