Cassandra open port native_transport_port after a random time

Question

I'm running a cluster of 10 Cassandra 3.10 and I saw a very strange behavior: after restart, a node won't open immediately native_transport_port (9042).

After one node restart, the flow is :

• node finishes to read all commitlog,
• update all its data,
• it's visible for other nodes in the cluster,
• wait for random time (from 1 minute to hours) to open 9042 port

My logs are in DEBUG mode, and nothing is written about opening this port. What is happening and how can I debug this problem?

Output for several nodetool commands are:

• nodetool enablebinary does not return at all
• nodetool compactionstats 0 pending tasks
• nodetool netstats Mode: STARTING. Not sending any streams.
• nodetool info: Gossip active : true Thrift active : false Native Transport active: false

Thank you.

Answer 1

Are you saving your key/row cache? It tends to take a lot of time when that is the case. Also, what is your file max limit?