CODEWITHSUNDEEP
javacassandracqlsh
Oak
Oak

Reputation: 69

I can't open Cassandra because of insufficient disk space to write

I set up a new Cassandra and use ./cassandra to open it, but it fails. In the system.log, it shows the following error:

ERROR 13:30:31 Exception encountered during startup java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Insufficient disk space to write 542 bytes
    at org.apache.cassandra.db.ColumnFamilyStore.runWithCompactionsDisabled(ColumnFamilyStore.java:2888) ~[apache-cassandra-2.2.10.jar:2.2.10]
    at org.apache.cassandra.db.ColumnFamilyStore.truncateBlocking(ColumnFamilyStore.java:2849) ~[apache-cassandra-2.2.10.jar:2.2.10]
    at org.apache.cassandra.db.SystemKeyspace.discardCompactionsInProgress(SystemKeyspace.java:404) ~[apache-cassandra-2.2.10.jar:2.2.10]
    at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:230) [apache-cassandra-2.2.10.jar:2.2.10]
    at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:533) [apache-cassandra-2.2.10.jar:2.2.10]
    at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:642) [apache-cassandra-2.2.10.jar:2.2.10] Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Insufficient disk space to write 542 bytes
    at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:394) ~[apache-cassandra-2.2.10.jar:2.2.10]
    at org.apache.cassandra.db.SystemKeyspace.forceBlockingFlush(SystemKeyspace.java:593) ~[apache-cassandra-2.2.10.jar:2.2.10]
    at org.apache.cassandra.db.SystemKeyspace.saveTruncationRecord(SystemKeyspace.java:432) ~[apache-cassandra-2.2.10.jar:2.2.10]
    at org.apache.cassandra.db.ColumnFamilyStore$13.run(ColumnFamilyStore.java:2843) ~[apache-cassandra-2.2.10.jar:2.2.10]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_141]
    at org.apache.cassandra.db.ColumnFamilyStore.runWithCompactionsDisabled(ColumnFamilyStore.java:2884) ~[apache-cassandra-2.2.10.jar:2.2.10]           ... 5 common frames omitted Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Insufficient disk space to write 542 bytes
    at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299) ~[guava-16.0.jar:na]
    at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286) ~[guava-16.0.jar:na]
    at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) ~[guava-16.0.jar:na]
    at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:390) ~[apache-cassandra-2.2.10.jar:2.2.10]           ... 10 common frames omitted Caused by: java.lang.RuntimeException: Insufficient disk space to write 542 bytes
    at org.apache.cassandra.db.Directories.getWriteableLocation(Directories.java:349) ~[apache-cassandra-2.2.10.jar:2.2.10]
    at org.apache.cassandra.db.Memtable.flush(Memtable.java:324) ~[apache-cassandra-2.2.10.jar:2.2.10]
    at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1187) ~[apache-cassandra-2.2.10.jar:2.2.10]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_141]

the output of command "df -h":

root@srv-machine-learning01:/var/lib/cassandra# df -h

Filesystem                                   Size  Used Avail Use% Mounted on

/dev/xvda1                                        61G   60G     0 100% /

udev                                              10M     0   10M   0% /dev

tmpfs                                            2.4G  8.3M  2.4G   1% /run

tmpfs                                            5.9G     0  5.9G   0% 

/dev/shm

tmpfs                                            5.0M     0  5.0M   0% 

/run/lock

tmpfs                                            5.9G     0  5.9G   0% 

/sys/fs/cgroup

bak02.sovanta.com:/volume1/bak_machine_learning   70T   19T   52T  27%

Upvotes: 1

Views: 1613

Answers (2)

shirel
shirel

Reputation: 91

go to mchine and delete unused schema of cassandra manually.

Upvotes: 0

Simon Fontana Oscarsson
Simon Fontana Oscarsson

Reputation: 2124

It looks like your root file system is full.

/dev/xvda1 61G 60G 0 100% /

Since Cassandra stores data by default in /var/lib/cassandra and you haven't set up a different file system to use for this path it will use your root file system. I can think of several solutions for this issue.

  • Free up space on root partition
  • Increase disk space for your root file system
  • Add a new disk to your system and create a new filesystem which mounts to /var/lib/cassandra

Upvotes: 3

Related Questions