4

We added a new node to cluster and applied nodetool repair several times in new node. we sure exist extra data in new node so run nodetool cleanup but after second cleanup completed and cassandra log following warns and cleanup not remove any data. why?

WARN  09:27:12 Small commitlog volume detected at /var/lib/cassandra/commitlog; setting commitlog_tota│Mode: NORMAL  
l_space_in_mb to 3854. You can override this in cassandra.yaml                                       │Not sending any streams.

WARN  09:27:12 Small cdc volume detected at /var/lib/cassandra/cdc_raw; setting cdc_total_space_in_mb │Read Repair Statistics:                                                                              
to 1927. You can override this in cassandra.yaml
Anower Perves
  • 724
  • 8
  • 15
Hossein Yousefi
  • 77
  • 1
  • 8

1 Answers1

0

I was seeing the same error when setting up Cassandra and trying to load SSTables from previously taken snapshots. It turned that during my previous experiments on JVM (specifically setting JAVA_OPTS="-Xms29G -Xmx29G" on a machine with 32G memory) massive java_pid*.hprof files were created in / which were preventing Cassandra from working properly; I was able to fix my problem by removing those files.

Unfortunately, I can't directly answer your question without more details, but I'd recommend the followings:

  1. Run cd / and sudo du -h | sort -h and inspect files and directories to see if any suspicious large log or memory allocation file has been dumped there which can be removed.
  2. If you have not set up your Cassandra directory explicitly at the installation time, Cassandra will try dumping runtime files in /var, which could be full or under-allocated. Inspect files for silent massive logs (e.g. Nifi alerts) and remove unnecessary ones. If this didn't work, you can re-install Cassandra at another location or (not recommended) increase your /var directory size.
Kiarahmani
  • 47
  • 1
  • 8