0

Cassandra service on one of my nodes went down and we couldnt restart it because of some corruption in one of the tables. So we tried rebuilding it by deleting all the data files and then starting the service, once it shows up in the ring we ran nodetool repair multiple times but it got hung throwing the same error

Caused by: org.apache.cassandra.io.compress.CorruptBlockException: (/var/lib/cassandra/data/profile/AttributeKey/profile-AttributeKey-ib-1848-Data.db): corruption detected, chunk at 1177104 of length 11576.

This occurs after 6gb of data is recovered. Also my replication factor is 3 so the same data is fine on the other 2 nodes.

I am a little new to Cassandra and am not sure what I am missing, has anybody seen this issue with repair? I have also tried scrubbing but it failed because of the corruption.

Please help.

I Bajwa PHD
  • 1,466
  • 1
  • 18
  • 41
neeraj
  • 1
  • 1

2 Answers2

3

rm /var/lib/cassandra/data/profile/AttributeKey/profile-AttributeKey-ib-1848-* and restart.

Scrub should not fail, please open a ticket to fix that at https://issues.apache.org/jira/browse/CASSANDRA.

jbellis
  • 19,274
  • 2
  • 34
  • 46
0

first use the nodetool scrub if it does not fix then shut down the node and run sstablescrub [yourkeyspace] [table] you will be able to remove the corrupted tables which were not done at nodetool scrub utility and run a repair you will be able to figure out the issue.

satish soma
  • 1
  • 2