0

I'm new to ES. We've recently setup a 3 node elasticsearch cluster for our Prod App. Just want to understand what would happen if ElasticSearch node or index or shard gets corrupted.

Thanks!

cloudtechnician
  • 545
  • 6
  • 15

2 Answers2

2

What would happen actually depends on how you have set up your ES cluster.

With respect to DATA

  • If you have a singular cluster, a corruption would render your ES setup useless.You would,pretty much,need to setup everything from scratch.
  • If you have multiple nodes in your cluster,there can be following scenarios-
    • If you configure a single node as data node and if that goes down,you would have the cluster running but queries would not return any result. You will then need to re-configure a node to behave as data node and restart the cluster.
    • If you have multiple nodes designated as data node,then a corruption/failure of a node will only affect that node.The rest of the nodes and the ES will in essence perform as usual. The only effect is that the data stored in the corrupted node will obviously be not available. The shards in the corrupted node will become unassigned shards and have to be reassigned to some other data node.
    • If you have replicas enabled,then there will be no effects in term of data loss. It would simply require the unassigned shards to be re-assigned to some new data node(if and when it is added).

Its best to have a multi-node cluster with at least 2 data nodes and replicas enabled to mitigate shards/data nodes corruption.

This Stackoverflow post explains shards and replicas in an excellent way.

Edit 1: This is in response to your comment.

Default settings dictate that each node is master eligible and also stores data and hence,each of your nodes can become Master and will also store data.

Lets consider nodes as A,B & C. Initially, one of them will be designated as master node,e.g. Node A.

Now if Node A goes down,one of the remaining nodes (B & C) will become the master nodes. Queries will now only return results from data stored in Node B & C.

Check out this page for more insights into how cluster works

Community
  • 1
  • 1
Mayur Buragohain
  • 1,199
  • 17
  • 33
  • Thanks Mayur, we've a 3 node ES cluster and we've not configured any master, data node etc. All are set to their default settings. – cloudtechnician Apr 04 '16 at 18:37
0

One way is to you need to take incremental snapshots of your indices and need to restore from that snapshot.

KARTHEEK GUMMALURI
  • 187
  • 1
  • 10