Incompatible clusterIDs 4


Error Scenario:

Incompatible clusterIDs

When we receive the error messages in data node logs similar to below then it belongs to this error scenario.

Root Cause:

This error message is received when the cluster ID of name node and cluster ID of data node are different. We can see the cluster ID of name node in <dfs.namenode.name.dir>/current/VERSION file and cluster ID of data node in <dfs.datanode.data.dir>/current/VERSION file. These files look like as shown below.

In Data Node:

datanode version   version file

In Name Node:

namenode version   version file2

Here in the above screen shots the cluster ID’s in name node and data node are matching so, in this scenario we will not get above error message. But if the cluster ID’s in data node is different than the cluster ID in name node, then it means that, without deleting files under <dfs.datanode.data.dir>/ location on data nodes, name node is formatted once again, which will create

  • new cluster ID with a fresh copy of VERSION file under  <dfs.namenode.name.dir>/current location on name node.
  • It will not create the VERSION file under <dfs.datanode.data.dir>/current location on data nodes if it already finds any existing VERSION file (which was created as part of previous name node format).

Precaution to avoid this error message:

  • Before formatting the name node, we need to delete the files under <dfs.datanode.data.dir>/ directories on all data nodes.

Resolution:

Once this error message is received, we can follow below two methods.

1. Quick Fix: Just update the Cluster ID in data node VERSION file with the cluster ID on name node VERSION file.

OR

2. Delete the <dfs.datanode.data.dir> / directory and <dfs.namenode.name.dir>/ directories and format the name node to start hdfs cluster with fresh copy of cluster ID.


About Siva

Senior Hadoop developer with 4 years of experience in designing and architecture solutions for the Big Data domain and has been involved with several complex engagements. Technical strengths include Hadoop, YARN, Mapreduce, Hive, Sqoop, Flume, Pig, HBase, Phoenix, Oozie, Falcon, Kafka, Storm, Spark, MySQL and Java.


Leave a comment

Your email address will not be published. Required fields are marked *

4 thoughts on “Incompatible clusterIDs


Review Comments
default image

I have attended Siva’s Spark and Scala training. He is good in presentation skills and explaining technical concepts easily to everyone in the group. He is having excellent real time experience and provided enough use cases to understand each concepts. Duration of the course and time management is awesome. Happy that I found a right person on time to learn Spark. Thanks Siva!!!

Dharmeswaran ETL / Hadoop Developer Spark Nov 2016 September 21, 2017

.