Error Repository


Oozie Share Lib does not exist error

In this post we will discuss about one of common/frequent error message faced by many oozie beginners due to incorrect/incomplete oozie setup. Oozie Share Lib does not exist error is received if oozie sharelib is not installed properly. Error Scenario: java.io.FileNotFoundException: File /user/user/share/lib does not exist This error message we will get when we submit any oozie job with below property set to true in job.properties file

and if you […]


Exhausted available authentication methods

In this post we will discuss about common problem in installing cloudera manager 5.3.2 on Ubuntu 14.04 machine and solution for the root cause. Problem: When installing Cloudera Manager 5.3.2 on Ubuntu 14.04.2 Machine getting below error messages after giving root privileges on ssh configuration page. Exhausted available authentication methods or Installation failed on all hosts. Installation failed. Failed to authenticate In this case, ssh to root might be a […]


Cannot create an instance of InputFormat

Error Scenario: java.io.IOException: Cannot create an instance of InputFormat class We will get this error message when we try to execute simple hadoop fs commands or running any hive queries. Below is the complete error message.

Root Cause: This error message will be received when there are any spaces or spelling mistakes in any of the site.xml configuration file. Suppose in below core-site.xml file value for io.compression.codecs has spaces which […]


NoClassDefFoundError or ClassNotFoundException

In this post we will discuss about most frequent error messages NoClassDefFoundError or ClassNotFoundException in hadoop mapreduce jobs execution and possible solutions for them. Error Scenario: java.lang.ClassNotFoundException       or java.lang.NoClassDefFoundError Error starting MRAppMaster   &  Container exited with a non-zero exit code 1 When we encounter a situation of NoClassDefFoundError or ClassNotFoundException even though we have required jar files added to our build path in eclipse, then we have […]


Hadoop Integration – Avro Errors 1

In this post we will discuss about some of the errors or exceptions that can occur when there is mismatch in the integration of Avro and Hadoop distributions. When we do not use the correct version of avro release then we will run into so many errors or exceptions. In this post, we will consider the version compatibility for Hadoop-2.3.0 release. As it is Hadoop2  we need to use the […]


Expected timestamp in the Flume event headers, but it was null 1

Error Scenario: Expected timestamp in the Flume event headers, but it was null – NullPointerException This error message is received in ~/logs/flume.log file when starting a flume agent with HDFS sink with format escape sequences.(%Y, %M, %D, %H, %M, %S).

If a sink expects a header but does not find it, events will become stuck in the channel and Flume will log NullPointer and EventDelivery exceptions. Root Cause: We receive […]


channel has been removed due to an error

Error Scenario: channel has been removed due to an error during configuration. This error message is received in ~/logs/flume.log file when starting a flume agent with missing or wrong configuration with JDBC channel setup.

Root Cause: DerbySchemaHandler.schemaExists error message is received when a flume agent’s sink is not able to receive the events from this JDBC channel and for some reason agent process stopped and when we try to […]


Mapreduce job stuck at map 0% reduce 0%

Error Scenario: Mapreduce job stuck at map 0% reduce 0%. : Could not resolve hostname

Mapreduce jobs are not running. And the below error message is coming when starting HDFS daemons.

Resolution: These above error messages are received when there is some issue with ssh communication. So, in these situations, we need to remove openssh-client and openssh-server and re-install them again.

Once the ssh is re-installed, we will […]


org.apache.flume.EventDeliveryException: Failed to open file

Error Scenario: org.apache.flume.EventDeliveryException: Failed to open file  We will receive this error message when a flume agent is started and it is trying to start FILE_ROLL sink with a given target sink directory. Below are the error messages sequence from ~/logs/flume.log file.

Root Cause: When a flume agent is started with FILE_ROLL sink type and without creating <sink.directory> folder prior to starting agent or  Flume user doesn’t have enough […]


UnsatisfiedLinkError

Error Scenario: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z We will receive this error message if we try to read snappy files without having snappy compression/decompression codec installed properly. 

Or

Resolution: We should install snappy compression/decompression codec to resolve this error message and its corresponding .so files should be available (copied from /usr/local/bin) in $HADOOP_HOME/lib/native directory location. For Snappy Installation instructions please read through this post.


Unable to open a test connection to the given database 1

Error Scenario 1: In this post, we will discuss about a few varieties of error message Unable to open a test connection to the given database and their resolutions. Unable to open a test connection to the given database / Communications link failure Below is the error message for this communications link failure:

Root Cause: We will get the above error message, if we specify incorrect hostname: port values in property javax.jdo.option.ConnectionURL in […]


Access denied for user ‘APP’@’localhost’ (using password: YES)

Error Scenario: Unable to open a test connection to the given database  or Access denied for user ‘APP’@’localhost’ (using password: YES) This error message (Access denied for user ‘APP’@’localhost’) will be received while trying to connect hive service through MySQL local metastore.

Root Cause: When we don’t provide username (javax.jdo.option.ConnectionUserName) and password (javax.jdo.option.ConnectionPassword) configurations in hive-site.xml for accessing MySQL local metastore database we will receive this error message. By default these two […]


datastore driver was not found in the CLASSPATH 1

Error Scenario: The specified datastore driver (“com.mysql.jdbc.Driver”) was not found in the CLASSPATH We usually get this error message while configuring Hive Local Metastore through MySQL database.

Root Cause: If the JDBC driver JAR file for MySQL (Connector/J) is not on Hive’s classpath, ( in Hive’s lib directory – $HIVE_HOME/lib), we will get this error message. Resolution: We have to download this MySQL Connector/J jar file from MySQL downloads pages . For […]


Datanode denied communication with namenode

Error Scenario: Datanode denied communication with namenode: We usually get this error message while adding a data node to an existing HDFS cluster or at the time of cluster setup. Below is the snapshot of this error message from log file in data node.

Root Cause: This error message is receive if the hadoop user (for example hduser) on namenode machine doesn’t have full (write) permissions to the directory location […]


Review Comments
default gravatar

I am a plsql developer. Intrested to move into bigdata.

Neetika Singh ITA

.