MRUnit Example for WordCount Algorithm 1


In this post we will discuss about basic MRUnit example for Wordcount algorithm.

Below are the tools used in this example

Eclipse 3.8, mrunit-1.0.0-hadoop2.jar

Procedure:

1. Download mrunit jar from this link and add this to the java project build path (File –> properties –> java build path –> add external jars) in eclipse.

2. As we are testing wordcount algorithm…Below is the code for the same.

3. To test the above algorithm we need to write new MRUnit java code. Below is the code for the same.

4. Inorder to test only mapper code we need to run testMapper() method with the required input and ouput as shown in the above code. To test reducer code we have to run testReducer() method and to test both we need to run testMapReduce() method as shown above.

How to test MRunit:

Right click on the java code in eclipse and click on run as JUnit Test and verify the console for any errors.

Test Strategy for testing mapper using testMapper() method:

1. In the above example inorder to test the mapper code we have given input as “hadoop is bigdata” “hadoop is emerging” and output as (hadoop,1) (is,1) (bigdata,1) (hadoop,1) (is,1) (emerging,2)…when we run the above test we will get the following error in the console.

Just change the output in the above code from  (emerging,2) to (emerging,1) and then test the code. Everything will work fine without any errors.

Test Strategy for testing Reducer using testReducer() method:

1. In the above example inorder to test the mapper code we have given input as (bigdata, <1>) (emerging, <1>)  (hadoop, <1,1>) (is, <1,1>)  and output as (bigdata,1) (emerging,1) (hadoop,2) (is,1) …when we run the above test we will get the following error in the console.

Just change the output in the above code from  (is,1) to (is,2) and then test the code. Everything will work fine without any errors.

Test Strategy for test both Mapper and Reducer using testMapReduce() method:

1. In the above example inorder to test the mapper code we have given input as “hadoop is bigdata” “hadoop is emerging”  and output as (bigdata,1) (emerging,1) (hadoop,2) (is,1) …when we run the above test we will get the following error in the console.

Just change the output in the above code from  (is,1) to (is,2) and then test the code. Everything will work fine without any errors.


Profile photo of Siva

About Siva

Senior Hadoop developer with 4 years of experience in designing and architecture solutions for the Big Data domain and has been involved with several complex engagements. Technical strengths include Hadoop, YARN, Mapreduce, Hive, Sqoop, Flume, Pig, HBase, Phoenix, Oozie, Falcon, Kafka, Storm, Spark, MySQL and Java.


Leave a comment

Your email address will not be published. Required fields are marked *

One thought on “MRUnit Example for WordCount Algorithm


Review Comments
default gravatar

I am a plsql developer. Intrested to move into bigdata.

Neetika Singh ITA

.