Eclipse Mapreduce Example


Running Sample Mapreduce – Word Count Program in Eclipse

This post is an extension of previous post about configuring eclipse for Hadoop. Once the configuration is done successfully, we can run the sample mapreduce programs in Eclipse IDE. In this Eclipse Mapreduce Example post, We will discuss the development of sample Word Count mapredue program from scratch and execute the jar file on hadoop cluster and verify the results.

1.  Start eclipse and choose Mapreduce perspective.

Go to Window –> Open Perspective –> Other

Mapreduce Perspective

After selecting Map/Reduce perspective, we can see DFS Locations under project explorer side bar and Map/Reduce Locations at the bottom as shown below screen.

Mapreduce Locations

These locations can be configured by right clicking on the empty box and selecting “New Hadoop Location” and then providing host name and port address for test development. Since, in production, the developed jar files are exported to a particular hadoop machine on cluster, we will follow the same approach here.

2.  Create a new Map/Reduce Project. Go to File –> New –> Other –> Map/Reduce Project.

Mapreduce wizard

Provide the project name, for example WordCount in our case.

WordCount MR

Click on Configure Hadoop install directory and provide the hadoop’s home directory.

Hadoop Home dir

3.  Now, lets create Mapper, Reducer and Driver classes under WordCount/src with the help of Eclipse configurations.

Wordcount-src

a)  Right click on src –> New –> Other –> Map/Reduce folder and Select Mapper for creating our mapper class WordCountMapper

Mapper wizard

WordCountMapper

Default Mapper

Copy the below source code into Mapper class and save the file and it will automatically compiles and generates class files.

[Read Next Page]


Profile photo of Siva

About Siva

Senior Hadoop developer with 4 years of experience in designing and architecture solutions for the Big Data domain and has been involved with several complex engagements. Technical strengths include Hadoop, YARN, Mapreduce, Hive, Sqoop, Flume, Pig, HBase, Phoenix, Oozie, Falcon, Kafka, Storm, Spark, MySQL and Java.

Leave a comment

Your email address will not be published. Required fields are marked *


Review Comments
default gravatar

I am a plsql developer. Intrested to move into bigdata.

Neetika Singh ITA

.