Fake Multi Node Cluster Creation on Single Machine with hadoop-2.6.0


Hadoop Eco System Forums Hadoop Discussion Forum Fake Multi Node Cluster Creation on Single Machine with hadoop-2.6.0

This topic contains 0 replies, has 1 voice, and was last updated by Profile photo of Siva Siva 2 years, 4 months ago.

Viewing 1 post (of 1 total)
  • Author
    Posts
  • #3021 Reply
    Profile photo of Siva
    Siva
    Keymaster

    Hi All,

    Below is the high level procedure for creation of 3 node hadoop cluster on a single Ubuntu machine.

    storage folder is for storing all the nodes’ files. This folder is just for convenience, as it’s easier to restart from scratch, just be deleting the storage folder content.

    Node1 will hosts:

    • HDFS namenode
    • HDFS datanode
    • Resource Manager
    • NodeManager

    So, the node1/etc/hadoop/core-site.xml file contains the location of the namenode. Following are core-site.xml, hdfs-site.xml, mapred-site.xml and yarn-site.xml files respectively.

    Node2 will hosts:

    • HDFS datanode
    • Yarn Nodemanager

    Below are its configuration properties that needs to be edited respectively for all the above *-site.xml files

    Node3 will hosts:

    • HDFS datanode
    • Yarn Nodemanager

    Similar to node2, node3 has its own below configuration properties that needs to be edited respectively for all the above *-site.xml files

    After making all the above changes, run the below commands from respective directories.

    Format the Namenode:

     

     

    We are now ready to start the namenode, datanode, resourcemanager and nodemanager on node1:

    We start a datanode and a nodemanager on node2:

    We start a datanode and a nodemanager on node3:

    We access to the HDFS web console (http://localhost:50070) to verify that the namenode is able to see the 3 live datanodes and resource manager at http://localhost:8088/cluster .

    we can also verify the services running on single machine.

Viewing 1 post (of 1 total)
Reply To: Fake Multi Node Cluster Creation on Single Machine with hadoop-2.6.0
Your information:




Review Comments
default image

I am a plsql developer. Intrested to move into bigdata.

Neetika Singh ITA Hadoop in Dec/2016 December 22, 2016

.