Copying Data from One machine to HDFS on remote machine


Hadoop Eco System Forums Hadoop Discussion Forum Copying Data from One machine to HDFS on remote machine

This topic contains 0 replies, has 1 voice, and was last updated by  Siva 2 years, 9 months ago.

Viewing 1 post (of 1 total)
  • Author
    Posts
  • #2999

    Siva
    Keymaster
    Option 1 – Efficient Method

    hadoop fs -cp <filename along with path(file://filepath)> hdfs://<hostname or IP of remote machine>:<hdfs port>/<path where file needs to be copied>

    Example-

    where NameNode1 is hostname of 192.16.4.43

    Option 2 – Inefficient Method

    Copying test.csv file from “machine1″(192.16.4.30) to hadoop residing machine 192.16.4.3

    run this command from 192.16.4.30

    This method is very slow because of displaying the contents on current machine and piping the results and connecting to remote machine through SSH and running the hadoop command remote machine.

Viewing 1 post (of 1 total)

The topic ‘Copying Data from One machine to HDFS on remote machine’ is closed to new replies.

Review Comments
default image

I have attended Siva’s Spark and Scala training. He is good in presentation skills and explaining technical concepts easily to everyone in the group. He is having excellent real time experience and provided enough use cases to understand each concepts. Duration of the course and time management is awesome. Happy that I found a right person on time to learn Spark. Thanks Siva!!!

Dharmeswaran ETL / Hadoop Developer Spark Nov 2016 September 21, 2017

.