Sqoop Export Commands 1


Below are some Sqoop Export Commands and Other Miscellaneous commands

Sqoop-export

It is nothing but exporting data from HDFS to database. To use ‘export‘ command, a table in database should already exist. then only export functionality in sqoop will works.

export command will works in two ways
1. insert
2. update

1. insert: insert mode will insert the new records from HDFS to RDBMS table.

command:

Note: if a record already present in the database table with same primary key, then it will raise MySQLIntegrityConstraintViolationException exception.

2. update: update mode will update the records in the RDBMS from HDFS data.

update mode only update already existing records, it will not insert new records in the RDBMS.

command:

Sqoop-JOB

Job is nothing but to save a sqoop command and excute n times when we require it. The job command allows you to create and work with saved jobs. Saved jobs remember the parameters used to specify a job, so they can be re-executed by invoking the job by its handle.

The following are Sqoop Job commands,

Create job:

Note:
There should be space between ‘–‘ and ‘import’.

list jobs: it will show all the jobs.

inspect job: it will show details about the job.

delete job: it will delete existing job.

execute job: it will execute the job.

Sqoop-Merge

Merge command in sqoop is to merge two existing tables in HDFS to single new table in HDFS.

Let us discuss merge functionality with the example:

The below are tables in RDBMS,

Table 1: emp_1

Table 2: emp_2

now,

Importing emp_1 into HDFS

Importing emp_2 into HDFS

After importing emp_1 and emp_2 into HDFS, sqoop will generate a jar into our local file system. with help of that jar we will able to merge the two tables in HDFS

1. create a database and table as shown below

2. inserted 10000 + records as shown below

Note: I have used mysql import to load 10000+ records, as below

3. Run below sqoop command to import from MYSQL to HDFS location

It created 4 Map jobs and stored in part files in HDFS location.

4. Updated some records randomly

5. Used below command to pic updated records and merged into single file.

Sqoop-import-hive

sqoop provides direct import into hive.There are two ways to import into hive

1. Create-hive-table: it will be used when data already imported into hdfs but not into hive

By above command, sqoop will create an empty table with DB column properties.To load data we need to use hive command as shown below:

2.hive-import: It is a direct import from RDBMS to Hive


Leave a comment

Your email address will not be published. Required fields are marked *

One thought on “Sqoop Export Commands


Review Comments
default gravatar

I am a plsql developer. Intrested to move into bigdata.

Neetika Singh ITA

.