CODEWITHSUNDEEP
hadoopsqoopdata-ingestion
AM_Hawk
AM_Hawk

Reputation: 681

Sqoop import multiple tables but not all

All the searches I've found show how to import one table or recommend the import-all-tables. What if I want 35 of 440 tables from my db. Can I just write one command and separate the tables by comma or do I have to put it in a script and copy and past the commands over and over and change the table name each time?

What I want to do:

echo "Sqoop Import"
--options-file ${path} 
--table tbl1,tbl2,tbl3\
--target-dir ${path}
--m 1\

What I fear I may have to do:

echo "Sqoop Import"
--options-file ${path} 
--table tbl1\
--target-dir ${path}
--m 1

wait 

echo "Sqoop Import"
--options-file ${path} 
--table tbl2\
--target-dir ${path}
--m 1

Upvotes: 2

Views: 9105

Answers (4)

Shyam Gupta
Shyam Gupta

Reputation: 513

sqoop import-all-tables \
--connect jdbc:mysql://localhost/sqoop \
--username root \
--password hadoop \
--warehouse-dir /Sqoop21/AllTables \
--exclude-tables table1,tables2

Upvotes: 0

satish silveri
satish silveri

Reputation: 358

You can also use the apply the same command for Hive import as:

sqoop import-all-tables \
--connect jdbc:mysql://your_ip_address:3306/database_name \
--driver com.mysql.jdbc.Driver \
--username root \
--warehouse-dir temp_dir_for_staging \
--hive-import \
--hive-overwrite \
--hive-database hive_db \
--exclude-tables list_of_tables_to_be_excluded \
-m 1

Remember in Hive you need staging area.

Upvotes: 0

mingchau
mingchau

Reputation: 470

Use --exclude-tables "table1,table2" option to ignore the table1 and table2.

Do NOT add white-space between the table names (aka. "table1, table2")

Upvotes: 1

hrobertv
hrobertv

Reputation: 158

Apparently a sqoop developer thought like you. :)

You can use import-all-tables.
And then add --exclude-tables Comma separated list of tables to exclude from import process.

https://sqoop.apache.org/docs/1.4.0-incubating/SqoopUserGuide.html#id1766722

Upvotes: 1

Related Questions