CODEWITHSUNDEEP
mysqlsqoop
Charlie Lin
Charlie Lin

Reputation: 1633

sqoop export specific columns from hdfs to mysql

Does SQOOP support that exporting specific columns from hdfs to mysql?

e.g. Now I have a hdfs file like this:

866138000211400 4   com.spreadst.validationtools    1   1310090804  0   0
866138000211400 4   com.spreadst.validdate  1   1310090804  0   0
866138000211400 4   com.spreadtrum.android.eng  1   1310090804  0   0
866138000211400 4   com.tencent.mm  261 1310090804  0   0
866138000211400 4   com.tencent.mobileqq    13  1310090804  0   0
866138000211400 4   com.thunderst.radio 1   1310090804  0   0
866138000211400 4   com.uucun51111531.android.cms   2013080901  1310090804  0   0
866138000211400 4   com.yeezonetech.firewall    1   1310090804  0   0
866138000211400 4   com.youku.phone 41  1310090804  0   0
866138000211400 4   org.openintents.cmfilemanager   20  1310090804  0   0

each line has 7 columns(imei, platform, packagename, softversion, gathertime, isHidden, isUninstalled) and each column is terminated by '\t'.

the mysql table structure is like: imei:varchar(100), platform:char(1), packagename:varchar(100), softversion:varchar(20).

How can I do to export the hdfs file to mysql using sqoop directly with the 4 specific columns?

Upvotes: 2

Views: 9497

Answers (2)

Rj1
Rj1

Reputation: 469

sqoop import --connect jdbc:mysql://localhost/DataBase_Name \
  --username root --table Table_Name --columns "Col1,Col2" \
  -m 1 --target-dir Hdfs_Dir_Name

Upvotes: 1

udachny
udachny

Reputation: 415

Yes, use --columns parameters to specify which columns to import.

Example from Sqoop Cookbook:

    sqoop export \
  --connect jdbc:mysql://mysql.example.com/sqoop \
  --username sqoop \
  --password sqoop \
  --table cities \
  --columns country,city

Upvotes: 3

Related Questions