CODEWITHSUNDEEP
sql-serverhadoopsqoop
ganeshran
ganeshran

Reputation: 3512

Sqoop : Truncate SQL Server table before exporting data from Hadoop

We are using Sqoop to export data from the hive to SQL Server. The new data is always appended to the existing data in SQL Server.

Is it possible to truncate the SQL Server table via Sqoop before starting the export?

Upvotes: 3

Views: 9688

Answers (3)

Tagar
Tagar

Reputation: 14891

There is a feature request to implement sqoop --truncate option https://issues.apache.org/jira/browse/SQOOP-1313

Please vote up on that JIRA page if you're interested.

Upvotes: 0

Jarek Jarcec Cecho
Jarek Jarcec Cecho

Reputation: 1726

Sqoop is not a general query tool, the "eval" functionality is provided only for evaluation purpose and should not be used in production mode. You can always put together simple java code that will do that in a way that is best for your use case.

Upvotes: 3

Daniel Koverman
Daniel Koverman

Reputation: 2573

You can use sqoop eval to execute arbitrary SQL on the database. This will allow you to truncate the table without "leaving" Sqoop. For example:

sqoop eval --connect 'jdbc:sqlserver://1.1.1.1;database=SomeDatabase;username=someUser;password=somePassword' --query "TRUNCATE TABLE some_table"

sqoop export --connect 'jdbc:sqlserver://1.1.1.1;database=SomeDatabase;username=someUser;password=somePassword' --export-dir /path/to/someTable/on/HDFS --table some_table --fields-terminated-by \001

--fields-terminated-by \001 assumes that the Hive table is using the default delimiters.

Upvotes: 8

Related Questions