CODEWITHSUNDEEP
sqoopoozie
user1714351
user1714351

Reputation: 61

Sqoop is importing an integer as a string

I am trying to use sqoop in a lookup from Microsoft SQL Server.

Here is my sqoop script:

sqoop import \
--connect 'jdbc:sqlserver://LOOKUPDB-INT;database=Lookup_INT' \
--query "SELECT a.xlate_id, a.foreign_id as XlateKey, CAST(a.main_id as int) as main_id FROM Legacy.Xlate a WHERE a.foreign_source_id = '11' AND \$CONDITIONS" \ 
--hbase-create-table --hbase-table DataXlate \
--column-family DataXlate \
--hbase-row-key XlateKey  --split-by xlate_id \
--map-column-java main_id=Integer

The sqoop command seems to work as the table is created with the columns I would expect. However, when I read in a row using a java app, the bytes for main_id that are returned are for a string and not an integer. So if the main_id is 1 I would expect to get 0x1 but I am getting 0x31 which is the ascii code for '1'. Note that a.main_id is already an integer. The cast was done as an attempt to help sqoop recognize that fact.

Also, I am running this as part of an oozie workflow.

Upvotes: 4

Views: 2456

Answers (1)

Jarek Jarcec Cecho
Jarek Jarcec Cecho

Reputation: 1726

This is intentional behaviour, Sqoop is serialising all columns in their string form. Please take a look into Sqoop User Guide for more background. You can file a feature request on Sqoop JIRA if you're interested in improving this.

Upvotes: 1

Related Questions