CODEWITHSUNDEEP
loadapache-pighcatalog
psmith
psmith

Reputation: 1813

Problems with type conversion - Pig -> HCatalog

I'm tryint to load table from HCatalog, do some exercises with data and store it into another table.

Source table: stage.iboa_event_definitions

inno_description  string    
inno_id           double    
inno_name     string    
inno_url      string    
inno_valid_from   string    
inno_valid_to     string

Destination table: 

create table dictionary (id int,src_id double,source_code string, src_code string,     src_description string, group_code string);

My script:

iboa_event_definitions = LOAD 'stage.iboa_event_definitions' USING org.apache.hcatalog.pig.HCatLoader();
iboa_event_definitions_filter = foreach iboa_event_definitions generate inno_id as src_id, 'IBOA' as source_code, inno_name as src_code, inno_description as src_description, '' as group_code;
iboa_event_definitions_filter_id = RANK iboa_event_definitions_filter;
final_table = foreach iboa_event_definitions_filter_id generate rank_iboa_event_definitions_filter as id:int, src_id, source_code as source_code, src_code, 
src_description, group_code;
store final_table into 'dictionary' using org.apache.hcatalog.pig.HCatStorer();

And I get error:

2013-11-26 13:18:06,140 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: RANK 2013-11-26 13:18:06,143 [main] INFO org.apache.pig.newplan.logical.rules.ColumnPruneVisitor - Columns pruned for iboa_event_definitions: $3, $4, $5 2013-11-26 13:18:06,212 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1115: Unsupported type: 10 in Pig's schema Details at logfile: /export/home/pig/pig_1385463241554.log

Why? let's check field types.

describe iboa_event_definitions_filter_id;
iboa_event_definitions_filter_id: {rank_iboa_event_definitions_filter: long,src_id: double,source_code: chararray,src_code: chararray,src_description: chararray,group_code: chararray}

describe final_table;
final_table: {id: int,src_id: double,source_code: chararray,src_code: chararray,src_description: chararray,group_code: chararray}

Maybe the error is caused by Long type? But that's why I'm tryint to convert it into int.

Can anyone help me with this issue?

Thanks

Pawel

Upvotes: 2

Views: 3216

Answers (3)

Anoop Velluva
Anoop Velluva

Reputation: 329

I also faced this issue when tried to store an int value [after casting to int] in a table where the corresponding column was a BIGINT.

HIVE

INT/INTEGER (4-byte signed integer) BIGINT (8-byte signed integer)

Corresponding in Pig

int Signed 32-bit integer

long Signed 64-bit integer

So i cast my value to long and it solved my issue.

Upvotes: 0

Dennis Jaheruddin
Dennis Jaheruddin

Reputation: 21563

The key part of your error message is:

Unsupported type: 10 in Pig's schema

This occurred for me when I had an INT and tried to store it in a table where, where the corresponding column was a BIGINT.

The solution for me was to change the table (so not the Pig script), after which the store went well.

Upvotes: 3

Helmut Zechmann
Helmut Zechmann

Reputation: 904

Type 10 stands for Integer (see http://pig.apache.org/docs/r0.11.1/api/constant-values.html#org.apache.pig.data.DataType.INTEGER). Your pig version does not support writing INT columns.

Use BIGINT as workaround.

Upvotes: 0

Related Questions