CODEWITHSUNDEEP
hadoophiveteradatabigdata
S. Kundu
S. Kundu

Reputation: 31

Type Casting error while importing Decimal values from Teradata to Hive(parquet) using TDCH

I am trying to import decimal data from Teradata to Hive Parquet format using TDCH, but it's giving type casting error. Not sure why it is trying to cast decimal to string. I am using Decimal data type at both side i.e. Hive and Teradata both. Same thing is happening with Timestamp field as well. This is happening while the table is in Parquet format in Hive, similar thing is working fine for RCfile format. Any help please?

Upvotes: 0

Views: 981

Answers (1)

danielsepulvedab
danielsepulvedab

Reputation: 674

There is no Decimal or Timestamp data type in Parquet according to Parquet Documentation, that must be why is mapping to a String.

The types supported by the file format are intended to be as minimal as possible, with a focus on how the types effect on disk storage. For example, 16-bit ints are not explicitly supported in the storage format since they are covered by 32-bit ints with an efficient encoding. This reduces the complexity of implementing readers and writers for the format. The types are: - BOOLEAN: 1 bit boolean - INT32: 32 bit signed ints - INT64: 64 bit signed ints - INT96: 96 bit signed ints - FLOAT: IEEE 32-bit floating point values - DOUBLE: IEEE 64-bit floating point values - BYTE_ARRAY: arbitrarily long byte arrays.

The data type in the Hive column definition and the data type representation in HDFS aren't the same thing. If you need those exact data types for your data, you should try another format.

Upvotes: 0

Related Questions