CODEWITHSUNDEEP
hadoophdfstalendhawq
vkumar
vkumar

Reputation: 31

ERROR: value too long for type character(50)

I have created external table in HDFS and internal table in HAWQ. I am fetching data from SQL Server, using talend for etl process Process flow is like SQLSERVER -> EXTERNAL TABLE(PXF HAWQ) -> INTERNAL TABLE(HAWQ)

On running the job I am getting below error

ERROR: value too long for type character(50) (seg0 slice1 phds01.aa.com:40000 pid=297176) Detail: External table podetails_stg0, line 17 of pxf://PHD-HA/test/PoDetails_stg0.csv?profile=HdfsTextSimple, column StockDes

How I should resolve this error ?

Upvotes: 0

Views: 924

Answers (2)

vkumar
vkumar

Reputation: 31

This is mainly data related error. Data contain many special characters like new line, comma. This causing data to move into another column while copying data into CSV in HDFS from mssqlserver. The best way to eliminate this issue by filter out special character's at source level while fetching data.

Upvotes: 0

Théo Capdet
Théo Capdet

Reputation: 1062

The string you try to put in the HDFS have more than 50 caracter. You have two choice :

  • Trucate the colum at the 50th caracter with a StringHandling.LEFT(yourcolumn,50)

  • Alter you table and make your column bigger like character(100) or even more.

Upvotes: 1

Related Questions