CODEWITHSUNDEEP
unicodeteradatateradata-sql-assistant
Burhan Khalid Butt
Burhan Khalid Butt

Reputation: 275

Unicode to Latin in Teradata Conversion

I have been trying to convert Unicode strings to Latin in Teradata version 16.20.32.23. I have seen many online forums but I was not able to formulate a solution. Following are some of the strings that I was unable to convert:

hyödyt
löydät

I have tried following solution but function translate_chk does not seems to work.

SELECT CASE WHEN Translate_Chk ( 'hyödyt' using UNICODE_TO_LATIN) <> 0 
THEN
''
WHEN Translate_Chk ( 'hyödyt' using UNICODE_TO_LATIN ) = 0 
THEN
Translate ( 'hyödyt' using UNICODE_TO_LATIN WITH ERROR)
END AS transalated

The error I receive is SELECT FAILED. 6706: The string contains untranslatable character.

I think I have reached a dead end, could anyone help me here?

Upvotes: 0

Views: 1709

Answers (2)

Burhan Khalid Butt
Burhan Khalid Butt

Reputation: 275

Following is the python code I used, it might help someone. In order to use below code you need to follow below instructions:

Download chilkat package as per your python release: https://www.chilkatsoft.com/python.asp#winDownloads

Follow installation guidelines in below URL: https://www.chilkatsoft.com/installWinPython.asp

Open IDLE shell and run the following code

import sys
import chilkat

charset = chilkat.CkCharset()

charset.put_FromCharset("utf-8")
charset.put_ToCharset("ANSI")


charset.put_ToCharset("Windows-1252")

success = charset.ConvertFile("source_file_name.csv","target_file_name.csv")
if (success != True):
    print(charset.lastErrorText())
    sys.exit()

print("Success.")

Upvotes: 0

Mark Tolonen
Mark Tolonen

Reputation: 177941

I'm not familiar with Teradata, but the strings you have are double-mis-decoded as Windows-1252, which is a variation of ISO-8859-1 a.k.a latin1. Example to fix in Python:

>>> s='hyödyt'
>>> s.encode('cp1252').decode('utf8').encode('cp1252').decode('utf8')
'hyödyt'
>>> s='löydät'
>>> s.encode('cp1252').decode('utf8').encode('cp1252').decode('utf8')
'löydät'

So not a Teradata solution, but should help you figure it out.

Upvotes: 1

Related Questions