CODEWITHSUNDEEP
sqlgoogle-bigquerydatabase-migrationnon-ascii-characters
dov.amir
dov.amir

Reputation: 11637

Cant Migrate to bigquery because bigquery column names allow only english characters

Bigquery column names (fields) can only contain English letters, numbers, and underscores.

I am using python and I want to create a script to migrate my data from Postgres to Bigquery and the Postgres tables have many non-english column names.

I will probably need to encode the column names to some format that Bigquery accepts, but I will need the ability to later decode it back to the original.

what is the best way to do this?

Upvotes: 1

Views: 702

Answers (1)

Rotem Grimberg
Rotem Grimberg

Reputation: 101

You can encode the column names to something like base64 and replace the +=/ characters to some kind of place holder.

If you don't care about fields length you can encode to base32 (its about 20% longer then base64 but don't use '+' or '/' and the '=' is used only for padding so you can discard it and it wont affect the string)

Except that you can make small conversion table for each non English character in your language to some combination in English chars, this will work only if you have small amount of non-english characters.

Upvotes: 1

Related Questions