CODEWITHSUNDEEP
pythongoogle-app-enginepython-2.7
Omair Shamshir
Omair Shamshir

Reputation: 2136

Google App engine python writing csv in arabic

I am trying to write a CSV in Arabic script. I have encoded the string to utf-8 and wrote it in the csv.

The problem is if I open the file in csv it shows strange characters like آلز سندويتش كاÙيه however if I open the file in notepad++ it shows the expected arabic text.

i checked notepad ++ and converted the encoding to utf-8 instead of utf-8 without BOM , now its working fine in csv reader(excel) too. so what should i do to set encoding to "utf-8 with BOM" in app engine

i am using unicodecsv.writer to write the csv 

writer = unicodecsv.writer(self.response.out)
row = []
row.append(transaction.name.encode('utf8'))
writer.writerow(row)

the data to be written is taken from the datastore

Upvotes: 0

Views: 344

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1124238

To write a CSV with UTF-8 BOM, simply write the BOM first; you can use codecs.BOM_UTF8 for that:

import codecs

self.response.out.write(codecs.BOM_UTF8)
writer = csv.writer(self.response.out)
row = []
row.append(transaction.name.encode('utf8'))
writer.writerow(row)

Excel 2007 and newer pick up on the BOM and correctly open such a CSV file with UTF-8. Silly Microsoft!

Upvotes: 1

Related Questions