CODEWITHSUNDEEP
pythoncsvencoding
Yebach
Yebach

Reputation: 1691

python csv encoding writerow

Using web2py on windows 2008 server I have a following problem

I am creating csv document from json and when writting list to file i get the following error. It crashes on csv writerow

<type 'exceptions.UnicodeEncodeError'> 'ascii' codec can't encode character u'\\u010c'

It works ok on my computer. Windows 7 but on server I have an encoding problems

any suggestions? thank you

My code for creating file is the following

dataDict = json.loads(data.replace("'", "\""))
path = path
scriptName = os.path.join(path, id + 'script.txt')
file = open(scriptName, 'wb') 
output = csv.writer(file, delimiter='\t')

##Month hours
file.write("begin month_hours \r\n")
file.write("delavec    mesec    month_hours_min    month_hours_max\r\n")
for rec in dataDict["mandatory"]:
    output.writerow(rec)
file.write("\r\nend month_hours \r\n")

Upvotes: 0

Views: 3539

Answers (2)

kdazzle
kdazzle

Reputation: 4300

It's been a while since this question was posted, so I'm guessing that the Python API has changed, but nowadays, you can actually open the file with an explicit encoding, which fixed a similar issue that I was having:

Changing this line in your code should do the trick. No need to make your code O(n2)ish with that inner loop.

open(scriptName, 'w', encoding='utf-8')

Or for the full example from the python docs:

import csv
with open('some.csv', newline='', encoding='utf-8') as f:
    reader = csv.reader(f)
    for row in reader:
        print(row)

Upvotes: 1

Martijn Pieters
Martijn Pieters

Reputation: 1122492

JSON strings are always Unicode values and in Python 2 need to be encoded when writing to a CSV file. If you don't do this explicitly, Python will use the ASCII codec. That's fine if all your data contains text in the ASCII range but fails when you encounter data beyond that.

Pick a different encoding and encode explicitly; UTF-8 is a good encoding to pick:

for rec in dataDict["mandatory"]:
    output.writerow([unicode(c).encode('utf8') for c in rec])

I first convert all values to unicode(), in case you have data in there that's not already a unicode() value; numbers or booleans or None for example. The result is then explicitly encoded to UTF-8.

Upvotes: 6

Related Questions