CODEWITHSUNDEEP
pythonjsoncsvescaping
godzilla
godzilla

Reputation: 3125

python csv writer not handling escaping '|' correctly

I am attempting to write some JSON to a file, the following is the code used to open the file, please keep in mind i need the quoting and "quotechar" parameters to ensure the JSON is valid when written to a file.

file = csv.writer(open("myfile" + ".txt",'ab'),delimiter = '|',quoting=csv.QUOTE_NONE, quotechar='')

For some reason if the JSON contains '|' an exception is thrown with the message "need to escape, but no escapechar set". Below is a a sample, is there a way of avoiding issues like this by passing a parameter when opening a file or in any other way?

"clickthrulink.Description": "|this is a test"

Upvotes: 1

Views: 4106

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1123410

Set an escape character to escape the | character inside JSON values. You need to re-use that same escape character when reading the CSV file again:

file = csv.writer(open("myfile" + ".txt",'ab'),
                  delimiter = '|', quoting=csv.QUOTE_NONE, quotechar='',
                  escapechar='\\')

Demo:

>>> from cStringIO import StringIO
>>> import csv
>>> f = StringIO()
>>> writer = csv.writer(f, delimiter = '|', quoting=csv.QUOTE_NONE, quotechar='', escapechar='\\')
>>> writer.writerow(['{"clickthrulink.Description": "|this is a test"}'])
>>> f.getvalue()
'{"clickthrulink.Description": "\\|this is a test"}\r\n'
>>> print f.getvalue()
{"clickthrulink.Description": "\|this is a test"}

Upvotes: 1

Related Questions