CODEWITHSUNDEEP
pythoneclipsecsvpython-3.xpydev
LukasT
LukasT

Reputation: 450

How can I stop Python's csv.DictWriter.writerows from adding empty lines between rows in Windows when using stdout?

When I run this code in Eclipse using PyDev, I see output with empty lines in between the rows. How can I get rid of them? It happens using python3 only.

import csv
import sys

fieldnames = ['first_name', 'last_name']
writer = csv.DictWriter(sys.stdout, fieldnames=fieldnames)

writer.writeheader()
writer.writerow({'first_name': 'Baked', 'last_name': 'Beans'})
writer.writerow({'first_name': 'Lovely', 'last_name': 'Spam'})
writer.writerow({'first_name': 'Wonderful', 'last_name': 'Spam'})

Output in Eclipse

first_name,last_name

Baked,Beans

Lovely,Spam

Wonderful,Spam

Output is correct in cygwin-bash, on Linux and incorrect when using redirection to file in cmd.exe + Windows python (Console window is apparently correct)

Output from cmd.exe redirected to file viewed in Notepad++

Upvotes: 4

Views: 1224

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1123860

The documentation is pretty clear that the CSV module needs to have control over the newlines; in Python 2 you are expected to open the file object in binary mode, in Python 3, you need to set the newline option to ''. See How can I stop Python's csv.DictWriter.writerows from adding empty lines between rows in Windows?

That's because the module writes \r\n line separators to the file by default. The windows text mode handling then replaces \n with \r\n and you see blank lines appear because now lines are separated by \r\r\n.

You can set the lineterminator argument to '\n' to change what the module uses to separate lines:

writer = csv.DictWriter(sys.stdout, fieldnames=fieldnames, 
                        lineterminator='\n')

Demo:

>>> import csv
>>> from StringIO import StringIO
>>> fieldnames = ['first_name', 'last_name']
>>> out = StringIO()
>>> writer = csv.DictWriter(out, fieldnames=fieldnames)
>>> writer.writeheader()
>>> out.getvalue()
'first_name,last_name\r\n'
>>> out = StringIO()
>>> writer = csv.DictWriter(out, fieldnames=fieldnames, lineterminator='\n')
>>> writer.writeheader()
>>> out.getvalue()
'first_name,last_name\n'

sys.stdout will still translate that \n to \r\n but now there is no \r preceding it.

Upvotes: 9

Related Questions