CODEWITHSUNDEEP
pythoncsv
LWZ
LWZ

Reputation: 12338

How to ignore NULL byte when reading a csv file

I'm reading a csv file generated from an equipment and I got this error message:

Error: line contains NULL byte

I opened the csv file in text editor and I do see there're some NUL bytes in the header section, which I don't really care about. How can I make the csv reader function to ignore the NUL byte and just goes through the rest of the file?

There're two blank lines between the header section and the data, maybe there's way to skip the whole header?

My code for reading the csv file is 

with open(FileName, 'r', encoding='utf-8') as csvfile:
  csvreader = csv.reader(csvfile)

Upvotes: 7

Views: 17222

Answers (3)

DennisLi
DennisLi

Reputation: 4154

This worked. replaced 'Null' character to '' in a file

sed -i 's/\x0//g' test.csv

Upvotes: 0

CENTURION
CENTURION

Reputation: 363

When reading in a csv, you have to remove those.

There was an article I read a while ago, it was called the taco bell method of programming. In it, the article posits that taco bell really only has 8 ingredients, but from it, makes all their chalupas, and bean thingy's, and other inedible food products.

Probably should add doritos to that ingredient list. Still, the point remains.

wget, awk, sed, etc. Those should be used when you can. No point in making it overly complicated and bringing in all these libs to do it in one language.

So, I ask, can you do it in UNIX first? And you can.

UNIX

tr < file-in -d '\000' > file-out

it's quick and will work.

...Now, get back to tacos.

Upvotes: 4

Marlon Abeykoon
Marlon Abeykoon

Reputation: 12465

This will replace your NULL bytes

csvreader = csv.reader(x.replace('\0', '') for x in csvfile)

Upvotes: 14

Related Questions