CODEWITHSUNDEEP
pythonpandasencoding
red79phoenix
red79phoenix

Reputation: 91

Python encoding problem \ufeff in pandas column name

I have the following dictionary (columnsc) which I am trying to use when renaming dataframe columns. The encoding is messing up the name adding \ufeff to the beginning of the string, this is only found when looking at the first item in the list. How do I get rid of the \ufeff or engineer a solution to rename these columns?

enter image description here

Upvotes: 2

Views: 2120

Answers (1)

digitalarbeiter
digitalarbeiter

Reputation: 2335

u'\ufeff is the "byte order marker" (BOM) in UTF-8 files (and UTF-16 and UTF-32). You need to open the file with open(fname, encoding="utf-8-sig"), i.e. with an encoding that expects the BOM (I'm assuming Python 3 here).

For more details, see this answer https://stackoverflow.com/a/17912811/189018

Upvotes: 3

Related Questions