CODEWITHSUNDEEP
python-3.xpycharmpython-unicode
amarchin
amarchin

Reputation: 2114

UnicodeDecodeError when running a test in PyCharm

I have a test in my project that involves opening and reading a file. The chunk of code is basically something like this:

import pandas as pd

with open('path/to/the/file') as file:
  df = pd.read_csv(file, comment='#')

When I run the test in PyCharm I got this error:

  File "/Users/andrea.marchini/.local/share/virtualenvs/customer-ontology-OqrlJ3HG/lib/python3.5/site-packages/pandas/io/parsers.py", line 655, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/Users/andrea.marchini/.local/share/virtualenvs/customer-ontology-OqrlJ3HG/lib/python3.5/site-packages/pandas/io/parsers.py", line 411, in _read
    data = parser.read(nrows)
  File "/Users/andrea.marchini/.local/share/virtualenvs/customer-ontology-OqrlJ3HG/lib/python3.5/site-packages/pandas/io/parsers.py", line 1005, in read
    ret = self._engine.read(nrows)
  File "/Users/andrea.marchini/.local/share/virtualenvs/customer-ontology-OqrlJ3HG/lib/python3.5/site-packages/pandas/io/parsers.py", line 1748, in read
    data = self._reader.read(nrows)
  File "pandas/_libs/parsers.pyx", line 890, in pandas._libs.parsers.TextReader.read (pandas/_libs/parsers.c:10862)
  File "pandas/_libs/parsers.pyx", line 912, in pandas._libs.parsers.TextReader._read_low_memory (pandas/_libs/parsers.c:11138)
  File "pandas/_libs/parsers.pyx", line 966, in pandas._libs.parsers.TextReader._read_rows (pandas/_libs/parsers.c:11884)
  File "pandas/_libs/parsers.pyx", line 953, in pandas._libs.parsers.TextReader._tokenize_rows (pandas/_libs/parsers.c:11755)
  File "pandas/_libs/parsers.pyx", line 2173, in pandas._libs.parsers.raise_parser_error (pandas/_libs/parsers.c:28589)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 202538: ordinal not in range(128)

Anyway if I run the same code from the python console everything works fine.

Upvotes: 1

Views: 178

Answers (1)

amarchin
amarchin

Reputation: 2114

Adding this line to the .bash_profile file solves the issue

if [ -z "$LANG" ]; then export LANG="$(defaults read -g AppleLocale | sed 's/@.*$//g').UTF-8"; fi

Upvotes: 2

Related Questions