CODEWITHSUNDEEP
pythonunicodeutf-8
Sadık
Sadık

Reputation: 4419

UnicodeEncodeError only when pipe into file and only on some PCs

Many people had this problem, but the presented solutions dind't help me.

On my ubuntu machine the script runs without errors. But on my raspi keep getting this error:

UnicodeEncodeError: 'ascii' codec can't encode character '\xd6' in position 21: ordinal not in range(128)

The error hints to a German character (umlaut, ö) which should be printed using print(a_name)

What confuses me: When calling python3 myscript.py I don't get any errors on raspi. But with the call python3 myscript.py> output I get the error. Same with nohup python3 myscript.py or when run from crontab:

@reboot LANG=de_DE.UTF-8 /home/pi/launcher.sh > /home/pi/bot/logs/cronlog 2>&1

where launcher.sh uses the following code:

python3 myscript.py > pythonlog 2>&1

I checked $LANG

on my ubuntu pc (where I don't get any errors): en_US.UTF-8

on raspi: de_DE.UTF-8

Why am I getting this error message and how can I get rid of it?

Upvotes: 1

Views: 465

Answers (2)

Alastair McCormack
Alastair McCormack

Reputation: 27704

Under normal conditions, Python uses the locale to work out what encoding to apply to stdout.

When Python doesn't have an interactive session, i.e. when input or output is piped or redirected, Python defaults to "ASCII" encoding on stdout and stdin.

To override this behaviour, use the PYTHONIOENCODING environment variable to set an appropriate input/output encoding. E.g.

PYTHONIOENCODING=utf-8 echo "test €3.20" | myscript.py > output.txt

Upvotes: 3

mozillazg
mozillazg

Reputation: 720

Have you tried setting environment variable PYTHONIOENCODING?

export PYTHONIOENCODING=UTF-8
python3 myscript.py > pythonlog 2>&1

Upvotes: 5

Related Questions