Python unicode file name with strange characters

Question

I am having problems with unicode in Python2.7. The thing is that I get from database some data and store it in a variable called country with the value u"Espa\xf1a".

If I go to the shell and write the following:

>>>country
>>>u"Espa\xf1a"
>>>print country
>>>España

That's ok. No problem with that. The problem comes when I try to create a file called España.txt as follows:

>>> country = u"Espa\xf1a"
>>> file = "%s.txt" % country
>>> file
u'Espa\xf1a.txt'
>>> print file
España.txt
>>> os.system("touch %s" % file)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf1' in position 10: ordinal not in range(128)

I don't know why this is happening. Could anyone help me? Thanks in advance!

Answer 1

os.system("touch %s" % file)

The POSIX command line and filesystem are a natively byte-based environment, Unicode strings are not available there. Non-ASCII characters are encoded into filenames and commands using some encoding, which can vary from system to system (though on modern Linux it would typically be UTF-8).

sys.getfilesystemencoding() will give you Python's best guess of what encoding is in use on the local filesystem (if you mount other filesystems all bets are off), from variables that hopefully defined in the environment.

You should never call os.system including variables in the command. If there are unexpected characters in the variable they can end up executing arbitrary commands, with disastrous security consequences.

You can use interfaces like subprocess.call(['touch', filename.encode(sys.getfilesystemencoding())]) to take care of the necessary argument escaping, but in general you should avoid launching an external command for anything like touch that you can do directly from Python.

For example:

open(filename, 'wb').close()

(When you open a Unicode filename, Python encodes the name to the default filesystemencoding for you.)

Answer 2

Could well be that your operating system isn't allowing you to create the file. Instead of using touch to create the file, try the python way with.

f = open(file, 'w')
...
f.close()

I am assuming that you are trying to write to the file, and you want the file to be called 'España.txt'.

Answer 3

try : os.system("touch %s" % file.encode('utf-8'))

Answer 4

Try this: print ("Espa\u00F1a"). That should print España.