CODEWITHSUNDEEP
pythonjsonutf-8invalid-characters
troex
troex

Reputation: 1100

Check for valid utf8 string in Python

I'm reading filenames from file system and I want to send them as JSON encoded array. The problem is that files on file system can be stored in invalid encoding, and I need to handle this situation to omit invalid filenames before passing it to json.dump, otherwise it will fail.

Is there a way to check that my string (filename) contains valid utf-8 chars?

Upvotes: 12

Views: 22431

Answers (1)

Mark Longair
Mark Longair

Reputation: 467231

How about trying the following?

valid_utf8 = True
try:
    filename.decode('utf-8')
except UnicodeDecodeError:
    valid_utf8 = False

... based on an answer to a similar question here: How to write a check in python to see if file is valid UTF-8?

Upvotes: 19

Related Questions