CODEWITHSUNDEEP
pythonstringtranslationspecial-characters
Aufwind
Aufwind

Reputation: 26258

Strip Non alpha numeric characters from string in python but keeping special characters

I know similar questions were asked around here on StackOverflow. I tryed to adapt some of the approaches but I couldn't get anything to work, that fits my needs:

Given a python string I want to strip every non alpha numeric charater - but - leaving any special charater like µ æ Å Ç ß... Is this even possible? with regexes I tryed variations of this

re.sub(r'[^a-zA-Z0-9: ]', '', x) # x is my string to sanitize

but it strips me more then I want. An example of what I want would be:

Input:  "A string, with characters µ, æ, Å, Ç, ß,... Some    whitespace  confusion  ?"
Output: "A string with characters µ æ Å Ç ß Some whitespace confusion"

Is this even possible without getting complicated?

Upvotes: 4

Views: 7870

Answers (4)

Justin Turner Arthur
Justin Turner Arthur

Reputation: 988

If you're ok with the Unicode Consortium's classification of what's a letter or a digit, an easy way to do this without RegEx or importing anything outside the built-ins:

filter(unicode.isalnum, u"A string, with characters µ, æ, Å, Ç, ß,... Some    whitespace  confusion  ?")

If you have a str instead of a unicode, you'll need to encode it first.

Upvotes: 1

Tugrul Ates
Tugrul Ates

Reputation: 9687

Eliminate characters in "Punctuation, Other" Unicode category.

# -*- coding: utf-8 -*-

import unicodedata

# This removes punctuation characters.
def strip_po(s):
  return ''.join(x for x in s if unicodedata.category(x) != 'Po')

# This reduces multiple whitespace characters into a single space.
def fix_space(s):
  return ' '.join(s.split())

s = u'A string, with characters µ, æ, Å, Ç, ß,... Some    whitespace  confusion  ?'
print fix_space(strip_po(s))

Upvotes: 3

Ray Toal
Ray Toal

Reputation: 88378

Use \w with the UNICODE flag set. This will match the underscore also, so you might need to take care of that separately.

Details on http://docs.python.org/library/re.html.

EDIT: Here is some actual code. It will keep unicode letters, unicode digits, and spaces.

import re
x = u'$a_bßπ7: ^^@p'
pattern = re.compile(r'[^\w\s]', re.U)
re.sub(r'_', '', re.sub(pattern, '', x))

If you did not use re.U then the ß and π characters would have been stripped.

Sorry I can't figure out a way to do this with one regex. If you can, can you post a solution?

Upvotes: 3

dfb
dfb

Reputation: 13289

You'll have to better define what you mean by special characters. There are certain flags that will group things like whitespace, non-whitespace, digits, etc. and do it specific to a locale. See http://docs.python.org/library/re.html for more details.

However, since this is a character by character operation, you may find it easier to simply explicitly specify every character, or, if the number of characters you want to exclude is smaller, writing an expression that only excludes those.

Upvotes: 1

Related Questions