How to replace specific punctuation with new name?

Question

My data sample is:

        comment sarc_majority
0        [?, ?]          sarc
1           [0]      non-sarc
2     [!, !, !]          sarc
3           [0]      non-sarc
4           [?]          sarc

I want to replace the punctuation with a new name. Such as ? = punct1, ! = punct2, ' = punct3. I tried using read from csv file.

replace_df = pd.read_csv('./final/eng-mly-punct.csv', sep=',', quoting=csv.QUOTE_NONE,
                       names=["punct", "replacer"])
replace_df.head()

    punct   replacer
0   ?       punct1
1   !       punct2
2   '       punct3

Then I stucked at replacing:

for punct, replacer in replace_df.itertuples(index=False,name=None):
    df.comment = df.comment.str.replace(r'\b{0}\b'.format(punct),replacer)

The error is: error: nothing to repeat

What have gone wrong? Or is there any possible way to do this? The desired output should be just like:

                       comment sarc_majority
0             [punct1, punct1]          sarc
1                          [0]      non-sarc
2     [punct2, punct2, punct2]          sarc
3                          [0]      non-sarc
4                     [punct1]          sarc

Thanks in advance. Cheers.

Answer 1

You can use replace by dict d - but need escape ? to \?:

d = {'\?':'punct1','!':'punct2',"'":'punct3'}
df.comment = df.comment.replace(d, regex=True)
print (df)
                    comment sarc_majority
0          [punct1, punct1]          sarc
1                       [0]      non-sarc
2  [punct2, punct2, punct2]          sarc
3                       [0]      non-sarc
4                  [punct1]          sarc

Also you can create d from replace_df:

df = pd.DataFrame({'comment': {0: '[?, ?]', 1: '[0]', 2: '[!, !, !]', 3: '[0]', 4: '[?]'}, 'sarc_majority': {0: 'sarc', 1: 'non-sarc', 2: 'sarc', 3: 'non-sarc', 4: 'sarc'}})
print (df)
     comment sarc_majority
0     [?, ?]          sarc
1        [0]      non-sarc
2  [!, !, !]          sarc
3        [0]      non-sarc
4        [?]          sarc

replace_df = pd.DataFrame({'replacer': {0: 'punct1', 1: 'punct2', 2: 'punct3'}, 'punct': {0: '?', 1: '!', 2: "'"}})
print (replace_df)
  punct replacer
0     ?   punct1
1     !   punct2
2     '   punct3

replace_df.punct = '\\' + replace_df.punct
d = replace_df.set_index('punct')['replacer'].to_dict()
print (d)
{'\\!': 'punct2', "\\'": 'punct3', '\\?': 'punct1'}

df.comment = df.comment.replace(d, regex=True)
print (df)
                    comment sarc_majority
0          [punct1, punct1]          sarc
1                       [0]      non-sarc
2  [punct2, punct2, punct2]          sarc
3                       [0]      non-sarc
4                  [punct1]          sarc

EDIT by comment:

df = pd.DataFrame({'comment':[['?', '?'],[0], ['!', '!', '!'], [0], ['?']], 'sarc_majority': [ 'sarc','non-sarc', 'sarc', 'non-sarc','sarc']})
print (df)
     comment sarc_majority
0     [?, ?]          sarc
1        [0]      non-sarc
2  [!, !, !]          sarc
3        [0]      non-sarc
4        [?]          sarc

print (type(df.ix[0,'comment']))
<class 'list'>

replace_df = pd.DataFrame({'replacer': {0: 'punct1', 1: 'punct2', 2: 'punct3'}, 'punct': {0: '?', 1: '!', 2: "'"}})
#print (replace_df)

replace_df.punct = '\\' + replace_df.punct.apply(lambda x: x.format())
d = replace_df.set_index('punct')['replacer'].to_dict()
print (d)
{'\\!': 'punct2', "\\'": 'punct3', '\\?': 'punct1'}

df.comment = df.comment.apply(lambda x: pd.Series(x).astype(str).replace(d, regex=True).tolist())
print (df)
                    comment sarc_majority
0          [punct1, punct1]          sarc
1                       [0]      non-sarc
2  [punct2, punct2, punct2]          sarc
3                       [0]      non-sarc
4                  [punct1]          sarc

Answer 2

Most punctuation characters have a special meaning in regular expressions. Here you end up with, eg: \b?\b, which means an optional boundary followed by a boundary. Not what you meant.

For passing arbitrary strings into a regexp, it must be escaped using re.escape:

import re
r'\b{0}\b'.format(re.escape(punct))

This will be \b\?\b, which means a boundary, followed by a ?, followed by another boundary.