CODEWITHSUNDEEP
pythonregexlistnltk
Rohan Manek
Rohan Manek

Reputation: 31

Replace multiple words in list

I have read a corpus using 

file_directory = 'path'
my_corpus = PlaintextCorpusReader(file_directory,'.*',encoding='latin1')

I perform preprocessing

    totalwords = my_corpus.words()
    docs = [my_corpus.words(f) for f in fids]
    docs2 = [[w.lower()for w in doc]for doc in docs]
    docs3 = [[w for w in doc if re.search('^[a-z]+$',w)]for doc in docs2]
    from nltk.corpus import stopwords
    stop_list = stopwords.words('english')
    docs4 = [[w for w in doc if w not in stop_list]for doc in docs3]
    wordscount = [w for doc in docs4 for w in doc]
    fd_dist_total = nltk.FreqDist(wordscount)
    print(fd_dist_total.most_common(common_words))

Output Received is

words = [('ubs', 131), ('pacific', 130), ('us', 121), ('credit', 113), ('aum', 108), ('suisse', 102), ('asia', 98), ('arm', 95)]

I would like to know if it is possible to replace 102 values of 'suisse' with 'credit-suisse'. Similarly replace 'asia' with 'asia-pacific'

Expected output --

words = [('credit-suisse', 102), ('credit', 11) , ('pacific', 32), ('asia-pacific', 98)]

I tried using 

wordscount1 = [w.replace('asia','asia-pacific').replace('suisse', 'credit-suisse') for w in wordscount]

However i run into obvious errors.

Kindly guide me.

Upvotes: 1

Views: 132

Answers (1)

Sylvain Leroux
Sylvain Leroux

Reputation: 52030

This is rather underspecified as we don't know how to ensure that, for example, count('suisse') >= count('credit'). Particularly, you want to:

  • replace 'suisse' by 'credit-suisse', keeping in credit (first term) credit minus suisse
  • but, in the same time, you want to replace 'asia' by 'asia-pacific', keeping in pacific (second term) pacific minus asia (the opposite of the first case)

You definitively have to clarify that requirement. Maybe are your replacement terms sorted somehow? Anyway, as a starting point:

words = [('ubs', 131), ('pacific', 130), ('us', 121), 
         ('credit', 113), ('aum', 108), ('suisse', 102), 
         ('asia', 98), ('arm', 95)]

d = dict(words)

for terms in (('credit', 'suisse'), ('asia', 'pacific')):
    v1 = d.get(terms[1])
    if v1:
        d['-'.join(terms)] = v1
        v0 = d.get(terms[0],0)
        d[terms[0]] = v0-v1 # how to handle zero or negative values here ?
                            # it is unclear if it should be v1-v0 or v0-v1
                            # or even abs(v0-v1) 


from pprint import pprint

pprint(d)
pprint(d.items())

Producing:

sh$ python3 p.py
{'arm': 95,
 'asia': -32,    # <- notice that value
 'asia-pacific': 130,
 'aum': 108,
 'credit': 11,   # <- and this one
 'credit-suisse': 102,
 'pacific': 130,
 'suisse': 102,
 'ubs': 131,
 'us': 121}
dict_items([('us', 121), ('suisse', 102), ('aum', 108), ('arm', 95),
            ('asia-pacific', 130), ('ubs', 131), ('asia', -32),
            ('credit', 11), ('credit-suisse', 102), ('pacific', 130)])

Upvotes: 1

Related Questions