CODEWITHSUNDEEP
python
johnred
johnred

Reputation: 97

Counting Word Frequency in lists with grouping

I have code that counting words frequency (top 3), but the result of the value is returned as a tuple.

Is there a way to improve this code?

result:
defaultdict(<class 'dict'>, {'first': [('earth', 2), ('Jellicle', 2), 
('surface', 2)], 'second': [('first', 2), ('university', 2), ('north', 2)]})


from collections import defaultdict, Counter
words = [
['earth total surface area land Jellicle ', 'first']
,['university earth surface pleasant Jellicle ', 'first']
,['first university east france north ', 'second']
,['first north university ', 'second']
]

result = defaultdict(list)
for row in words:
    sstr = list(row)
    words = [sstr[0],sstr[1]]
    temp = words[0].split()
    for i in range(len(temp)):
        result[words[1]].append(temp[i])
result_finish = defaultdict(dict)
for key in result.keys():
    temp_dict = {key: Counter(result[key]).most_common(3)}
    result_finish.update(temp_dict)
print(result_finish)

Upvotes: 0

Views: 67

Answers (2)

Totoro
Totoro

Reputation: 887

This is a shorter (hopefully cleaner) version of your code using dict comprehensions:

result = defaultdict(list)

for sentence, key in words:
    result[key].extend(sentence.split())

result_count = {k: Counter(v).most_common(3) for k,v in result.items()}

>> result_count: 
>> {'first': [('earth', 2), ('surface', 2), ('Jellicle', 2)], 
>>  'second': [('first', 2), ('university', 2), ('north', 2)]}

If you want it without the count tuple:

result = {k: [w for w,_ in Counter(v).most_common(3)] for k,v in result.items()}

>> result_without_count_per_word
>> {'first': ['earth', 'surface', 'Jellicle'], 
>>  'second': ['first', 'university', 'north']}

Upvotes: 1

jpp
jpp

Reputation: 164623

One way is to aggregate by category via pandas, then use collections.Counter:

import pandas as pd
from collections import Counter

words = [['earth total surface area land Jellicle ', 'first'],
         ['university earth surface pleasant Jellicle ', 'first'],
         ['first university east france north ', 'second'],
         ['first north university ', 'second']]

df = pd.DataFrame(words).groupby(1)[0].apply(lambda x: x.sum())

result = {df.index[i]: Counter(df.iloc[i].split(' ')).most_common(3) \
                       for i in range(len(df.index))}

# {'first': [('earth', 2), ('surface', 2), ('Jellicle', 2)],
#  'second': [('first', 2), ('university', 2), ('north', 2)]}

Upvotes: 1

Related Questions