Reputation: 571
I have a large dictionary (10,000+ entries) of ReviewIDs. The dictionary has two keys, the first is the ReviewID # and the second is the language of the Review.
My task is to compute the total # of reviews in each language and then display it in a bar plot.
import pandas as pd
import csv
import matplotlib.pyplot as plt
import sys
RevDict = {}
with open('ReviewID.txt','r') as f:
for line in f:
a,b = line.split(":")
RevDict[a] = str(b)
This results in a dictionary that looks like this:
My idea, was to convert the dictionary into a Dataframe with the Review ID being one column and the language being a second column. I could then iterate through the rows using a counter and end up with a final count for each language. This could easily be converted into a bar plot.
Unfortunately, I can't figure out how to do this.
I also suspect that the more pythonic approach would be to simply count the # of instances of each string within the dictionary itself rather than going through the step of making a dataframe. I tried this:
from collections import Counter
Counter(k['b'] for k in data if k.get('b'))
It is throwing the following error:
AttributeError: 'str' object has no attribute 'get'
Upvotes: 0
Views: 154
Reputation: 48659
Using collections.Counter
import collections as coll
data = {
'A': 'English',
'B': 'German',
'C': 'English'
}
print(coll.Counter(data.values()))
--output:--
Counter({'English': 2, 'German': 1})
Using pandas
:
import pandas as pd
data = {
'A': 'fr\n',
'B': 'de\n',
'C': 'fr\n',
'D': 'de\n',
'E': 'fr\n',
'F': 'en\n'
}
df = pd.DataFrame(
{
'id': list(data.keys()),
'lang': [val.rstrip() for val in data.values()],
}
)
print(df)
output:
id lang
0 B de
1 A fr
2 F en
3 D de
4 E fr
5 C fr
grouped = df.groupby('lang')
print(grouped.size())
output:
lang
de 2
en 1
fr 3
Response to comment
Plotting
:
import collections as coll
import matplotlib.pyplot as plt
import numpy as np
from operator import itemgetter
data = {
'A': 'fr\n',
'B': 'de\n',
'C': 'fr\n',
'D': 'de\n',
'E': 'fr\n',
'F': 'en\n'
}
counter = coll.Counter(
[val.rstrip() for val in data.values()]
)
langs, lang_counts = zip(
*sorted(counter.items(), key=itemgetter(1))
)
total_langs = sum(lang_counts)
bar_heights = np.array(lang_counts, dtype=float) / total_langs
x_coord_left_side_of_bars = np.arange(len(langs))
bar_width = 0.8
plt.bar(
x_coord_left_side_of_bars,
bar_heights,
bar_width,
)
plt.xticks(
x_coord_left_side_of_bars + (bar_width * 0.5), #position of tick marks
langs #labels for tick marks
)
plt.xlabel('review language')
plt.ylabel('% of all reviews')
x = plt.plot()
#plt.show() #Can use show() instead of savefig() until everything works correctly
plt.savefig('lang_plot.png')
plot:
Upvotes: 2
Reputation: 1125398
In your for k in data
loop, each k
is a string key (the review id). Strings have no .get()
method, nor does the original variable b
have any bearing on this loop.
If you wanted to count the values, just pass the values of the dictionary straight to the Counter
:
Counter(data.values())
You probably want to remove the newline characters first:
for line in f:
review_id, lang = line.split(":")
RevDict[review_id] = lang.strip()
Upvotes: 1