Python list of lists manipulation

Question

I have a Python problem that can be solved with multiple nested for loops but I was wondering if there is an easier way to solve this, maybe by adding list items together and dropping duplicates.

My list looks like this:

main_list = [["user1@test.com", "Administration", "100"],
             ["user2@test.com", "Testing", "30"],
             ["user2@test.com", "Development", "45"],
             ["user2@test.com", "Development", "90"],
             ["user2@test.com", "Development", "35"],
             ["user3@test.com", "Development", "400"],
             ["user3@test.com", "Administration", "95"],
             ["user3@test.com", "Testing", "200"]]

I need to merge the email address and category (the first two list elements) and add the duplicate 3rd entries together.

So [user2, development] goes from:

["user2@test.com", "Development", "45"],
["user2@test.com", "Development", "90"],
["user2@test.com", "Development", "35"],

to:

["user2@test.com", "Development", "170"]

It this possible with list manipulation?

Thank you!

Answer 1

Exemplified, step by step.

main_dict = {}
for email, category, value in main_list:
    token = (email, category)
    if token in main_dict:
        main_dict[token] += int(value)
    else:
        main_dict[token] = int(value)

main_list_converted = []
for k, v in main_dict.iteritems():
    main_list_converted.append(list(k) + [v])

main_list_converted.sort()

"""
for item in main_list_converted:
    print (item)

[['user1@test.com', 'Administration', 100]
['user2@test.com', 'Development', 170]
['user2@test.com', 'Testing', 30]
['user3@test.com', 'Administration', 95]
['user3@test.com', 'Development', 400]
['user3@test.com', 'Testing', 200]]
"""

Answer 2

With pandas module:

import pandas as pd
out_d = (pd.DataFrame(main_list).set_index([0,1])[2].astype(int).groupby(level=[0,1]).sum()).to_dict()
out_d

Out[1]:
{('user1@test.com', 'Administration'): 100,
 ('user2@test.com', 'Development'): 170,
 ('user2@test.com', 'Testing'): 30,
 ('user3@test.com', 'Administration'): 95,
 ('user3@test.com', 'Development'): 400,
 ('user3@test.com', 'Testing'): 200}

#for list
[[u[0],u[1],v] for u,v in out_d.items()]

Out[2]:
[['user1@test.com', 'Administration', 100],
 ['user2@test.com', 'Development', 170],
 ['user2@test.com', 'Testing', 30],
 ['user3@test.com', 'Administration', 95],
 ['user3@test.com', 'Development', 400],
 ['user3@test.com', 'Testing', 200]]

Answer 3

You can use the function groupby():

from itertools import groupby
from operator import itemgetter

iget = itemgetter(0, 1)
[[*k, sum(int(i[2]) for i in g)] for k, g in groupby(sorted(main_list), key=iget)]

Result:

[['user1@test.com', 'Administration', 100],
 ['user2@test.com', 'Development', 170],
 ['user2@test.com', 'Testing', 30],
 ['user3@test.com', 'Administration', 95],
 ['user3@test.com', 'Development', 400],
 ['user3@test.com', 'Testing', 200]]

Answer 4

Using collections.defaultdict

Ex:

from collections import defaultdict


main_list = [["user1@test.com", "Administration", "100"],
             ["user2@test.com", "Testing", "30"],
             ["user2@test.com", "Development", "45"],
             ["user2@test.com", "Development", "90"],
             ["user2@test.com", "Development", "35"],
             ["user3@test.com", "Development", "400"],
             ["user3@test.com", "Administration", "95"],
             ["user3@test.com", "Testing", "200"]]
result = defaultdict(int)
for k, v, n in main_list:
    result[(k, v)] += int(n)
result = [list(k) + [v] for k, v in result.items()]
print(result)

Output:

[['user1@test.com', 'Administration', 100],
 ['user2@test.com', 'Testing', 30],
 ['user2@test.com', 'Development', 170],
 ['user3@test.com', 'Development', 400],
 ['user3@test.com', 'Administration', 95],
 ['user3@test.com', 'Testing', 200]]