CODEWITHSUNDEEP
pythonpython-itertools
Scott Zimmerman
Scott Zimmerman

Reputation: 35

Python itertools.product challenge to expand a dict with tuples

Given a dictionary like this with some items being tuples...

params = {
 'a': 'static',
 'b': (1, 2),
 'c': ('X', 'Y')
}

I need the "product" of the items into a list of dict like this, with the tuples expanded so each item in b will be matched with each item in c...

[{ 'a': 'static', 'b': 1, 'c': 'X' },
 { 'a': 'static', 'b': 1, 'c': 'Y' },
 { 'a': 'static', 'b': 2, 'c': 'X' },
 { 'a': 'static', 'b': 2, 'c': 'Y')}]

I can easily separate the initial input into a list of non-tuple items and tuple items, and apply the key of each tuple to the values as a "tag" prior to multiplication so they look like this: 'b##1', 'b##2', 'c##X', 'c##Y'. Then parse those back into the above dict after multiplication. If I would always see 2 tuple items (like b and c), I could easily pass both to itertools.products. But there could be 0..n tuple items, and product() doesn't multiply a list of lists in this way. Can anyone think of a solution?

TAG = '##'      
# separate tuples and non-tuples from the input, and prepend the key of each tuple as a tag on the value to parse out later
for key, value in params.items():
    if type(value) is tuple:
        for x in value:
            tuples.append(f'{key}{TAG}{x}')
    else:
        non_tuples.append({key: value})
print(list(product(tuples))      # BUG: doesn't distribute each value of b with each value of c

Upvotes: 2

Views: 118

Answers (3)

no comment
no comment

Reputation: 10166

The obvious itertools.product solution was already posted, so here's a fast alternative. Expands one param at a time (Attempt This Online!):

params = {
    'a': 'static',
    'b': (1, 2),
    'c': ('X', 'Y')
}

out = [{}]
for k, v in params.items():
    if not isinstance(v, tuple):
        v = v,
    out = [d.copy()
           for d in out
           for d[k] in v]

print(out)

Times for three cases:

19 parameters with 2 options each:
  0.75 seconds  no_comment
  1.26 seconds  Andrej

10 parameters with 4 options each:
  0.60 seconds  no_comment
  1.63 seconds  Andrej

6 parameters with 10 options each:
  0.46 seconds  no_comment
  1.18 seconds  Andrej

Benchmark script:

from itertools import product
from time import time
from string import ascii_lowercase

params = {c: (i, -i) for i, c in enumerate(ascii_lowercase[:19])}
params = {c: tuple(range(4)) for i, c in enumerate(ascii_lowercase[:10])}

def no_comment(params):
    out = [{}]
    for k, v in params.items():
        if not isinstance(v, tuple):
            v = v,
        out = [d.copy()
               for d in out
               for d[k] in v]
    return out

def Andrej(params):
    return [
        dict(zip(params, vals))
        for vals in product(
            *(v if isinstance(v, (tuple, list)) else (v,) for v in params.values())
        )
    ]

print(no_comment(params) == Andrej(params))

for f in [no_comment, Andrej] * 3:
    t0 = time()
    f(params)
    print(time() - t0, f.__name__)

Attempt This Online!

A possible optimization is hidden in a markdown comment, view the answer's source to see it.

Upvotes: 0

pho
pho

Reputation: 25489

product takes multiple iterables, but the key thing to remember is that an iterable can contain a single item. In cases where a value in your original dict isn't a tuple (or maybe a list), you want to convert it to a tuple containing a single value and pass that to product:

params_iterables = {}
for k, v in params.items():
    if isinstance(v, (tuple, list)):
        params_iterables[k] = v     # v is already a tuple or a list
    else:
        params_iterables[k] = (v, ) # A tuple containing a single value, v

which gives:

params_iterables = {'a': ('static',), 'b': (1, 2), 'c': ('X', 'Y')}

Then, simply get the product of the values in params_iterables:

result = []
for values in product(*params_iterables.values()):
    result.append(dict(zip(params, values)))

The dict(zip(params, values)) line creates a dict where the first element of values is assigned the first key in params, and so on. This dict is then appended to result, which gives the desired output:

[{'a': 'static', 'b': 1, 'c': 'X'},
 {'a': 'static', 'b': 1, 'c': 'Y'},
 {'a': 'static', 'b': 2, 'c': 'X'},
 {'a': 'static', 'b': 2, 'c': 'Y'}]

Upvotes: 3

Andrej Kesely
Andrej Kesely

Reputation: 195428

Try:

from itertools import product

params = {"a": "static", "b": (1, 2), "c": ("X", "Y")}


out = []
for vals in product(
    *[v if isinstance(v, (tuple, list)) else (v,) for v in params.values()]
):
    out.append(dict(zip(params, vals)))

print(out)

Prints:

[
    {"a": "static", "b": 1, "c": "X"},
    {"a": "static", "b": 1, "c": "Y"},
    {"a": "static", "b": 2, "c": "X"},
    {"a": "static", "b": 2, "c": "Y"},
]

One-liner:

out = [
    dict(zip(params, vals))
    for vals in product(
        *(v if isinstance(v, (tuple, list)) else (v,) for v in params.values())
    )
]

Upvotes: 4

Related Questions