CODEWITHSUNDEEP
pythonpython-itertools
unice
unice

Reputation: 2842

cycle through multiple list using itertools.cycle()

I have a list of servers. Every server has a list of name on it. example:

server1 = ['a','b','c']
server2 = ['d','e','f']
server3 = ['g','h','i']

I want to iterate per server name not per server. For example after picking 'a' in server1, move to 'd' (not 'b') and so on. If I'm going to use itertools.cycle(), do I have to create a list of server to cycle through? My expected result is ['a','d','g','b','e','h','c','f','i']. Can you give me a simple example on how to cycle in multiple list.

Upvotes: 14

Views: 4274

Answers (8)

Saksham Varma
Saksham Varma

Reputation: 2130

Using chain you can simply do this:

from itertools import chain, izip
server1 = [1, 2]
server2 = [3, 4]
server3 = [4, 5]
print list(chain(*izip(server1, server2, server3))) # [1, 3, 4, 2, 4, 5]


Or you can use chain.from_iterable, it expects an iterable that itself generates iterators.

In your case zip is that iterable, it generates iterators in the form of tuples:

print list(chain.from_iterable(zip(server1, server2, server3))) # [1, 3, 4, 2, 4, 5]


yield too could be used here:

def f():
    server1 = [1, 2]
    server2 = [3, 4]
    server3 = [4, 5]
    for a, b, c in zip(server1, server2, server3):
        yield a
        yield b
        yield c

val = f()
print [val.next() for _ in range(6)] # [1, 3, 4, 2, 4, 5]

Upvotes: 2

Tanveer Alam
Tanveer Alam

Reputation: 5275

We can also use itertools.chain.from_iterable() which is faster in comparison.

import itertools

server1 = ['a','b','c']
server2 = ['d','e','f']
server3 = ['g','h','i']

print list(itertools.chain.from_iterable(zip(server1,server2,server3)))

Results:

['a', 'd', 'g', 'b', 'e', 'h', 'c', 'f', 'i']

Upvotes: 12

Kasravnd
Kasravnd

Reputation: 107297

You can do it the with zip and reduce built-in functions (and in python3 functools.reduce):

>>> list_of_servers=[server1,server2,server3]
>>> s=reduce(lambda x,y:x+y,zip(*list_of_servers))
>>> s
('a', 'd', 'g', 'b', 'e', 'h', 'c', 'f', 'i')

Or instead of reduce() for long lists you can use itertools.chain to concatenate the sub-lists that return a generator:

>>> list(chain(*zip(*[server1,server2,server3])))
['a', 'd', 'g', 'b', 'e', 'h', 'c', 'f', 'i']

NOTE that if you want to iterate over your result you don't have to use list on the result of chain. You can just do something like:

for element in chain(*zip(*[server1,server2,server3])):
     #do stuff

Benchmarking on the preceding recipes:

#reduce()
:~$ python -m timeit "server1 = ['a','b','c'];server2 = ['d','e','f'];server3 = ['g','h','i'];reduce(lambda x,y:x+y,zip(*[server1,server2,server3]))"
1000000 loops, best of 3: 1.11 usec per loop
#itertools.chain()
:~$ python -m timeit "server1 = ['a','b','c'];server2 = ['d','e','f'];server3 = ['g','h','i'];from itertools import chain;chain(*zip(*[server1,server2,server3]))"
100000 loops, best of 3: 2.02 usec per loop

Note that if you don't put the servers within a list it would be faster :

:~$ python -m timeit "server1 = ['a','b','c'];server2 = ['d','e','f'];server3 = ['g','h','i'];reduce(lambda x,y:x+y,zip(server1,server2,server3))"
1000000 loops, best of 3: 0.98 usec per loop

Upvotes: 9

Marcin
Marcin

Reputation: 238269

You can use chain:

import itertools

server1 = ['a','b','c']
server2 = ['d','e','f']
server3 = ['g','h','i']


all_servers = [server1, server2, server3] 

out_list = [s_name for a in itertools.chain(zip(*all_servers)) for s_name in a]

print(out_list)
#['a', 'd', 'g', 'b', 'e', 'h', 'c', 'f', 'i']

Or shorter:

out_list = list(itertools.chain.from_iterable(zip(*all_servers)))

Upvotes: 2

Matt
Matt

Reputation: 744

from itertools import chain
for s in chain(*zip(server1, server2, server3)):
    # do work

Upvotes: 3

mgilson
mgilson

Reputation: 309929

This one works alright:

>>> from itertools import chain, islice, izip, cycle
>>> list(islice(cycle(chain.from_iterable(izip(server1, server2, server3))), 0, 18))
['a', 'd', 'g', 'b', 'e', 'h', 'c', 'f', 'i', 'a', 'd', 'g', 'b', 'e', 'h', 'c', 'f', 'i']

Note, the list and islice are just for demonstration purposes to give something to display and prevent infinite output...

Now, it gets more interesting if you have unequal length lists. Then izip_longest will be your friend, but it might be worth a function at this point:

import itertools
def cycle_through_servers(*server_lists):
    zipped = itertools.izip_longest(*server_lists, fillvalue=None)
    chained = itertools.chain.from_iterable(zipped)
    return itertools.cycle(s for s in chained if s is not None)

demo:

>>> from itertools import islice
>>> server3 = ['g', 'h', 'i', 'j']
>>> list(islice(cycle_through_servers(server1, server2, server3), 0, 20))
['a', 'd', 'g', 'b', 'e', 'h', 'c', 'f', 'i', 'j', 'a', 'd', 'g', 'b', 'e', 'h', 'c', 'f', 'i', 'j']

Upvotes: 7

itzMEonTV
itzMEonTV

Reputation: 20349

Try this:

from itertools import cycle
for k in cycle([j for i in zip(server1,server2,server3) for j in i]):
   print(k)
   #do you operations

a
d
g
b
...

But care this provides infinite loop

So better do this:

c = cycle([j for i in zip(server1,server2,server3) for j in i])

>>>next(c)
a
>>>next(c)
b
....

Upvotes: 3

Paul Hankin
Paul Hankin

Reputation: 58271

The standard library documentation provides this function as a recipe in itertools.

def roundrobin(*iterables):
    "roundrobin('ABC', 'D', 'EF') --> A D E B F C"
    # Recipe credited to George Sakkis
    pending = len(iterables)
    nexts = cycle(iter(it).next for it in iterables)
    while pending:
        try:
            for next in nexts:
                yield next()
        except StopIteration:
            pending -= 1
            nexts = cycle(islice(nexts, pending))

This code works even when the iterables are of uneven lengths, cycling through remaining ones when the shorter ones are used up. That may or may not be relevant to your use-case.

Upvotes: 5

Related Questions