CODEWITHSUNDEEP
pythonmultithreadingparallel-processinggenerator
McDizzy
McDizzy

Reputation: 239

How to parallize/multithreading multiple generator initialization?

I have multiple generators which need some time for initialization before each of the generators can yield the first result. In my code example below is an example. Each genearator needs 5 seconds for initialization. So the total time is 10s. Is there a way to initialize g1 and g2 parallel? So that the total initialization time is only 5s?

from random import random
from time import sleep


def my_generator():
    sleep(5)
    for i in range(5):
        yield random()

# this is what I want to do in parallel
g1 = my_generator()
g2 = my_generator()

x = [(r1, r2) for r1, r2 in zip(g1, g2)]

Upvotes: 1

Views: 231

Answers (1)

McDizzy
McDizzy

Reputation: 239

I found a soulution with async:

import asyncio
from random import random
from aiostream import stream

async def my_generator():
    await asyncio.sleep(5)
    for i in range(5):
        yield random()

async def main():
    combine = stream.zip(my_generator(), my_generator())
    x = []
    async with combine.stream() as streamer:
        async for item in streamer:
            x.append(item)
    print(x)

asyncio.run(main())

Upvotes: 1

Related Questions