CODEWITHSUNDEEP
pythonpython-itertools
elsonwx
elsonwx

Reputation: 1629

How to slice process itertools.product in python?

I want to process a very large itertools.product object. The issue is something like this:

import string
from itertools import product
text = string.lowercase[:] + string.uppercase[:] + '0123456789'
items = product(text, repeat=5)
for item in items:
    #do something

I know the items's length is 62**5. If I want to process the elements of items whose indices range from 300000 to 600000, how to achieve this?

I have tried to convert the itertools.product to python list, like this:

items = list(product(text, repeat=5))[300000:600000+1]
for item in items:
    #do something

but it seems the conversion has consumed a large amount of memory since I have been waiting for a long time for this convert, and finally gave it up.

I have this demand because I want to do this thing in python gevent, so I want to slice the large itertool.product to small items for gevent spawn.

Upvotes: 6

Views: 1705

Answers (1)

Sede
Sede

Reputation: 61225

You can use islice to do this.

from itertools import product, islice
import string 

text = string.ascii_letters + string.digits
for item in islice(product(text,repeat=5), 300000, 600000):
    # do something

Upvotes: 10

Related Questions