CODEWITHSUNDEEP
pythonparallel-processing
Geeocode
Geeocode

Reputation: 5797

How could I resolve python - parallel issue?

I wrote a code which is something like this:

import time
import pp

class gener(object):
    def __init__(self, n):
        self.n = n

    def __call__(self):
        return self    

    def __iter__(self):
        n = self.n
        for i in range(n):
            time.sleep(2)
            yield i

def gen():
    return gener(3)

job_server = pp.Server(4)
job = job_server.submit(
                        gen, 
                        args=(),
                        depfuncs=("gener",),
                        modules=("time",),
                        )

print job()

following @zilupe useful comment I got the following output:

<__main__.gener object at 0x7f862dc18a90>

How can I insert class gener iteration to parallel?

I'd like to run it parallel with other functions. I need this generator like class, because I want to replace an other code from a modul of a rather complicated program package and it would be hard to refactor that code.

I tried a lot of thing without any success so far. I made a rather solid investigation on this site, but I couldn't find the right answer suitable for me. Anybody, any help? Thank you.

ADDITION:

According to @zilupe useful comments some additional information:

The main purpose is to parallelize the iteration inside of class gener:

 def __iter__(self):
        n = self.n
        for i in range(n):
            time.sleep(2)
            yield i

I only created of gen() function because I wasn't be able to figure out, how could I use it directly in parallel.submit().

Upvotes: 0

Views: 49

Answers (1)

jbasko
jbasko

Reputation: 7330

This doesn't answer your question but you get the error because depfuncs has to be a tuple of functions, not strings -- the line should be: depfuncs=(gener,),

Do you realise that function gen is only creating an instance of gener and is not actually calling it?

What are you trying to parallelise here -- creation of a generator or iteration over the generator? If it's the iteration, you should probably create the generator first, pass it to gen in args and then in gen iterate over it:

def gen(g):
    for _ in g:
        pass

job_server.submit(gen,args=(gener(3),),...)

Upvotes: 2

Related Questions