CODEWITHSUNDEEP
pythonmultiprocessing
Winnie-c
Winnie-c

Reputation: 209

python multiprocessing module cause infinite loop

I am trying to use pool from multiprocessing to speed up my read/write function. However, when I try to use pyinstaller and run it with an exe. The result is a new process that spawns a new process that spawns a new process ad infinitum until I log out and the OS forcefully terminates all my user processes. I have no idea what is happening

There is no problem when running in the pycharm or even running the code with running the .py.

The version is python 3.6.3.

Platform: Windows 7

The code will be like:

if __name__ == "__main__":
    pool = Pool(2)
    pool.map(readwritevalue, [file for file in gettxtpath()])
    pool.close()
    pool.join()
    GetFormated()  ##Get the Final output of .csv
    Getxlsx()  ##Get the Report.xls
    GetDocx()  ##Get the docx format
    t1 = time.time()
    print("Parallel time{t}s".format(t=time.time() - t1))

Would you please shed some lights on this? I searched the Google and some answers are to put the multiprocessing module under the "__name__=="__main__"", However, I already did that. So what else will cause that infinite explosion of processes?

Thank you very much.

Upvotes: 3

Views: 2465

Answers (1)

Winnie-c
Winnie-c

Reputation: 209

Thanks MilanVelebit for helping to find out the answer.

I post my answer here. You just need to add one line when you import multiprocessing and use pyinstaller.

from multiprocessing import Pool,freeze_support

if __name__ == "__main__":

    ##Add support for when a program which uses multiprocessing has been frozen to produce a Windows executable. (Has been tested with py2exe, PyInstaller and cx_Freeze.)
    #One needs to call this function straight after the '__main__' line of the main module.

    freeze_support()
    t1 = time.time()
    pool = Pool(2)
    pool.map(readwritevalue, [file for file in gettxtpath()])
    pool.close()
    pool.join()
    GetFormated()  ##Get the Final output of Xunjian-Report.csv
    Getxlsx()  ##Get the Xunjian-Report.xls
    GetDocx()  ##Get the docx format
    print("Cost time {t}".format(t=time.time() - t1))

Upvotes: 2

Related Questions