CODEWITHSUNDEEP
pythonfilemultiprocessing
Анастасия Белоусова
Анастасия Белоусова

Reputation: 25

Multiprocessing for reading files (Python)

I have a list of files (as classes, see the realisation below).

class F:
 def __init__(self,path):
  self.path=path
  self. size=0
 def calculate_size()
   with open(self.path,”rb”) as f:
         self.size=len(f.read())

I want to use the multiprocessing library to calculate file sizes parallel. I tried to do it with Pool,but all sizes was 0. I need to parallel this:

for fail in fails:
    fail.calculate_size()

Upvotes: 1

Views: 40

Answers (1)

Mehrdad Saberi
Mehrdad Saberi

Reputation: 31

You can use python's threading library like the example below.

from threading import Thread

class F:
    def __init__(self, path):
        self.path = path
        self.size = 0
    def calculate_size(self):
        print("calculation started")
        with open(self.path, 'rb') as f:
            self.size = len(f.read())
        print(self.size)
        print("calculation ended")
        

fails = [F(x) for x in ['x', 'y']] # file names here

threads = []
for fail in fails:
    thread = Thread(target=fail.calculate_size,args=())
    thread.start()
    threads.append(thread)
for thread in threads:
    thread.join()

Upvotes: 2

Related Questions