CODEWITHSUNDEEP
pythonmultithreadingperformancemultiprocessing
Roman
Roman

Reputation: 3241

Running multiple programs from the for loop

This program prints out the number of lines of each text file one by one:

files = glob.glob('*.txt') # 5 files
for f in files:
  with open(f,'r') as fi:
     lines = fi.read().splitlines()
     print len(lines)

How can I write my code so that it runs 5 simultaneous programs and prints number of lines at each program separately?

Upvotes: 0

Views: 133

Answers (1)

beiller
beiller

Reputation: 3135

Here is a multi-threaded version of your program:

import threading

# called by each thread
def do_task(f) :
  with open(f,'r') as fi:
     lines = fi.read().splitlines()
     print len(lines)

threads = []
files = glob.glob('*.txt') # 5 files
for f in files:
    t = threading.Thread(target=do_task, args = (f))
    threads.append(t)
    t.start()

for t in threads :
    t.join()

And for fun here is the multi-process version:

from multiprocessing import Process

# called by each thread
def do_task(f, l) :
  with open(f,'r') as fi:
     lines = fi.read().splitlines()
     l.append(lines)

lengths = []
processes = []
files = glob.glob('*.txt') # 5 files
for f in files:
    p = Process(target=do_task, args = (f,lengths,))
    processes.append(p)
    p.start()

for p in processes :
    p.join()

for l in lengths:
    print l

Upvotes: 2

Related Questions