CODEWITHSUNDEEP
pythonmultithreading
kvv
kvv

Reputation: 356

Multithreading works slower

Good day!

I'm trying to learn multithreading features in python and I wrote the following code:

import time, argparse, threading, sys, subprocess, os

def item_fun(items, indices, lock):
    for index in indices:
        items[index] = items[index]*items[index]*items[index]

def map(items, cores):  

    count = len(items)
    cpi = count/cores
    threads = []
    lock = threading.Lock()
    for core in range(cores):
        thread = threading.Thread(target=item_fun, args=(items, range(core*cpi, core*cpi + cpi), lock))
        threads.append(thread)
        thread.start()
    item_fun(items, range((core+1)*cpi, count), lock)
    for thread in threads:
        thread.join()


parser = argparse.ArgumentParser(description='cube', usage='%(prog)s [options] -n')
parser.add_argument('-n', action='store', help='number', dest='n', default='1000000', metavar = '')
parser.add_argument('-mp', action='store_true', help='multi thread', dest='mp', default='True')
args = parser.parse_args()

items = range(NUMBER_OF_ITEMS)
# print 'items before:'
# print items
mp = args.mp
if mp is True:
    NUMBER_OF_PROCESSORS = int(os.getenv("NUMBER_OF_PROCESSORS"))
    NUMBER_OF_ITEMS = int(args.n)
    start = time.time()
    map(items, NUMBER_OF_PROCESSORS)
    end = time.time()
else:
    NUMBER_OF_ITEMS = int(args.n)
    start = time.time()
    item_fun(items, range(NUMBER_OF_ITEMS), None)
    end = time.time()       
#print 'items after:'
#print items
print 'time elapsed: ', (end - start)

When I use mp argument, it works slower, on my machine with 4 cpus, it takes about 0.5 secs to compute result, while if I use a single thread it takes about 0.3 secs.

Am I doing something wrong?

I know there's Pool.map() and e.t.c but it spawns subprocess not threads and it works faster as far as I know, but I'd like to write my own thread pool.

Upvotes: 2

Views: 226

Answers (2)

Prof. Falken
Prof. Falken

Reputation: 24937

Python has no true multithreading, due to an implementation detail called the "GIL". Only one thread actually runs at a time, and Python switches between the threads. (Third party implementations of Python, such as Jython, can actually run parallel threads.)

As to why actually your program is slower in the multithreaded version depends, but when coding for Python, one needs to be aware of the GIL, so one does not believe that CPU bound loads are more efficiently processed by adding threads to the program.

Other things to be aware of are for instance multiprocessing and numpy for solving CPU bound loads, and PyEv (minimal) and Tornado (huge kitchen sink) for solving I/O bound loads.

Upvotes: 5

KayEss
KayEss

Reputation: 2300

You'll only see an increase in throughput with threads in Python if you have threads which are IO bound. If what you're doing is CPU bound then you won't see any throughput increase.

Turning on the thread support in Python (by starting another thread) also seems to make some things slower so you may find that overall performance still suffers.

This is all cpython of course, other Python implementations have different behaviour.

Upvotes: 4

Related Questions