CODEWITHSUNDEEP
pythonmultithreading
user3869692
user3869692

Reputation: 527

Python multi threading with queue

Here is my code;

import threading
from Queue import Queue



words = open("words.txt")


lock = threading.Lock()
threads = 10


q = Queue(maxsize=0)


def myfunction(q):
    lock.acquire()
    print q.get()
    lock.release()
    q.task_done()


for x in range(threads):
    m = threading.Thread(target=myfunction, args=(q,))
    m.setDaemon(True)
    m.start()

for word in words:
    q.put(word.strip())




q.join()

raw_input()

This will output:

word1
word2
word3
word4
word5
word6
word7
word8
word9
word10

Then it will stop. There are many more words in the file, how can I have it keep continuing? From my understanding, q.join() should wait until the queue is empty to add more.

I thought about putting it in a loop like this:

for word in words:
    q.put(word.strip())
    for x in range(threads):
        m = threading.Thread(target=myfunction, args=(q,))
        m.setDaemon(True)
        m.start()

But I sometimes get an error saying "cannot start new thread".

Upvotes: 0

Views: 183

Answers (1)

wim
wim

Reputation: 362707

Your threads just finish after they have processed one queue item.

Put the worker code (i.e. the body of myfunction) into a while: True loop so that it keeps on getting more items from the queue, rather than returning once q.task_done() was called.

Upvotes: 2

Related Questions