CODEWITHSUNDEEP
pythonlinuxmultithreading
user3383192
user3383192

Reputation: 43

How do I modify a list inside a thread?

I need to modify a list by adding some elements at the end from a thread.

This is my code:

def go():
  while queueCommand.qsize() > 0:
      command = queueCommand.get(False)
      res = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=None, shell=True)
      output = res.communicate()
      output = str(output)
      star = output.find("address") + 8
      en = output.find(",")
      ip = output[star:en-3]  
      alist.append(ip)  #<================== her i use the liste

if __name__ == '__main__':      
  with open ("icq.txt","r") as myfile:
    text = myfile.read()
  end = 0
  alist = []
  queueCommand = Queue.Queue()
  while True:
    start = text.find("href=") + 13
    if start == 12:
      break
    end = text.find("/",start)
    if text[start:end].find("icq.com") != -1:
      hostname="host "+ text[start:end]
      queueCommand.put(hostname) 
    text = text[end:len(text)]  

  for i in range(10):   
    threading.Thread(target=go,args=(alist)).start() #<====== i give the list as argument

  print alist

The last print statement display an empty list, []. Any ideas?

Upvotes: 1

Views: 2395

Answers (1)

sberry
sberry

Reputation: 132138

You have a few problems.

  1. You specify alist as the args, but you need to pass it as a tuple which it looks like you tried to do, but a one item tuple would look like this (alist,). Right now you are just using the alist global which is likely not what you want.

  2. Your go method does not expect an argument (namely, the alist).

  3. To be thread safe, I believe you will need to use some sort of semaphore/mutex/lock primitive. The threading module comes with a Lock implementation which you can use to restrict access to alist during the append operation.

  4. Most importantly, you are not waiting for your threads to finish before you print the result. To wait for thread to finish you need to call .join() on the thread.

I would probably opt for using another Queue instance to put the results into, then you could read all from the queue to construct your list once the threads are complete.

Here is an updated version (working) of your code. Like I said, I would probably opt to use a Queue instead, and I have not used the threading module much since I switched to eventlet/gevent... so there may be ways to improve on what I provided.

import threading
import Queue
import subprocess

lock = threading.Lock()

def go(alist):
  while queueCommand.qsize() > 0:
      command = queueCommand.get(False)
      res = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=None, shell=True)
      output = res.communicate()
      output = str(output)
      star = output.find("address") + 8
      en = output.find(",")
      ip = output[star:en-3]
      lock.acquire()
      alist.append(ip)  #<================== her i use the liste
      lock.release()

def foo(alist):
    alist.append("bar")

if __name__ == '__main__':
  with open ("icq.txt","r") as myfile:
    text = myfile.read()
  end = 0
  alist = []
  queueCommand = Queue.Queue()
  while True:
    start = text.find("href=") + 13
    if start == 12:
      break
    end = text.find("/",start)
    if text[start:end].find("icq.com") != -1:
      hostname="host "+ text[start:end]
      queueCommand.put(hostname)
    text = text[end:len(text)]

  threads = []
  for i in range(10):
    thread = threading.Thread(target=go,args=(alist,)) #<====== i give the list as argument)
    thread.start()
    threads.append(thread)
  for thread in threads:
    thread.join()

  print alist

Upvotes: 2

Related Questions