CODEWITHSUNDEEP
pythonpython-3.xpython-multithreading
shaike
shaike

Reputation: 59

Increment and print a variable every thread run() function gives weird results

i want to increment a counter variable using a count() function every time a thread starts run() function . The same thread calls a function from different module and prints out the counter value that should be assigned to each thread . Thing is i get a weird results when printing the variable , instead of getting a list of 1 , 2 , 3 ... i get the total number of threads that are currently running .

This is my threading module :

import threading
import proxies

threads_list = []
good_counter = 0
run_counter = 0
worker = None
num_threads=10
timeout=10

class Worker(threading.Thread):

    def __init__(self, timeout, proxy_list):
        threading.Thread.__init__(self)
        self.timeout = timeout
        self.proxy_list = proxy_list

    def run(self):
        global good_counter
        count()
        proxy_ip = proxies.get_proxy(proxies.get_proxylist())
        if proxies.is_proxy_good(proxy_ip):
            good_count()
            save_proxy(proxy_ip)
            print('[+] HIT ! - %s' % (proxy_ip))

def set_threads(num_threads, timeout, proxy_list):
    for i in range(num_threads):
        worker = Worker(timeout, proxies.get_proxylist())
        worker.setDaemon(True)
        worker.start()
        threads_list.append(worker)


def run_loop():
    while proxies.proxy_list.qsize() > 0:
        set_threads(num_threads, timeout, proxies.get_proxylist())
        for item in threads_list:
            item.join()
    print('[!] Proxylist Qsize < 0 QUITTING ....')


def get_counter():
    return run_counter

def count():
    global run_counter
    run_counter += 1

And this is is_proxy_good() method which is being used in proxies module and prints out the run_counter whenever there is an exception handling :

def is_proxy_good(proxy_ip):

try:
    r = requests.get('https://www.example.com',proxies=proxy_ip,timeout=15,headers=headers)
    if r.status_code is 200:
        return True
    return False

except requests.exceptions.Timeout:
    print('N%d - %s - Proxy Timeout\n' % (threads.get_counter(),proxy_ip))
    return False
except requests.exceptions.ProxyError:
    print('N%d - %s - Proxy ProxyError\n' % (threads.get_counter(),proxy_ip))
    return False
except requests.exceptions.SSLError:
    print('N%d - %s - Proxy SSLError\n' % (threads.get_counter(),proxy_ip))
    return False
except requests.exceptions.ConnectionError:
    print('N%d - %s - Proxy ConnectionError\n' % (threads.get_counter(),proxy_ip))
    return False

The output is :

N10 - {'https': 'https://x.xxx.xx.xxx:1080'} - Proxy ProxyError
N10 - {'https': 'https://x.xxx.xx.xxx:1080'} - Proxy ProxyError
N10 - {'https': 'https://x.xxx.xx.xxx:1080'} - Proxy ProxyError
..............
N20 - {'https': 'https://x.xxx.xx.xxx:1080'} - Proxy ProxyError
N20 - {'https': 'https://x.xxx.xx.xxx:1080'} - Proxy ProxyError
N20 - {'https': 'https://x.xxx.xx.xxx:1080'} - Proxy ProxyError
...........

Why it prints the total number of threads from the first thread ? How should i increment the correct way so it prints out ascending numbers incremented by 1 ? Thanks !

Upvotes: 0

Views: 329

Answers (1)

g.d.d.c
g.d.d.c

Reputation: 47968

In your code, run_counter is a global variable. By the time you ever use it, it's been modified by the creation of all of your threads. You need to store the value somewhere that's persistent to the instance of the Thread you're going to be using. I'd probably approach it with something like this:

class Worker(thread):
  _ids = count()

  def __init__(self):
    self.id = next(self._ids)

Then, somewhere in your proxies code you can do something like getCurrentThead().id.

Upvotes: 2

Related Questions