Python thread won't start using socket

Question

i have a problem with my progam using socket and thread.

I have made a socket server who add client in a thread, but the client thread never start...

here is my code:

socket server

import socket, threading, logging, sys
from client_thread import ClientThread

class SocketServer:
    CLIENTS = list()

    def __init__(self, server_ip, server_port, max_connections):
        try:
            self.tcpsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            self.tcpsock.bind((server_ip, server_port))
            self.tcpsock.listen(10)

            logging.info('Socket server successfully started !')
    except Exception as e:
            logging.error(format(e))

    def start(self):
        from src.realmserver.core.hetwan import EmulatorState, Core

        while (Core.STATE == EmulatorState.IN_RUNNING):
            try:
                (clientsock, (ip, port)) = self.tcpsock.accept()
                new_client = threading.Thread(target=ClientThread, args=[len(self.CLIENTS), ip, port, clientsock])
                self.CLIENTS.append(new_client)
                new_client.start()
            except Exception as e:
                print format(e)

        for client in self.CLIENTS:
            client.join()

and client thread

import logging, string, random

class ClientThread:
    def __init__(self, client_id, client_ip, client_port, socket):
        self.client_id = client_id
        self.client_ip = client_ip
        self.client_port = client_port
        self.socket = socket
        logging.debug('(%d) Client join us !', client_id)

    def run(self):
        key = ''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase + string.digits) for _ in range(32))

        print self.send('HC%s' % key)

        while True:
            entry = self.socket.recv(4096)
            entry.replace("\n", "")

            if not entry:
                break
            else:
                logging.debug('(%d) Packet received : %s', self.client_id, str(entry))

        self.kill()

    def send(self, packet):
        return self.socket.send("%s\x00" % packet)

    def kill(self):
        self.socket.close()
        logging.debug('(%d) Client is gone...', self.client_id)

sorry for bad indentation, it's the form, not my file.

Please help me :(

Thank you in advance (sorry for bad english i'm french....)

Answer 1

You have this line of code in your Server instance start function:

new_client = threading.Thread(target=ClientThread,
     args=[len(self.CLIENTS), ip, port, clientsock])

The target= argument to threading.Thread needs to be a callable function. Here ClientThread is the name of the constructor function for your class ClientThread, so it is a callable function, returning an instance of that class. Note that it is not actually called yet! The args= argument is more normally a tuple, but a list actually works. These are the arguments that will be passed to the target function once it's eventually called, when you use this particular threading model. (You can also pass keyword arguments using kwargs= and a dictionary.)

What happens now is a bit tricky. Now that the two parameters (target= and args=) have been evaluated, the Python runtime creates a new instance of a threading.Thread class. This new instance is, at the moment, just a data object.

If we add a print statement/function (it's not clear whether this is py2k or py3k code) we can see the object itself:

print('new_client id is', id(new_client))

which will print something like:¹

new_client id is 34367605072

Next, you add this to a list and then invoke its start:

self.CLIENTS.append(new_client)
new_client.start()

The list add is straightforward enough, but the start is pretty tricky.

The start call itself actually creates a new OS/runtime thread (whose ID is not related to the data object's ID—the raw thread ID is an internal implementation detail). This new thread starts running at its run method.² The default run method is in fact:³

try:
    if self.__target:
        self.__target(*self.__args, **self.__kwargs)
finally:
    # Avoid a refcycle if the thread is running a function with
    # an argument that has a member that points to the thread.
    del self.__target, self.__args, self.__kwargs

Since you are using a regular threading.Thread instance object, you are getting this default behavior, where new_thread.start() creates the new thread itself, which then calls the default run method, which calls its self.__target which is your ClientThread class-instance-creation function.

So now, inside the new thread, Python creates an instance of a ClientThread object, calling its __init__ with the self.__args and self.__kwargs saved in the new_thread instance (which is itself shared between the original Python, and the new thread).

This new ClientThread object executes its __init__ code and returns. This is the equivalent of having the run method read:

def run(self):
    ClientThread(**saved_args)

Note that this is not:

def run(self):
    tmp = ClientThread(**saved_args)
    tmp.run()

That is, the run method of the ClientThread instance is never called. Only the run method of the threading.Thread instance is called. If you modify your ClientThread's __init__ method to print out its ID, you will see that this ID differs from that of the threading.Thread instance:

class ClientThread:
    def __init__(self, client_id, client_ip, client_port, socket):
        print('creating', id(self), 'instance')

which will print a different ID (and definitely print after the new_client id is line):

new_client id is 34367605072
creating 34367777464 instance

If you add additional prints to your run method you will see that it is never invoked.

What to do about this

You have two main options here.

• You can either make your ClientThread a subclass of threading.Thread:

  class ClientThread(threading.Thread):
      def __init__(self, client_id, client_ip, client_port, socket):
          ...
          threading.Thread.__init__(self)
  

  In this case, you would create the client object yourself, rather than using threading.Thread to create it:

  new_thread = ClientThread(...)
  ...
  new_thread.start()
  

  The .start method would be threading.Thread.start since you have not overridden that, and that method would then create the actual OS/runtime thread and then call your run method, which—since you did override it—would be your run.

• Or, you can create a standard threading.Thread object, supply it with a target, and have this target invoke your object's run method, e.g.:

  new_client = ClientThread(...)
  new_thread = threading.Thread(target=new_client.run, ...)
  ...
  new_thread.start()
  

  The choice is yours: to subclass, or to use separate objects.

---

¹The actual ID is highly implementation-dependent.

²The path by which it reaches this run function is somewhat convoluted, passing through bootstrap code that does some internal initialization, then calls self.run for you, passing no arguments. You are only promised that self.run gets entered somehow; you should not rely on the "how".

³At least, this is the code in Python 2.7 and 3.4; other implementations could vary slightly.