How to handle multithreading with sockets in Python?

Question

I am currently working on a problem where I need to have a server handle multiple clients at the same time. I have a server.py file and a client.py file. Here is a snippet from server.py:

connections = []
addresses = []

for c in connections:
    c.close()
del connections[:]
del addresses[:]  #clears anything pre-existing

while True:
    try:
        csock, address = s.accept() #takes in connection and stores info
        s.setblocking(1) #prevents timeout
        connections.append(csock)
        addresses.append(address)
        print(f"Connection from {address} has been established!")
    except:
           print("Error has occurred")

When I run my server in terminal and then connect to it with a client. It behaves as I would expect, with it printing out Connection from ('192.168.1.84', 50824) has been established!.

When I open up another terminal and run client.py to make an additional connection, nothing happens. That is, until I close out of my first client process and then the server prints out

Error occurred
Connection from ('192.168.1.84', 50826) has been established!

I can kind of see what is happening here, but I'm very new to networking and I'm not super great at multithreading, so could anyone give me some insight as to what's going on and what I can do to make these processes run simultaneously as I would expect?

Answer 1

After s.accept() you should use threading to run code in separated thread - and this thread should continue connection with client. At the same time main thread may go back to s.accept() to wait for next client.

---

Minimal working code with some extra settings.

Server:

import socket
import threading
import time

# --- functions ---

def handle_client(conn, addr):
    print("[thread] starting")

    # recv message
    message = conn.recv(1024)
    message = message.decode()
    print("[thread] client:", addr, 'recv:', message)
    
    # simulate longer work
    time.sleep(5)

    # send answer
    message = "Bye!"
    message = message.encode()
    conn.send(message)
    print("[thread] client:", addr, 'send:', message)
    
    conn.close()

    print("[thread] ending")
   
# --- main ---

host = '0.0.0.0'
port = 8080

s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)  # solution for "[Error 89] Address already in use". Use before bind()
s.bind((host, port))
s.listen(1)

all_threads = []

try:
    while True:
        print("Waiting for client")
        conn, addr = s.accept()
    
        print("Client:", addr)
        
        t = threading.Thread(target=handle_client, args=(conn, addr))
        t.start()
    
        all_threads.append(t)
except KeyboardInterrupt:
    print("Stopped by Ctrl+C")
finally:
    if s:
        s.close()
    for t in all_threads:
        t.join()
    

Client (for test)

import socket

# --- main ---

host = '0.0.0.0'
port = 8080

s = socket.socket()
s.connect((host, port))

print("Connected to the server")

message = "Hello"
print('send:', message)
message = message.encode()
s.send(message)

message = s.recv(1024)
message = message.decode()
print('recv:', message)