Socket in Python3, listening port

Question

I want to listen port in Python3.

import socket

sock = socket.socket()
sock.bind(('', 9090))
sock.listen(1)
conn, addr = sock.accept()

print 'connected:', addr

while True:
    data = conn.recv(1024)
    if not data:
        break
    conn.send(data.upper())

conn.close()

The data which I want to listen, looks like:

8,0,0,test,0,2016/07/19,14:40:57.938,2016/07/19,14:40:57.938,,,,,,,,,,,,0
8,0,0,test,0,2016/07/19,14:40:57.965,2016/07/19,14:40:57.965,,,,,,,,,,,,0
3,0,0,test,0,2016/07/19,14:41:04.687,2016/07/19,14:41:04.687,,2475,,,,,,,,,,0
..

that is I need read until '\n' So I need to change this block, but I dont know how..

data = conn.recv(1024)
if not data:
    break
conn.send(data.upper())

I want replace nc:

nc -k -l -p 30003 | python3 script.py

where script.py

while True:
    for string in sys.stdin:

Also I need reconnect if something wrong, server must be ready take all data any time, just like nc -k -l -p 30003 | python3 script.py

Answer 1

The main idea is to read until you find \n character in your stream. Of course \n may be beyond 1024 bytes that you are reading thus you need to store everything you read in a buffer. This can be emulated with for example such class:

class SocketLineReader:
    def __init__(self, socket):
        self.socket = socket
        self._buffer = b''

    def readline(self):
        pre, separator, post = self._buffer.partition(b'\n')
        if separator:
            self._buffer = post
            return pre + separator

        while True:
            data = self.socket.recv(1024)
            if not data:
                return None

            pre, separator, post = data.partition(b'\n')
            if not separator:
                self._buffer += data
            else:
                data = self._buffer + pre + separator
                self._buffer = post
                return data

And usage:

import socket

sock = socket.socket()
sock.bind(('', 9090))
sock.listen(1)

conn, addr = sock.accept()

print('connected:', addr)

reader = SocketLineReader(conn)
while True:
    data = reader.readline()
    print(data)
    if not data:
        break
    conn.send(data.upper())

conn.close()

---

If you wish for the server to serve data forever use another while loop:

import socket

sock = socket.socket()
sock.bind(('', 9090))
sock.listen(1)

while True:
    conn, addr = sock.accept()
    print('connected:', addr)
    reader = SocketLineReader(conn)
    # The other code goes here

The problem with this approach is that there is no parallelism. Your server won't handle parallel connections. One way to fix that is to send each new connection to a separate thread:

import socket
import threading

def handle(conn):
    print('connected:', addr)
    reader = SocketLineReader(conn)
    # The other code goes here

sock = socket.socket()
sock.bind(('', 9090))
sock.listen(1)

while True:
    conn, addr = sock.accept()
    threading.Thread(target=handle, args=(conn,)).start()

This should be fine until you hit performance limit. There are ways to improve efficiency (e.g. event loops) but I supposes it's beyond this question.