Continous receiving & sending on a socket

Question

I'm working on assignment where I need to connect to a server (no details of server are disclosed), capture the reply, modify it and send it back for verification.

I have created following code which does what I need, but the issue is that after 1st correct reply, server sends another.

Code:

# -*- encoding: utf-8 -*-

import socket
from decryptmsg import decryptmsg
from cleanmsg import cleanmsg

#connection 
ip="<IP>"
port=4000
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip,port))

def recvall(sock):
    BUFFER = 8192
    data = b''
    while True:
        part = sock.recv(BUFFER)
        data += part
        if len(part) < BUFFER:
            break
    return data


while True:

    print "[+] start communication"
    data = recvall(sock)
    print data
    data = cleanmsg(data)
    if data != None:
        valmis = decryptmsg(str(data))
        if valmis == None:
            print "[-] no results"
            break
        else:
            print "[+] sending message... "
            sock.send(valmis) 
            continue

When I hit the second question, I get the input captured fine with this code and processed as expected, but when I try to send the 2nd reply back I get error:

Traceback (most recent call last):
File "challenge.py", line 28, in <module>
     sock.send(valmis)
socket.error: [Errno 32] Broken pipe

If I do not close or shutdown the socket, no reply is ever sent to server.

How can I tell my client to send the message and wait for reply without socket.shutdown? Or if I need to open new socket for each loop, how should the loop be constructed? The reply from server changes each time so if I open new connection completely and request for data, I get new reply and the process starts from beginning again.

UPDATE: the issue seems to be when trying to receive the second reply from server, only the first line of message is received by client.

Answer 1

How do you know it does not send anything? I modified your code a bit (there is something odd in the else: clause, I will come back to that later).

import socket

#connection 
ip="localhost"
port=4000
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip,port))

while True:

    data = sock.recv(8192)
    if not data: break;
    print data
    if data != None:
        valmis = data
        if valmis == None:
            print "[-] no results"
            break
        else:
            print "[+] sending message... "
            sock.send(valmis) #this never gets sent without
            continue

Basically this is a stripped version of your code - no decrypting or external functionality. It just sends back whatever it receives from the server.

Then I ran a "server" with ncat:

ncat -l 4000

start your program and start typing in lines (1, 2, 3, 4 etc) and this happens at "server". The client promptly echoes my messages back:

test@xyzzy:/tmp$ ncat -l 4000
1
1
2
2
3
3

And this happens at the client:

test@xyzzy:/tmp$ python so.py 
1

[+] sending message... 
2

[+] sending message... 
3

[+] sending message... 

To me it looks this code works fine. If the server does not receive your reply, it might be that there is a problem on the server side. It might for example expect a terminator character in the response. Does your cleanmsg clean the message too much and for example remove a trailing newline and the server expects to receive one?

There is a problem in your original else clause as you do another sock.recv() there. Which means after receiving a reply, you block there to wait for the next message from server and when you do receive one, you will continue your loop and hit sock.recv() again. The second message was consumed already in your else clause.

This may be intentional if your server somehow acknowledges your decryption. If your protocol is this:

server -> client  (request)
client -> server  (decrypted message)
server -> client  (some kind of acknowledgement - unclear from your code)

server -> client  (request 2)
etc. 

Then you have probably hit the issue in Jason's comment. TCP sockets are completely agnostic to the concept of a message. They just transmit data. When your code hits sock.recv(), one of five things can happen:

1. There is nothing in the socket and the call blocks
2. There is a full "message" and only that in the socket and you receive that
3. There is a partial message and you will receive that. Either because the message is over 8192 bytes, or your code just decides to read when the server has only transmitted some of the message data.
4. There are two or more complete "messages" waiting and you will receive them all.
5. As four, but the last message is partial

Always when operating with TCP sockets, you must cater for scenarios 2-5. You must parse the data, make sure everything is there, and if not, wait for more. And if there was more than you expected, process them accordingly. If they are complete messages, process them. If the last message is partial, process everything else and wait for more.

If messages seem to "disappear" in a self-made communication protocol using TCP sockets, 99% of the problems are caused by making the assumption that sockets would know or care what is the structure of your "message". A very common mistake is to read your socket empty and ignore everything you received after your first message.

Hope this is helpful. Your code without the additional recv seems to work fine from socket communication perspective - no need to shut down the socket. It may be a server side issue, protocol issue or message parsing problem.

For this reason, always have only one recv call for your socket. Even if you are expecting some kind of an acknowledgement instead of a new message, have only one place where you process socket data. Then do some kind of a conditional there to detect what kind of a message you received and then decide what to do with it.