CODEWITHSUNDEEP
pythonfilesocketssenddirectory
mont_
mont_

Reputation: 311

Python Client/Server send big size files

I am not sure if this topic have been answered or not, if was I am sorry:

I have a simple python script the sends all files in one folder:

Client:

import os,sys, socket, time
def Send(sok,data,end="292929"):
    sok.sendall(data + end);
def SendFolder(sok,folder):
    if(os.path.isdir(folder)):
        files = os.listdir(folder);
        os.chdir(folder);
        for file_ in files:
            Send(sok,file_);#Send the file name to the server
            f = open(file_, "rb");
            while(True):
                d = f.read();#read file
                if(d == ""):
                    break;
                Send(sok, d, "");#Send file data
            f.close();
            time.sleep(0.8);#Problem here!!!!!!!!!!!
            Send(sok,"");#Send termination to the server
            time.sleep(1);#Wait the server to write the file
        os.chdir("..");
        Send(sok,"endfile");#let the server know that we finish sending files
    else:
        Send("endfile")#If not folder send termination
try:
    sok1 = socket.socket();
    sok1.connect(("192.168.1.121",4444))#local ip
    time.sleep(1);
    while(True):
        Send(sok1,"Enter folder name to download: ");
        r = sok1.recv(1024);
        SendFolder(sok1,r);
        time.sleep(0.5);
except BaseException, e:
    print "Error: " + str(e);
    os._exit(1);

Server:

import sys,os,socket,time
# receive data
def Receive(sock, end="292929"):
    data = "";
    while(True):
        if(data.endswith(end)):
            break;
        else:
            data = sock.recv(1024);
    return data[:-len(end)];#return data less termination
def FolderReceive(sok):
    while(True):
        r = Receive(sok);# recv filename or folder termination("endfile")
        if(r == "endfolder"):
            print "Folder receive complete.";
            break;
        else:
            filename = r;#file name
            filedata = Receive(sok);# receive file data
            f = open(filename,"wb");
            f.write(filedata);
            f.close();#finish to write the file
            print "Received: " + filename;
try:
    sok1 = socket.socket();
    sok1.bind(("0.0.0.0",4444));
    sok1.listen(5);
    cl , addr = sok1.accept();#accepts connection
    while(True):
        r = Receive(cl);
        sys.stdout.write("\n" + r);
        next = raw_input();
        cl.sendall(next);#send folder name to the client
        FolderReceive(cl);
except BaseException, e:
    print "Error: " + str(e);
    os._exit(1);

I know this not best server ever...but is what I know. This just work for a folder with small files because if I send big files(like 5mb...) it crashes because the time the client waits for the server is not enough.

So my question is how can I send the files to the server without client need to wait??or know exactly how many time the client needs to wait for the server to receive the file?? Some code that does the same but handling any file size, any help?

Upvotes: 2

Views: 6525

Answers (1)

abarnert
abarnert

Reputation: 365767

TCP sockets are byte streams, not message streams. If you want to send a series of separate messages (like your separate files), you need to define a protocol, and then write a protocol handler. There is no way around that; just guessing at the timing or trying to take advantage of packet boundaries cannot possibly work.

The blog post linked above shows one way to do it. But you can do it with string delimiters if you want. But you have to deal with two problems:

  • The delimiter can appear anywhere in a read packet, not just at the end.
  • The delimiter can be split on packet boundaries—you may get "2929" at the end of one read, and the other "29" at the start of the next.

The usually way you do that is to accumulate a buffer, and search for the delimiter anywhere in the buffer. Something like this:

def message(sock, delimiter):
    buf = ''
    while True:
        data = sock.read(4096)
        if not data:
            # If the socket closes with no delimiter, this will
            # treat the last "partial file" as a complete file.
            # If that's not what you want, just return, or raise.
            yield buf
            return
        buf += data
        messages = buf.split(delimiter)
        for message in messages[:-1]:
            yield message
        buf = message[-1]

Meanwhile, you have another problem with your delimiter: There's nothing stopping it from appearing in the files you're trying to transmit. For example, what if you tried to send your script, or this web page?

That's one of the reasons that other protocols are often better than delimiters, but this isn't hard to deal with: just escape any delimiters found in the files. Since you're sending the whole file at once, you can just use replace on right before the sendall, and the reverse replace right before the split.

Upvotes: 3

Related Questions