CODEWITHSUNDEEP
pythontcptwistedfile-transfer
scape
scape

Reputation: 707

twisted not sending entire file with twisted client and server tcp transfer

Edit: since I was appending via text the file was not being saved properly, I decided to rewrite the way I originally was hoping to and save the file as a stream instead: Twisted server: 

from twisted.internet import reactor, protocol
import os,json

class Echo(protocol.Protocol):
    f = file
    def dataReceived(self, data):
        try:
            try:
                print format(json.loads(data))
                print "got jason"
                self.f=open("test.png","wb")

                self.transport.write("ready")
            except:
                print "filedata incoming!"
                self.f.write(data)
        except:
            print "unknown error" #happens if we don't receive json first

    def connectionLost(self, reason):
        if self.f!=file:self.f.close()

def main():
    """This runs the protocol on port 8000"""
    factory = protocol.ServerFactory()
    factory.protocol = Echo
    reactor.listenTCP(8000,factory)
    reactor.run()

# this only runs if the module was *not* imported
if __name__ == '__main__':
    main()

original post below

Twisted sends like 99.9% of the file and then that seems to be it, I'm thinking I am writing the file incorrectly.

Twisted Server:

from twisted.internet import reactor, protocol
import os,json

class Echo(protocol.Protocol):

    def dataReceived(self, data):
        try:
            print format(json.loads(data))
            print "got jason"
            self.transport.write("ready")
        except:
            print "filedata incoming!"
            f = open("test.png","a")
            f.write(data)
            f.close()


def main():
    """This runs the protocol on port 8000"""
    factory = protocol.ServerFactory()
    factory.protocol = Echo
    reactor.listenTCP(8000,factory)
    reactor.run()

# this only runs if the module was *not* imported
if __name__ == '__main__':
    main()

Twisted Client:

from twisted.internet import reactor, protocol
import os,json

fname="pic.png"

class EchoClient(protocol.Protocol):
    """Once connected, send a message, then print the result."""

    def connectionMade(self):

        fsize = os.path.getsize(fname) 
        self.transport.write(json.dumps({"file":{"size":fsize}}))

    def sendFile(self):
        print "sending file" 
        f = open(fname,"rb")
        self.transport.write(f.read())
        f.close()
        print "closing conn"
        self.transport.loseConnection()

    def dataReceived(self, data):
        "As soon as any data is receive"
        print "Server said: ", data
        self.sendFile()


    def connectionLost(self, reason):
        print "connection lost"

class EchoFactory(protocol.ClientFactory):
    protocol = EchoClient

    def clientConnectionFailed(self, connector, reason):
        print "Connection failed - goodbye!"
        reactor.stop()

    def clientConnectionLost(self, connector, reason):
        print "Connection lost - goodbye!"
        reactor.stop()


# this connects the protocol to a server runing on port 8000
def main():
    f = EchoFactory()
    reactor.connectTCP("localhost", 8000, f)
    reactor.run()

# this only runs if the module was *not* imported
if __name__ == '__main__':
    main()

Basically the server is running and listening, the client connects and immediately sends json, server receives packet and tells send client the 'ok', the client then sends the file; then the server receives the file and writes it to disk. I'm just testing things out so this program might not make much sense, especially the use of file append-- but I noticed that after the transfer and final write the file is just about the same size as the original, but not quite and is smaller by about 300 bytes and therefore just about useless. Am I sending the file incorrectly? Or just writing it incorrectly? Oh yea, I'm testing the server and client on the same computer.

Ultimately I plan on sending files as large as 1GB to and from two local computers for backup purposes and want the files to be written as a stream of data, I don't like the append method I'm using but I don't know how to reference the file object without actually opening the file first, and that's something I only want to do when I first receive the json object.

Thanks!

Upvotes: 2

Views: 2536

Answers (2)

Glyph
Glyph

Reputation: 31860

The problem is that you're expecting all your data to be received at once by dataReceived. That's not how the Internet works: see this Twisted FAQ for an explanation of why this is so and how to fix your code.

Upvotes: 0

MRAB
MRAB

Reputation: 20654

You're opening "test.png" for appending text. Is that intentional?

You also have a bare except, which is a bad idea because it catches all exceptions. Catch only those exceptions you expect.

Upvotes: 1

Related Questions