CODEWITHSUNDEEP
pythonxmltwistedfile-transfer
Spencer Rathbun
Spencer Rathbun

Reputation: 14910

Using twisted to process files

I'm trying to set up a twisted xmlrpc server, which will accept files from a client, process them, and return a file and result dictionary back.

I've used python before, but never the twisted libraries. For my purposes security is a non issue, and the ssh protocol seems like overkill. It also has problems on the windows server, since termios is not available.

So all of my research points to xmlrpc being the best way to accomplish this. However, there are two methods of file transfer available. Using the xml binary data method, or the http request method.

Files can be up to a few hundred megs either way, so which method should I use? Sample code is appreciated, since I could find no documentation for file transfers over xml with twisted.

Update:

So it seems that serializing the file with xmlrpclib.Binary does not work for large files, or I'm using it wrong. Test code below:

from twisted.web import xmlrpc, server

class Example(xmlrpc.XMLRPC):
    """
    An example object to be published.
    """

    def xmlrpc_echo(self, x):
        """
        Return all passed args.
        """
        return x

    def xmlrpc_add(self, a, b):
        """
        Return sum of arguments.
        """
        return a + b

    def xmlrpc_fault(self):
        """
        Raise a Fault indicating that the procedure should not be used.
        """
        raise xmlrpc.Fault(123, "The fault procedure is faulty.")

    def xmlrpc_write(self, f, location):
        with open(location, 'wb') as fd:
            fd.write(f.data)

if __name__ == '__main__':
    from twisted.internet import reactor
    r = Example(allowNone=True)
    reactor.listenTCP(7080, server.Site(r))
    reactor.run()

And the client code:

import xmlrpclib
s = xmlrpclib.Server('http://localhost:7080/')
with open('test.pdf', 'rb') as fd:
    f = xmlrpclib.Binary(fd.read())
s.write(f, 'output.pdf')

I get xmlrpclib.Fault: <Fault 8002: "Can't deserialize input: "> when I test this. Is it because the file is a pdf?

Upvotes: 2

Views: 698

Answers (1)

Jean-Paul Calderone
Jean-Paul Calderone

Reputation: 48335

XML-RPC is a poor choice for file transfers. XML-RPC requires the file content to be encoded in a way that XML supports. This is expensive in both runtime costs and network resources. Instead, try just POSTing or PUTing the file using plain old HTTP.

Upvotes: 1

Related Questions