Python appending file remotely

Question

It seems easy enough in python to append data to an existing file (locally), although not so easy to do it remotely (at least that I've found). Is there some straight forward method for accomplishing this?

I tried using:

import subprocess

cmd = ['ssh', 'user@example.com',
       'cat - > /path/to/file/append.txt']

p = subprocess.Popen(cmd, stdin=subprocess.PIPE)

inmem_data = 'foobar\n'

for chunk_ix in range(0, len(inmem_data), 1024):
    chunk = inmem_data[chunk_ix:chunk_ix + 1024]
    p.stdin.write(chunk)

But maybe that's not the way to do it; so I tried posting a query:

import urllib
import urllib2

query_args = { 'q':'query string', 'foo':'bar' }

request = urllib2.Request('http://example.com:8080/')
print 'Request method before data:', request.get_method()

request.add_data(urllib.urlencode(query_args))
print 'Request method after data :', request.get_method()
request.add_header('User-agent', 'PyMOTW (http://example.com/)')

print
print 'OUTGOING DATA:'
print request.get_data()

print
print 'SERVER RESPONSE:'
print urllib2.urlopen(request).read()

But I get connection refused, so I would obviously need some type of form handler, which unfortunately I have no knowledge about. Is there recommended way to accomplish this? Thanks.

Answer 1

At the time of posting this, the first script above (posted by @Juan Fco. Roco) didn't work for me. What worked for me instead is as follows:

from fabric import Connection

my_host                     = '127.0.0.1'
my_username                 = "jfroco"
my_password                 = '*********'
remote_file_path            = "/home/jfroco/development/fabric1/test.txt"
local_file_path             = "local.txt"

ssh_conn = Connection(host=my_host,
                      user=my_username,
                      connect_kwargs={"password": my_password}
                      )

with ssh_conn as my_ssh_conn:
    local_log_file_obj = open(local_file_path, 'ab', encoding="utf_8")

    my_ssh_conn.get(remote_file_path, local_log_file_obj)

    local_log_file_obj.close()

The main difference is 'ab' (append in binary mode) instead of 'a'.

Answer 2

If I understands correctly you are trying to append a remote file to a local file...

I'd recommend to use fabric... http://www.fabfile.org/

I've tried this with text files and it works great.

Remember to install fabric before running the script:

pip install fabric

Append a remote file to a local file (I think it's self explanatory):

from fabric.api import (cd, env)
from fabric.operations import get

env.host_string = "127.0.0.1:2222"
env.user = "jfroco"
env.password = "********"

remote_path = "/home/jfroco/development/fabric1"
remote_file = "test.txt"
local_file = "local.txt"

lf = open(local_file, "a")

with cd(remote_path):
    get(remote_file, lf)

lf.close()

Run it as any python file (it is not necessary to use "fab" application)

Hope this helps

EDIT: New script that write a variable at the end of a remote file:

Again, it is super simple using Fabric

from fabric.api import (cd, env, run)
from time import time

env.host_string = "127.0.0.1:2222"
env.user = "jfroco"
env.password = "*********"

remote_path = "/home/jfroco/development/fabric1"
remote_file = "test.txt"

variable = "My time is %s" % time()

with cd(remote_path):
    run("echo '%s' >> %s" % (variable, remote_file))

In the example I use time.time() but could be anything.