CODEWITHSUNDEEP
pythonbashshellsubprocessmininet
Toni
Toni

Reputation: 141

Passing a "shell script" as a string to Python subprocess.Popen

I want to execute a string as if it were a shell script in Mininet's host.popen() module, which is essentially a wrapper for Python's subprocess.Popen(). The script is as follows: 

#!/bin/bash

T="$(date +%s%N)" 
nc 10.0.0.7 1234 < somefile.txt
T="$(($(date +%s%N)-T))" 
echo $T

If I save it as a file and pass that into popen(), the output is as expected (prints the duration of the nc command): 

dst_cmd = 'nc -l 1234 > /dev/null'
dst.popen( dst_cmd, shell=True )

p = src.popen( ['sh', './timer.sh'], stdout=subprocess.PIPE )

The thing is that the file getting sent, somefile.txt, is different every time I run this script, and it is run several times a second for a few minutes. I'd prefer not to have to write a new .sh script for every file. When I try to pass the script into popen() as a string, like this, 

dst_cmd = 'nc -l 1234 > /dev/null'
dst.popen( dst_cmd, shell=True )

src_cmd = '''\
    #!/bin/bash

    T=\"$(date +%s%N)\" 
    nc 10.0.0.7 1234 < somefile.txt
    T=\"$(($(date +%s%N)-T))\" 
    echo $T'''

p = src.popen( dedent(src_cmd), shell=True, 
                stdout=subprocess.PIPE )

the output is 

Execution utility for Mininet

Usage: mnexec [-cdnp] [-a pid] [-g group] [-r rtprio] cmd args...
...

Why is that? Am I missing something about the formatting that's causing a different (unexpected) output?

Upvotes: 0

Views: 6553

Answers (4)

Serge Ballesta
Serge Ballesta

Reputation: 149195

Assuming that mininet popen interface is the same as Subprocess.popen, what you write is wrong. You must pass a command, that can be :

  • either an executable file and its parameters
  • or a shell command, provided you use shell=True - that is something you could type at a shell prompt

But it cannot be the content of a shell script

Of course, if the script can be seen as a multi-line command and if your shell supports multi-line commands, the individual commands will be executed. So if you default shell is already bash, it could work, but if the default shell is /bin/ash for example, all the commands will be executed by that default shell, because the line #!/bin/sh will be seen as a mere comment and ignored.

Example :

foo.py :

#! /usr/local/bin/python

a = 5
for i in range(10): print i

Execution of file is correct

$ sh -c ./foo.py
0
1
2
3
4

But execution of the content of the file causes an error because the shebang (#!) is seen as a mete comment :

$ sh -c '#! /usr/local/bin/python

a = 10
for i in range(10): print i
'
a: not found
Syntax error: "(" unexpected

Upvotes: 0

jfs
jfs

Reputation: 414915

To pass a bash script as a string, specify executable parameter:

#!/usr/bin/env python
import subprocess

bash_string = r'''#!/bin/bash
T="$(date +%s%N)" 
nc 10.0.0.7 1234 < somefile.txt
T="$(($(date +%s%N)-T))" 
echo $T
'''
output = subprocess.check_output(bash_string, shell=True, executable='/bin/bash')

Though you don't need neither the shell nor any other external process in this case. You could reimplement the shell script in pure Python using socket module:

#!/usr/bin/env python3
import socket
import sys
from shutil import copyfileobj
from timeit import default_timer as timer

start = timer()
with socket.create_connection(('10.0.0.7', 1234)) as s, \
     open('somefile.txt', 'rb') as input_file:
    s.sendfile(input_file) # send file
    with s.makefile() as f: # read response
        copyfileobj(f, sys.stdout)
print("It took %.2f seconds" % (timer() - start,))

Upvotes: 2

Dunes
Dunes

Reputation: 40903

It's best to avoid using the shell entirely. You can do exactly what you want more simply by:

from subprocess import Popen, DEVNULL
from time import time

start = time()
with open("somefile.txt", "rb") as f:
    p = Popen(["nc", "10.0.0.7", "1234"], stdin=f, stdout=DEVNULL)
end = time()

duration = end - start

If you're worried about the time of the spawning of the subprocess being significant then try:

from subprocess import Popen, DEVNULL, PIPE
from time import time
from os import devnull


with open("somefile.txt", "rb") as f:
    data = f.read()

p = Popen(["nc", "10.0.0.7", "1234"], stdin=PIPE, stdout=DEVNULL)
start = time()
p.communicate(data)
end = time()

duration = end - start

print(duration)

Upvotes: 1

Claude
Claude

Reputation: 9990

I think that what you want is pass the filename to the script right, that would do the trick?

p = src.popen( ['sh', './timer.sh', 'filename.txt'], stdout=subprocess.PIPE )

and in the script do

FILENAME="$1"
....
nc 10.0.0.7 1234 < "$FILENAME"

That should do the trick. Or possibly I'm completely misunderstanding the problem.

======

EDIT:

As an alternative (and a closer answer to the actual question), you could do:

cmd = """echo $(date) HELLO $(date)"""
p = subprocess.Popen(["sh", "-c", cmd])

sh -c tells shell to execute the next command as literal. Note that you don't need a shell=True, since you don't need any parsing of the code

EDIT 2:

That's what you get for answering too fast. You can indeed just pass a full shell script to the shell this way. Just don't do any escaping (and possibly get rid of the shebang):

cmd = """echo $(date) HELLO $(date);
   sleep 1;
   echo $(date)
"""
subprocess.Popen(cmd, shell=True)

Upvotes: 2

Related Questions