CODEWITHSUNDEEP
pythoncommand-line-arguments
tpg2114
tpg2114

Reputation: 15110

Using command line arguments in python rather than shell

Assume a python script is callable with:

python myscript.py ./mycommand > myoutput.txt

How do I make sure that myscript.py gets ./mycommand > myoutput.txt in the list of command line arguments rather than the shell piping the output of python myscript.py ./mycommand to the file?

EDIT

Using quotes poses additional problems. Consider:

import subprocess
import sys

argList = sys.argv[1].split()

subprocess.call(argList)

which is then called by:

python myscript.py 'ls -l > ls.out'

results in

ls: cannot access >: No such file or directory
ls: cannot access ls.out: No such file or directory

The motivation is that only the output from the command I am trying to run should be saved in the file. Any other output from myscript.py to stdout shouldn't go to the file. It would appear that the pipe needs to be set with the stdout option to subprocess.call(). Is there a way to use the list of arguments directly so I don't have to parse it for things like >, 2>, 2&>, < etc?

Upvotes: 1

Views: 847

Answers (2)

Sean Vieira
Sean Vieira

Reputation: 160073

If you are willing to take on a dependency you could use Kenneth Reitz's envoy (you'll still need to use quotes if you want to pass arbitrary commands to your script from the command line).

from envoy import run
from sys import argv

run(argv[1])

And then you could invoke it using:

$ python myscript.py './mycommand > myoutput.txt'

Alternately, if you want shell-like syntax in your Python files you could use sh (formerly pbs) or plumbum:

from sh import ls

with open("myoutput.txt", "w") as outfile:
    ls("-l", _out=outfile)

# Or, in plumbum
from plumbum.cmd import ls

# Yes, this looks crazy, but the docs say it works.
ls["-l"] > "myoutput.txt"

Upvotes: 3

Matt Anderson
Matt Anderson

Reputation: 19809

A standard bash or tcsh shell will parse the command you give it itself, and interpret certain special characters as direction to the shell process itself (the ">" character in your example). The called program has no control over this; the command line parsing happens before your program is called.

In your example, you could say:

python myscript.py ./mycommand ">" myoutput.txt

And the quoted ">" will tell the shell not to use its special behavior.

The subprocess module has two modes of operation; one with a shell (typically /bin/sh) intermediate process -- this is when you give the Popen object a shell=True keyword argument. When you invoke it in this way, it invokes the shell to parse the argument text, and then the shell invokes the command you specified as a subprocess-once-removed from your program.

The other mode of operation, the default, is when shell=False. This creates a subprocess and populates its ARGV array directly without any intermediate shell trying to interpret the characters in your command.

The end result is, if you want to issue a command from a shell that interprets special characters, and include special characters, you have to escape them or quote them. If you're invoking a subprocess from the subprocess standard library module, you can pass in arbitrary text as the elements of the subprocess-to-be's ARGV, and not worry about them being interpreted in some special way by an intermediary shell process.

Upvotes: 2

Related Questions