How to read from stdin or from a file if no data is piped in Python?

Question

I have a CLI script and want it to read data from a file. It should be able to read it in two ways :

• cat data.txt | ./my_script.py
• ./my_script.py data.txt

—a bit like grep, for example.

What I know:

• sys.argv and optparse let me read any args and options easily.
• sys.stdin let me read data piped in
• fileinput make the full process automatic

Unfortunately:

• using fileinput uses stdin and any args as input. So I can't use options that are not filenames as it tries to open them.
• sys.stdin.readlines() works fine, but if I don't pipe any data, it hangs until I enter Ctrl + D
• I don't know how to implement "if nothing in stdin, read from a file in args" because stdin is always True in a boolean context.

I'd like a portable way to do this if possible.

Answer 1

Process your non-filename arguments however you'd like, so you wind up with an array of non-option arguments, then pass that array as the parameter to fileinput.input():

import fileinput
for line in fileinput.input(remaining_args):
    process(line)

Answer 2

I'm a noob, so this might not be a good answer, but I'm trying to do the same thing (allow one or more files on the command line, default to STDIN otherwise).

The final combo I put together:

parser = argparse.ArgumentParser()
parser.add_argument("infiles", nargs="*")
args = parser.parse_args()

for line in fileinput.input(args.infiles):
    process(line)

This seems like the only way to get all the desired behavior in one elegant package, without requiring named args. Just like unix commands are used as such:

cat file1 file2
wc -l < file1

Not:

cat --file file1 --file file2

Would appreciate feedback/confirmation from veteran idiomatic Pythonistas to make sure I've got the best answer. Haven't seen this complete solution mentioned anywhere else, just fragments.

Answer 3

You can use this function to detect if the input is from a pipeline or not.

sys.stdin.isatty()

It returns false if the input is from pipeline or true otherwise.

Answer 4

For unix/linux you can detect whether data is being piped in by looking at os.isatty(0)

$ date | python -c "import os;print os.isatty(0)"
False
$ python -c "import os;print os.isatty(0)"
True

I'm not sure there is an equivalent for Windows.

edit Ok, I tried it with python2.6 on windows XP

C:\Python26>echo "hello" | python.exe -c "import os;print os.isatty(0)"  
False

C:\Python26> python.exe -c "import os;print os.isatty(0)"  
True

So maybe it it not all hopeless for windows

Answer 5

Argparse allows this to be done in a fairly easy manner, and you really should be using it instead of optparse unless you have compatibility issues.

The code would go something like this:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--input', type = argparse.FileType('r'), default = '-')

Now you have a parser that will parse your command line arguments, use a file if it sees one, or use standard input if it doesn't.

Answer 6

There is no reliable way to detect if sys.stdin is connected to anything, nor is it appropriate do so (e.g., the user wants to paste the data in). Detect the presence of a filename as an argument, and use stdin if none is found.