CODEWITHSUNDEEP
pythonwindowspipe
nik
nik

Reputation: 8765

Pipe out of a command or script to another python script

I'm relatively to to python and I'm trying to write a python script to which one can pipe the output of a command or another script.

example command | python_sript.py

In python script I'll basically analyze the output of command and save it to file.

I thought I'll be able to do this with redirecting sys.stdin to subprocess.PIPE, but it didn't work. 

sys.stdin = subprocess.PIPE

Can some one please suggest how should I approach this?

Also what would happen if command or script pipes data at faster rate then what python script can process.

Note: When I use this script

import sys
data = sys.stdin.readline()
print(data)

I get this

D:\>echo "test" | tee.py
The process tried to write to a nonexistent pipe.
Traceback (most recent call last):
  File "D:\Profiles\Administrator\My Documents\Workspace\tee.py", line 3, in <mo
dule>
    data = sys.stdin.readline()
AttributeError: 'NoneType' object has no attribute 'readline'

and when I use this

import sys
data = input()
print(data)

I get this

D:\>echo "test" | tee.py
The process tried to write to a nonexistent pipe.
Traceback (most recent call last):
  File "D:\Profiles\Administrator\My Documents\Workspace\tee.py", line 3, in <mo
dule>
    data = input()
RuntimeError: input(): lost sys.stdin

Upvotes: 3

Views: 5570

Answers (3)

jfs
jfs

Reputation: 414795

On (old) Windows when you use pipes you need to call python scripts using python executable by default due to NT redirection bug:

D:\> echo "test" | python tee.py

After that sys.stdin, raw_input(), fileinput.input() should work as expected.

Upvotes: 4

ThomasH
ThomasH

Reputation: 23536

Question 1 (reading from a pipe) has been answered by others: Just read stdin in python_script.py. The OS handles the redirection through pipes.

Question 2 (writing/reading speed) has also been answered: OS pipes are buffered, so nothing to be concerned about here.

Question 3 (the stack traces): Your first program runs fine with me. - Are you really giving the whole code here?

As for subprocess.PIPE: You really only want to use this, if you start a new command from within your running Python script, so you can communicate with the child process. As you are not doing this, it is of no use to you. Pipes on the shell level don't constitute parent-child processes.

Upvotes: 0

Katriel
Katriel

Reputation: 123772

You seem to be a bit unclear on how pipes work. They are handled by the OS, not the individual programs -- so if you write a Python script designed to take input data from a pipe, it should just read stdin as normal and the redirection will be handled for you. The data will be consumed as fast as they are generated; if the script consumes data more slowly than they are generated then they will be stored in a buffer.

Or are you trying to communicate between two Python scripts? If so there are better ways than through stdin and stdout.

Upvotes: 1

Related Questions