CODEWITHSUNDEEP
pythonstdoutstdinflush
Marcin Skiba
Marcin Skiba

Reputation: 103

python stdout.flush() does NOT flush last white characters like '\n'

I have two programs. First one (subprocess.cpp) written in C++:

#include <stdio.h>

int main() {

    char * line = new char[1000];

    // first scan
    scanf("%s\n", line);
    printf("%s\n", line);
    fflush(stdout);

    // second scan
    scanf("%s\n", line);
    printf("%s\n", line);
    fflush(stdout);

    return 0;
}

This program simply gets two string from stdin and prints in on stdout, flushing after all. Here is the second program (test.py) written in Python 2.7:

from subprocess import Popen
from subprocess import PIPE
from subprocess import Popen

# create process
my_process = Popen("./subprocess", stdin = PIPE, stdout = PIPE)

# send initial data
my_process.stdin.write('abc\n')
my_process.stdin.flush()
my_process.stdin.write('xyz\n')
my_process.stdin.flush()

# read data from subprocess
print "Subprocess line 1 :: " + str(my_process.stdout.readline())
print "Subprocess line 2 :: " + str(my_process.stdout.readline())

This script should start the subproccess, send and retrieve two strings. Look what is happpening: 

marcin@marcin-Aspire-7540 ~/Desktop/inzynierka $ g++ subprocess.cpp -o subprocess
marcin@marcin-Aspire-7540 ~/Desktop/inzynierka $ python test.py 
Subprocess line 1 :: abc

test.py is waiting for the second line from program subprocess. Program subprocess cannot send the 2nd string because it's waiting for '\n' character.

If I change the second scanf in subprocess.cpp to scanf("%s\n", line); (no \n) everything works fine. Same happens when I send one additional line in test.py:

# send initial data
my_process.stdin.write('abc\n')
my_process.stdin.flush()
my_process.stdin.write('xyz\n')
my_process.stdin.flush()
my_process.stdin.write('ADDITIONAL\n')
my_process.stdin.flush()

It seems that Python does NOT send the last \n character after flushing (see example with changed scanf). Adding one additional write and flush in test.py proves, that missing \n character is still in the buffer after flush.

What to do with that? How to make Python's flush to flush ALL characters?

Upvotes: 0

Views: 915

Answers (2)

Alper
Alper

Reputation: 13220

The problem is with scanf("%s\n", line), the second scanf() need to read another line before it can find non-whitespace character.

There is a good explanation for similar case here.

I also checked it with strace, Python sends the last '\n' character.

pid 6611: test.py
pid 6613: subprocess

> strace -f python test.py
[pid  6611] write(4, "abc\n", 4 <unfinished ...>
...
[pid  6611] write(4, "abc\n", 4 <unfinished ...>
...
[pid  6613] read(0, "abc\nxyz\n", 4096) = 8
...
[pid  6613] write(1, "abc\n", 4)        = 4
[pid  6611] <... read resumed> "a", 1)  = 1
[pid  6613] read(0,  <unfinished ...>
[pid  6611] read(5, "b", 1)             = 1
[pid  6611] read(5, "c", 1)             = 1
[pid  6611] read(5, "\n", 1)            = 1
...
[pid  6611] write(1, "Subprocess line 1 :: abc\n", 25Subprocess line 1 :: abc) = 25
[pid  6611] write(1, "\n", 1)           = 1
[pid  6611] read(5,

Upvotes: 1

Noxitu
Noxitu

Reputation: 386

The fault is scanf behavior with \n i format strings. It seams that scanf("%s\n") will return after first non-whitespace character after \n while using pipes or after first non-empty line while using terminal (because first non-whitespace character will be flushed after next \n).

The point is that you probably didn't mean to use scanf("%s\n"). My guess is that what you really want is: 

scanf("%s%*[^\n]", line);

which will read word, then discard all characters in the same line.

Upvotes: 2

Related Questions