Interactive stdin, stdout with subprocess

Question

I've read some discussions about this but I couldn't find anything on how to correct the error I have.

I have a C program, for a device, that is interactive, user input gives out hardware status, measurement, etc. The program returns the asked data. Initialization takes a few seconds so I can't afford to re-run the program each time I want a measurement, as some sub-routine needs a few thousand samples a second. I want my data to be exported into python somehow to make all my calculations and graph displays. I could adapt the C program but I'd rather change the python script to accommodate for it.

For testing purposes I have this simple C program:

#include <stdio.h>

int main(int argc, char const *argv[])
{
    int command;
    printf("Line1\n");
    printf("Line2\n");

    while(1){
        scanf("%d", &command);
        if (command == 0) break;
        else if (command == 1) printf("result : s_0 = 1, s_1 = 0.05, s_0 = 0.5, s_0 = -0.85\n");
        else printf("Unknown\n");
    }
    return 0;
}

And I have tried to run it inside a subprocess and use the stdin, stdout as communicate() ends the communication afterward :

import subprocess as sp
from subprocess import PIPE
import time

p = sp.Popen(["a.exe"], stdin=PIPE, stdout=PIPE)

# p.stdin.write(b'1\n') # Blocks here
print(p.stdout.readlines()) # Here too
p.stdout.flush()
p.stdin.write(b'0\n')
p.stdin.flush()
time.sleep(0.1)

print(p.stdout.readlines())

if not p.poll():p.kill()

I tried a lot of things from what I've seen over my research. Should I use a queue or a thread ? Why am I not getting anything out, it looks like I got deadlock since the first time I try to either write or read the pipe first. Python froze (not responsive to Ctrl+C but my keys are still put in the buffer as they appear after python is killed) and I have to kill the Python process each time.

Is there something better to accommodate for my case?

EDIT

As I don't want to create a new post that is similar I post my new, related issue, here. I implemented @Paul Cornelius solution and it worked. Hence I tried to make a command input to make the command transit throw python and go into the C program process :

p = sp.Popen(["a.exe"], stdin=PIPE, stdout=PIPE)

print("Line 1", p.stdout.readline())
print("Line 2", p.stdout.readline())

while(True):
    command = input("Command > ")
    p.stdin.write((command + '\n').encode())
    p.stdin.flush()
    if command == '0':
        break
    print("Answer : ", p.stdout.readline())

if not p.poll():
    p.kill()

I have one issue (maybe more but I can't know) and a question.

The first two lines appear, no issue here, but once I reach the input I can't type anything, I'd like to know if that is normal and what is the fix.

I changed the C program like @Paul Cornelius made it with the flush of the stdout buffer.

My question is about the first two Line, I tried using p.stdout.readlines() instead of p.stdout.readline() twice but it didn't work (freeze like in my original question). Why doesn't it work here but work at the end of the script? EOF character?

Answer 1

C programs buffer stdout. So you need to flush stdout after each write or else the Python program won't see anything:

#include <stdio.h>

int main(int argc, char const *argv[])
{
    int command;
    printf("Line1\n");
    printf("Line2\n");
    fflush(stdout);

    while(1){
        scanf("%d", &command);
        if (command == 0) break;
        else if (command == 1) printf("result : s_0 = 1, s_1 = 0.05, s_0 = 0.5, s_0 = -0.85\n");
        else printf("Unknown\n");
        fflush(stdout);
    }
    return 0;
}

In the Python program I rearranged a few things to get a better match to your C program logic. (Perhaps some of it was unnecessary.) The calls to p.stdin.flush are required. This program works:

import subprocess as sp
from subprocess import PIPE
import time

p = sp.Popen([r"c:\gcc64proj\so\so1.exe"], stdout=PIPE, stdin=PIPE)

print("Line 1", p.stdout.readline())
print("Line 2", p.stdout.readline())
p.stdin.write(b'1\n')
p.stdin.flush()
print("Line 3", p.stdout.readline())
p.stdin.write(b'2\n')
p.stdin.write(b'0\n')
p.stdin.flush()
time.sleep(0.1)

print(p.stdout.readlines())

if not p.poll():
    p.kill()