CODEWITHSUNDEEP
pythonpython-3.xsubprocess
cmouse
cmouse

Reputation: 672

Script incompatibility for Python 2.x and Python 3.x

I cannot understand what happens with python3 that prevents this from working, when I try with python3, it just hungs on line 11. 

import io,re,unittest,os,json,sys
from subprocess import PIPE, STDOUT, Popen
sub = Popen(["/usr/bin/python3.4", "test2.py", "pipe"], stdin=PIPE, stdout=PIPE, stderr=sys.stderr, close_fds=True, shell=False)
(writer,reader) = (sub.stdin, sub.stdout)
writer.write("HELO\t1\n".encode("utf-8"))
writer.flush()
sub.poll()
line = reader.readline().decode("utf-8")
assert(re.match("^OK\t", line))
writer.write("Q\ttest.com\tIN\tSOA\t-1\t127.0.0.1\n".encode("utf-8"))
line = reader.readline().decode("utf-8")
assert(re.match("^DATA\ttest.com\tIN\tSOA\t300\t-1\tsns.dns.icann.org. noc.dns.icann.org. 2013073082 7200 3600 1209600 3600", line))
sub.stdout.close()
sub.stdin.close()
sub.kill()
sub.wait()

Here's the test2.py file that the above code calls:

import sys
sys.stderr.write(sys.stdin.readline())
sys.stdout.write("OK\tversion foo bar hello\n")
sys.stdout.flush()
sys.stderr.write(sys.stdin.readline())
sys.stdout.write("DATA\ttest.com\tIN\tSOA\t300\t-1\tsns.dns.icann.org. noc.dns.icann.org. 2013073082 7200 3600 1209600 3600")
sys.stdout.flush()

Upvotes: 1

Views: 135

Answers (1)

Jonathan Eunice
Jonathan Eunice

Reputation: 22463

On my system, I reproduced your problem and fixed it by adding a writer.flush() after your second writer.write() call. E.g.:

import io,re,unittest,os,json,sys
from subprocess import PIPE, STDOUT, Popen
sub = Popen(["/usr/bin/python3.4", "test2.py", "pipe"], stdin=PIPE, stdout=PIPE, stderr=sys.stderr, close_fds=True, shell=False)
(writer,reader) = (sub.stdin, sub.stdout)
writer.write("HELO\t1\n".encode("utf-8"))
writer.flush()
sub.poll()
line = reader.readline().decode("utf-8")
assert(re.match("^OK\t", line))
writer.write("Q\ttest.com\tIN\tSOA\t-1\t127.0.0.1\n".encode("utf-8"))
writer.flush() # <<< INSERTED FLUSH <<<
line = reader.readline().decode("utf-8")
assert(re.match("^DATA\ttest.com\tIN\tSOA\t300\t-1\tsns.dns.icann.org. noc.dns.icann.org. 2013073082 7200 3600 1209600 3600", line))
sub.stdout.close()
sub.stdin.close()
sub.kill()
sub.wait()
sub.wait()

It now executes properly if the main script is run in Python 2 or Python 3 (tested: 2.7 and 3.4)

I say "on my system" because this sort of interleaved reader-writer coordination across text pipes is intrinsically fragile. When it works great, it's great! But very small variations in how the system, system libraries, language implementations, etc. handle I/O can cause you all manner of grief.

Upvotes: 1

Related Questions