CODEWITHSUNDEEP
pythonpython-3.x
Ivan
Ivan

Reputation: 409

Converting file descriptor (pipe) to file object in python

I'd like to use anonymous pipes to communicate pickled objects between two processes in linux. I don't want to use the subprocess module and instead manually create the necessary pipes and then call pickle.dump and pickle.load. The last two functions, however, require a file object. So, while testing things around, I was faced with the following problem. The following code works fine:

import os
r, w = os.pipe()
os.write(w, b"test")
os.read(r, 1024)

but this code stalls at the read operation

import os
r, w = os.pipe()
fr = os.fdopen(r, "rb")
fw = os.fdopen(w, "wb")
fw.write(b"test")
fr.read()

Additionally, select.select([r],[],[],0) shows the pipe is empty. My question: what is going on during file object creation and why doesn't it work for pipes? Is there a way to get a file object this way at all?

Upvotes: 4

Views: 2089

Answers (1)

DYZ
DYZ

Reputation: 57033

You have a combination of two problems. First, a file object created by os.fdopen is buffered by default. Either make it unbuffered:

fr = os.fdopen(r, "rb", buffering=0)
fw = os.fdopen(w, "wb", buffering=0)

Or flush the written data:

fw.write(b"test")
fw.flush()

Second, function fr.read() when called without parameters reads to the end of file. In the case of a pipe, it means, until the pipe is closed. You should pass the number of bytes you want to read:

fr.read(4)
> b'test'

If you are not sure how much data will be written, read it piecewise, N bytes per call, and then recombine.

Upvotes: 2

Related Questions