Python and Windows Named Pipes

Question

What is the proper way of communicating with named pipes on Windows from Python? I've googled it, and can't find any packages that wrap this communication.

There are:

• some descriptions of how to do it with pywin32 (I could not find how to connect to an existing pipe with it, though).
• This package: https://pypi.python.org/pypi/PyWPipe/ (had no luck with it either)
• A piece of code here, that I haven't yet tested: https://jonathonreinhart.com/posts/blog/2012/12/13/named-pipes-between-c-and-python/

I need just to connect to an existing named pipe and read/write to it. I previously had only tried communication with serial port (using pySerial), and I'm surprised how little info I could find on named pipes in comparison to it. There's usually tons of guides for any purpose for Python.

I'll appreciate any help.

Answer 1

A little late to the party, but if anybody is interested, I published namedpipe package on PyPI to enable named pipes in both Posix and Windows. Here is a link to the GitHub repo.

Its API is modeled after built-in open() and IO classes. A quick usage sample:

import subproces as sp
from namedpipe import NPopen

with NPopen('r+') as pipe: # bidirectional (duplex) binary pipe
    # - for an inbound pipe, specify the read access mode 'rb' or 'rt' 
    # - for an outbound pipe, specify the write access mode 'wb or 'wt'

    sp.run(['my_client', pipe.path])

    stream = pipe.wait() # Wait for the client to connect and create a stream

    b = stream.read(64)      # read 64 bytes from the client
    in_bytes = bytearray(128)
    nread = stream.readinto(in_bytes) # read 128 bytes of data from client
    b_rest = stream.readall() # read all bytes sent by the client, block
    stream.write(out_bytes) # send bytes in out_bytes to the client

Note: I wrote it a few years ago to work with the ffmpegio but I haven't yet encountered a use-case myself ironically. So, I hope someone can make a good use of it (although it is very much in a beta stage).

Answer 2

I had the same issue for using pipes in Audacity v3.4.2 on Windows.

Eventually, I created a script that tests Audacity's pipe for both Windows and macOS (I haven't tested on Linux).

""""
Source - https://github.com/unfor19/audacity-scripting

Instructions:
1. Start Audacity
2. Enable mod-script-pipe - https://manual.audacityteam.org/man/scripting.html
3. Restart Audacity
4. (Windows) Install pywin32 - `pip install pywin32`
4. (Windows) Download "pipelist" - https://learn.microsoft.com/en-us/sysinternals/downloads/pipelist
5. (Windows) Run "pipelist" to make sure Audacity's pipes are available - "FromSrvPipe" and "ToSrvPipe"
6. Run this script to send a command to Audacity and get the response

Usage:
   python audacity_pipetest.py
"""

import time
import sys
import os

if sys.platform == 'win32':
    import win32pipe
    import win32file
else:
    # No need to import anything for macOS or Linux
    pass


def send_command(TOFILE, EOL, command):
    """Send a single command."""
    time.sleep(0.5)
    full_command = command + EOL
    print(f"Send: >>> '{full_command}'")
    TOFILE.write(full_command)
    print("TOFILE Written")
    TOFILE.flush()
    print("TOFILE Flushed")


def get_response(FROMFILE):
    """Return the command response."""
    result = ''
    line = ''
    while True:
        result += line
        line = FROMFILE.readline()
        if line == '\n' and len(result) > 0:
            break
    print(f"Result: {result}")
    return result


def main():
    # Initialize variables for Windows and macOS/Linux
    # Pipe names and EOL is set according to - https://manual.audacityteam.org/man/scripting.html
    pipe_name_send = ''
    pipe_name_from = ''
    pipe_send = None  # For Windows only
    EOL = ''

    if sys.platform == 'win32':
        pipe_name_send = r'\\.\pipe\ToSrvPipe'
        pipe_name_from = r'\\.\pipe\FromSrvPipe'
        EOL = '\r\n\0'
        pipe_send = win32pipe.CreateNamedPipe(
            pipe_name_send,
            win32pipe.PIPE_ACCESS_DUPLEX,
            win32pipe.PIPE_TYPE_MESSAGE | win32pipe.PIPE_READMODE_MESSAGE | win32pipe.PIPE_WAIT,
            1, 65536, 65536,
            0,
            None)
    else:
        # macOS or Linux
        pipe_name_send = '/tmp/audacity_script_pipe.to.' + str(os.getuid())
        pipe_name_from = '/tmp/audacity_script_pipe.from.' + str(os.getuid())
        EOL = '\n'
    print(f"Trying to access pipe {pipe_send}")
    try:
        # Set command to send to Audacity
        # According to - https://manual.audacityteam.org/man/scripting_reference.html
        CMD = f'GetInfo: Preferences'  # Sample command

        # Open file buffer in write according to the platform
        print(f"Accessing send pipe - '{pipe_name_send}' ...")
        with open(pipe_name_send, 'w') as fp:
            print("Accessed send pipe")
            # Send command to Audacity using the write pipe
            send_command(fp, EOL, CMD)

        print(f"Accessing from pipe - '{pipe_name_from}' ...")
        # Open file buffer in text mode - must set encoding as Windows uses cp1252 by default
        with open(pipe_name_from, 'rt', encoding='utf-8') as fp:
            print("Accessed from pipe")
            # Get response from Audacity using the read pipe
            response = get_response(fp)
            print(f"Response:\n{response}")
    except Exception as e:
        raise e
    finally:
        if sys.platform == 'win32':
            win32file.CloseHandle(pipe_send)


if __name__ == '__main__':
    main()

Source - https://github.com/unfor19/audacity-scripting/blob/feature/initial-test/scripts/audacity_pipetest.py

I tested it locally on Windows 11 Pro 22H2 and Python 3.9.13; I intend to make it work in GitHub Actions windows-* instances and will update here once it's done.

Answer 3

In order to connect to an existing named pipe you can utilize the CreateFile API provided through the pywin32 package. Since it took me a while to put a working base together here is an example client/server which works fine for me (python 3.6.5, pywin32 223 on Windows 10 Pro x64):

import time
import sys
import win32pipe, win32file, pywintypes


def pipe_server():
    print("pipe server")
    count = 0
    pipe = win32pipe.CreateNamedPipe(
        r'\\.\pipe\Foo',
        win32pipe.PIPE_ACCESS_DUPLEX,
        win32pipe.PIPE_TYPE_MESSAGE | win32pipe.PIPE_READMODE_MESSAGE | win32pipe.PIPE_WAIT,
        1, 65536, 65536,
        0,
        None)
    try:
        print("waiting for client")
        win32pipe.ConnectNamedPipe(pipe, None)
        print("got client")

        while count < 10:
            print(f"writing message {count}")
            # convert to bytes
            some_data = str.encode(f"{count}")
            win32file.WriteFile(pipe, some_data)
            time.sleep(1)
            count += 1

        print("finished now")
    finally:
        win32file.CloseHandle(pipe)


def pipe_client():
    print("pipe client")
    quit = False

    while not quit:
        try:
            handle = win32file.CreateFile(
                r'\\.\pipe\Foo',
                win32file.GENERIC_READ | win32file.GENERIC_WRITE,
                0,
                None,
                win32file.OPEN_EXISTING,
                0,
                None
            )
            res = win32pipe.SetNamedPipeHandleState(handle, win32pipe.PIPE_READMODE_MESSAGE, None, None)
            if res == 0:
                print(f"SetNamedPipeHandleState return code: {res}")
            while True:
                resp = win32file.ReadFile(handle, 64*1024)
                print(f"message: {resp}")
        except pywintypes.error as e:
            if e.args[0] == 2:
                print("no pipe, trying again in a sec")
                time.sleep(1)
            elif e.args[0] == 109:
                print("broken pipe, bye bye")
                quit = True


if __name__ == '__main__':
    if len(sys.argv) < 2:
        print("need s or c as argument")
    elif sys.argv[1] == "s":
        pipe_server()
    elif sys.argv[1] == "c":
        pipe_client()
    else:
        print(f"no can do: {sys.argv[1]}")

Example output client

> python pipe_test.py c
pipe client
no pipe, trying again in a sec
no pipe, trying again in a sec
no pipe, trying again in a sec
message: (0, b'0')
message: (0, b'1')
message: (0, b'2')
message: (0, b'3')
message: (0, b'4')
message: (0, b'5')
message: (0, b'6')
message: (0, b'7')
message: (0, b'8')
message: (0, b'9')
broken pipe, bye bye

Example output server

> python pipe_test.py s
pipe server
waiting for client
got client
writing message 0
writing message 1
writing message 2
writing message 3
writing message 4
writing message 5
writing message 6
writing message 7
writing message 8
writing message 9
finished now

Obviously you'd need some error checking around the various calls but that should work.

Additional side note: A colleague of mine ran into trouble with the pipe being closed the moment the client tried to perform I/O on it (exception claiming that "all pipe instances are busy"). It turned out that he was using os.path.exists in the client code to test whether the named pipe already existed before running CreateFile on it. This somehow breaks the pipe. So using the approach above (CreateFile wrapped in a try-except) is the safe way of trying to connect to a pipe until it has been created by the server end.

Answer 4

I have success with something like the following fragment. This code is derived from CaptureSetup/Pipes — Python on Windows — The Wireshark Wiki. It requires win32pipe and win32file from the pywin32 package.

# pipename should be of the form \\.\pipe\mypipename
pipe = win32pipe.CreateNamedPipe(
        pipename,
        win32pipe.PIPE_ACCESS_OUTBOUND,
        win32pipe.PIPE_TYPE_MESSAGE | win32pipe.PIPE_WAIT,
        1, 65536, 65536,
        300,
        None)
try:
    win32pipe.ConnectNamedPipe(pipe, None)

    while True:
        some_data = b'12345...'
        win32file.WriteFile(pipe, some_data)
        ...
finally:
    win32file.CloseHandle(pipe)

I don't know if it's 100% correct in the way it closes the pipe.

You referred to The Perils and Triumphs of Being a Geek: Named Pipes between C# and Python—Jonathon Reinhart. I tried it, but it wasn't able to create the named pipe. I wonder if that code only works to open a named pipe that has already been created by another process.