Concurrent read/write of named pipe in Java (on windows)

Question

I'm trying to provide communication between a C# app and a Java app on windows using named pipes with the method described by v01ver in this question: How to open a Windows named pipe from Java?

I'm running into a problem on the Java side because I have a reader thread constantly waiting for input on the pipe and when I try to write to the pipe from my main thread it gets stuck forever.

final RandomAccessFile pipe;
try {
   pipe = new RandomAccessFile("\\\\.\\pipe\\mypipe", "rw");
}
catch (FileNotFoundException ex) {
   ex.printStackTrace();
   return;
}

Thread readerThread = new Thread(new Runnable() {
   @Override
   public void run() {
      String line = null;
      try {
         while (null != (line = pipe.readLine())) {
            System.out.println(line);
         }
      }
      catch (IOException ex) {
         ex.printStackTrace();
      }
   }
});
readerThread.start();

try { Thread.sleep(500); } catch (InterruptedException e) {}

try {
   System.out.println("Writing a message...");
   pipe.write("Hello there.\n".getBytes());
   System.out.println("Finished.");
}
catch (IOException ex) {
   ex.printStackTrace();
}

The output is:

Writing a message...

and then it waits forever.

How can I write to a named pipe while waiting for input in another thread?

Answer 1

Pipes are uni-directional meaning, a pipe can do only child to parent read operations and parent to child write operations, or vice-versa, not both. You need to use two pipes for this, on both the C# and Java side, in order for both of them to perform read and write operations.

Pipes vs Sockets: https://www.baeldung.com/cs/pipes-vs-sockets

[ EDIT ]
As a result, you cannot perform read and write operations on the same pipe. Some languages like C# provide duplex-pipes, which can perform read and write operations on the same pipe, but surely these are two pipes "under the hood" that are used for read and write operations respectively. So as long as is an abstraction and you do not really know what is happening "under the hood", you are better off segmenting the read and write operations between two pipes. Another factor that may cause the issue is that Java cannot officially support pipe servers, in order to do that you will need to use the Java Native Access library: https://github.com/java-native-access/jna. The RandomAccessFile Java implementation is using the OS file system in order to perform read write operations on the pipe, so as a result it has some limitations and as a result, reading and writing asynchronously within the OS file system object may result in a thread to be locked if two threads are accessing a block of memory, and could result even in memory corruption (Race Condition). Another issue is when I tried to perform two read operations on a pipe from a different thread, I got the \\.\pipe\testpipe1 (All pipe instances are busy) exception. This means that most probably, in your scenario, your pipe may be busy and got locked in a waiting for a response state when performing the write operation. As a result of the previous factors, the best option is to perform synchronous Read/Write operations on the pipe, preferably using two pipes, in order to have maximum control over the performed read and write operations.

[C# Application]

using System;
using System.IO;
using System.IO.Pipes;

class C_Sharp_Pipe
{
    static void Main()
    {
        Write_Pipe();
        Read_Pipe();

        Console.ReadLine();
    }


    static async void Write_Pipe()
    {
        using (NamedPipeServerStream pipeServer = new NamedPipeServerStream("testpipe1", PipeDirection.Out))
        {
            try
            {
                pipeServer.WaitForConnection();

                using (StreamWriter sw = new StreamWriter(pipeServer))
                {
                    sw.WriteLine("Hello Java process");
                    Console.WriteLine("Message sent to testpipe1 Java client: " + "\"Hello\"");

                    await sw.FlushAsync();
                }

            }
            catch (Exception e)
            {
                Console.WriteLine("ERROR: " + e.Message);
            }
        }
    }


    static async void Read_Pipe()
    {
        using (NamedPipeServerStream pipeServer = new NamedPipeServerStream("testpipe2", PipeDirection.InOut))
        {
            try
            {
                pipeServer.WaitForConnection();

                using (StreamReader sr = new StreamReader(pipeServer))
                {
                    string received_message = await sr.ReadLineAsync();
                    Console.WriteLine("Message received from Java testpipe2 server: " + received_message);

                }
            }
            catch (Exception e)
            {
                Console.WriteLine("ERROR: " + e.Message);
            }
        }

    }
}

[Java Application]

package com.algorithms;

import java.io.RandomAccessFile;
import java.nio.charset.StandardCharsets;
import java.util.Scanner;

public class Main {

    public static void main(String[] args)
    {
        Read_Pipe();
        Write_Pipe();

        Scanner s = new Scanner(System.in);
        s.nextLine();
    }


    private static void Read_Pipe()
    {
        try
        {
            RandomAccessFile pipe = new RandomAccessFile("\\\\.\\pipe\\testpipe1", "r");
            byte[] buffer = new byte[1024];

            pipe.read(buffer);

            System.out.println("Response from C# testpipe1 pipe server: " + new String(buffer, StandardCharsets.UTF_8) );
            pipe.close();
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }
    }



    private static void Write_Pipe()
    {
        try
        {
            RandomAccessFile pipe = new RandomAccessFile("\\\\.\\pipe\\testpipe2", "rw");
            String buffer = "Hello C# process";

            pipe.write(buffer.getBytes(StandardCharsets.UTF_8));

            System.out.println("Message sent to C# testpipe2 pipe client: " + buffer);
            pipe.close();
        }
        catch (Exception e)
        {
            System.out.println(e.getMessage());
        }
    }



}

[ C# Inter-Process Communication Result ]

[Java Inter-Process Communication Result]

Answer 2

I'm not familiar with JAVA, and my C# is pretty elementary too. However I'm had a similar problem with a multithreaded C++ client that I fixed by opening the pipe for overlapped IO. Until I did this, Windows serialized reads and writes, effectively causing an unsatisfied (blocking) ReadFile to prevent completion of a subsequent WriteFile until the read was done.

See CreateFile function
FILE_FLAG_OVERLAPPED

Answer 3

Don't worry, using RandomAccessFile to access a named pipe is correct. A named pipe is a file system object. Under Linux/Unix it is also called "fifo". Those objects are readable just like a file. (and not the same as pipes used between processes which are abstracted by Java Pipe class).

However I see two problems with your program. I cannot test it currently as I would need your test server (feel free to publish). Your reader thread waits for answers from the other side (i.e. the server). It uses readLine(), I would use a different method (for debugging reading char by char might be the best).

With Java (without JNI) you cannot actually create a named pipe (server side). Opening a named pipe with the generic method used by RandomAccessFile you will get a byte-type stream which can be one-way or duplex.

BTW: JTDS (the free JDBC driver for SQL Server) can optionally use a named pipe to access SQL server, even over the network. And it is using exactly the RandomAccessFile method.

BTW2: there is a makepipe.exe test server on older MS SQL Server installation media, however I did not find a trusted source to get that file.

Answer 4

I have a same problem -- communication between a C#/Python app and a Java app on windows using named pipes:

We have example of Client Code written on Java, but in line String echoResponse = pipe.readLine(); tread waits forever.

try {
    // Connect to the pipe
    RandomAccessFile pipe = new RandomAccessFile("\\\\.\\pipe\\testpipe", "rw");
    String echoText = "Hello word\n";
    // write to pipe
    pipe.write ( echoText.getBytes() );
    // read response
    String echoResponse = pipe.readLine();
    System.out.println("Response: " + echoResponse );
    pipe.close();

    } catch (Exception e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
    }

Solution of problem: I have a ServerPipe code written on Python from here Example Code - Named Pipes: and run its on Python 2.6.6

from ctypes import *

PIPE_ACCESS_DUPLEX = 0x3
PIPE_TYPE_MESSAGE = 0x4
PIPE_READMODE_MESSAGE = 0x2
PIPE_WAIT = 0
PIPE_UNLIMITED_INSTANCES = 255
BUFSIZE = 4096
NMPWAIT_USE_DEFAULT_WAIT = 0
INVALID_HANDLE_VALUE = -1
ERROR_PIPE_CONNECTED = 535

MESSAGE = "Default answer from server\0"
szPipename = "\\\\.\\pipe\\mynamedpipe"


def ReadWrite_ClientPipe_Thread(hPipe):
    chBuf = create_string_buffer(BUFSIZE)
    cbRead = c_ulong(0)
    while 1:
        fSuccess = windll.kernel32.ReadFile(hPipe, chBuf, BUFSIZE,
byref(cbRead), None)
        if ((fSuccess ==1) or (cbRead.value != 0)):
            print chBuf.value
            cbWritten = c_ulong(0)
            fSuccess = windll.kernel32.WriteFile(hPipe,
                                                 c_char_p(MESSAGE),
                                                 len(MESSAGE),
                                                 byref(cbWritten),
                                                 None
                                                )
        else:
            break
        if ( (not fSuccess) or (len(MESSAGE) != cbWritten.value)):
            print "Could not reply to the client's request from the
pipe"
            break
        else:
            print "Number of bytes written:", cbWritten.value

    windll.kernel32.FlushFileBuffers(hPipe)
    windll.kernel32.DisconnectNamedPipe(hPipe)
    windll.kernel32.CloseHandle(hPipe)
    return 0

def main():
    THREADFUNC = CFUNCTYPE(c_int, c_int)
    thread_func = THREADFUNC(ReadWrite_ClientPipe_Thread)
    while 1:
        hPipe = windll.kernel32.CreateNamedPipeA(szPipename,
                                                 PIPE_ACCESS_DUPLEX,
                                                 PIPE_TYPE_MESSAGE |
                                                 PIPE_READMODE_MESSAGE
|
                                                 PIPE_WAIT,

PIPE_UNLIMITED_INSTANCES,
                                                 BUFSIZE, BUFSIZE,

NMPWAIT_USE_DEFAULT_WAIT,
                                                 None
                                                )
        if (hPipe == INVALID_HANDLE_VALUE):
            print "Error in creating Named Pipe"
            return 0

        fConnected = windll.kernel32.ConnectNamedPipe(hPipe, None)
        if ((fConnected == 0) and (windll.kernel32.GetLastError() ==
ERROR_PIPE_CONNECTED)):
            fConnected = 1
        if (fConnected == 1):
            dwThreadId = c_ulong(0)
            hThread = windll.kernel32.CreateThread(None, 0,
thread_func, hPipe, 0, byref(dwThreadId))
            if (hThread == -1):
                print "Create Thread failed"
                return 0
            else:
                windll.kernel32.CloseHandle(hThread)
        else:
            print "Could not connect to the Named Pipe"
            windll.kernel32.CloseHandle(hPipe)
    return 0


if __name__ == "__main__":
    main()

After server have start you can use slightly modified version of the Java Client code:

try {
    // Connect to the pipe
    RandomAccessFile pipe = new RandomAccessFile("\\\\.\\pipe\\mynamedpipe", "rw");
    String echoText = "Hello world\n";
    // write to pipe
    pipe.write(echoText.getBytes());

    //String aChar;
    StringBuffer fullString = new StringBuffer();

    while(true){
        int charCode = pipe.read();
        if(charCode == 0) break;
        //aChar = new Character((char)charCode).toString();
        fullString.append((char)charCode);
    }

    System.out.println("Response: " + fullString);
    pipe.close();
}
catch (Exception e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}

It works well in NetBeans 6.9.1.

Answer 5

This is expected behaviour of pipes. It is supposed to hang untill other process connects to the pipe and reads it.

Answer 6

I suppose that RandomAccessFile is not the right API here. Try a FileInputStream + FileOutputStream on the Java side. But that is only a guess, as I last used the Windows API in times when named pipes didn't yet exist.