Special Characters in StreamWriter

Question

I am using streamwriter to write a string into stream. Now when I access the data from the stream, it adds "\0\0\0" characters to end of the content. I have to append the stream contents, so it creates problem as I am not able to remove these characters by trim() or remove() or replace() methods.

Below is the code I am using:

FOR WRITING :

using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
    using (StreamWriter writer = new StreamWriter(stream, System.Text.Encoding.Unicode))
    {
        try
        {
            string[] files = System.IO.Directory.GetFiles(folderName, "*.*", System.IO.SearchOption.AllDirectories);
            foreach (string str in files)
            {
                writer.WriteLine(str);
            }
            // writer.WriteLine(folderName);
        }
        catch (Exception ex)
        {
            Debug.WriteLine("Unable to write string. " + ex);
        }
        finally
        {
            mutex.ReleaseMutex();
            mutex.WaitOne();
        }
    }
}

FOR READING :

 StringBuilder sb = new StringBuilder();
            string str = @"D:\Other Files\Test_Folder\New Text Document.txt";
 using (var stream = mmf.CreateViewStream())
                {
                    System.IO.StreamReader reader = new System.IO.StreamReader(stream);
                    sb.Append(reader.ReadToEnd());
                    sb.ToString().Trim('\0');
                    sb.Append("\n" + str);
                }

How can I prevent this?

[UPDATES] Writing

// Lock
            bool mutexCreated;
            Mutex mutex = new Mutex(true, fileName, out mutexCreated);
            if (!mutexCreated)
                mutex = new Mutex(true);
            try
            {
                using (MemoryMappedViewStream stream = mmf.CreateViewStream())
                {
                    using (BinaryWriter writer = new BinaryWriter(stream))
                    {
                        try
                        {
                            string[] files = System.IO.Directory.GetFiles(folderName, "*.*", System.IO.SearchOption.AllDirectories);
                            foreach (string str in files)
                            {
                                writer.Write(str);
                            }
                            writer.Flush();
                        }
                        catch (Exception ex)
                        {
                            Debug.WriteLine("Unable to write string. " + ex);
                        }
                        finally
                        {
                            mutex.ReleaseMutex();
                            mutex.WaitOne();
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                Debug.WriteLine("Unable to monitor memory file. " + ex);
            }

Reading

StringBuilder sb = new StringBuilder();
            string str = @"D:\Other Files\Test_Folder\New Text Document.txt";
            try
            {
                using (var stream = mmf.CreateViewStream())
                {
                    System.IO.BinaryReader reader = new System.IO.BinaryReader(stream);
                    sb.Append(reader.ReadString());
                    sb.Append("\n" + str);
                }
                using (var stream = mmf.CreateViewStream())
                {
                    System.IO.BinaryWriter writer = new System.IO.BinaryWriter(stream);
                    writer.Write(sb.ToString());
                }
                using (var stream = mmf.CreateViewStream())
                {
                    System.IO.BinaryReader reader = new System.IO.BinaryReader(stream);
                    Console.WriteLine(reader.ReadString());
                }
            }
            catch (Exception ex)
            {
                Debug.WriteLine("Unable to monitor memory file. " + ex);
            }

Answer 1

Your combination of a MMF and TextWriter/TextReader, especially ReadToEnd() is not a good match.

A textReader needs the EOF concept of the underlying file and a MMF just does not supply that in the same way. You will get your strings stuffed with \0\0... up to the capacity of the MMF.

As a possible fix:

• collect the strings to write in a StringBuilder
• use a BinaryWriter to write it as 1 string
• read it back with a BinaryReader.

Another options is to use WriteLine/ReadLine and define some EOF marker (empty line or special string).

The BinaryWriter will prefix the string with a length-prefix so that the Reader knows when to stop.

Answer 2

No '\0' are getting appended by StreamWriter. These are just the content of the memory-mapped file, stuff that was there before you started writing. The StreamReader needs an end-of-file indicator to know when the stop reading. There isn't any in an mmf beyond the size of the section. Like the 2nd argument you pass to MemoryMappedFile.CreateNew(string, long).

Or in other words, you created a mmf that's too large to fit the stream. Well, of course, you didn't have the time machine to guess how large to make it. You definitely need to do something about it, trimming the zeros isn't good enough. That goes wrong the second time you write a stream that's shorter. The reader will now still sees the bytes from the previous stream content and they won't be zero.

This is otherwise a common headache with mmfs, they are just chunks of memory and a stream is a very poor abstraction of that. One of the big reasons it took so long for mmfs to get supported by .NET, even though they are a very core OS feature. You need pointers to map a mmf and that's just not well supported in a managed language.

I don't see a good way to teach StreamReader new tricks in this case. Copying the bytes from the mmf into a MemoryStream would fix the problem but rather defeats the point of a mmf.

Consider using a pipe instead.