Override StreamReader's ReadLine method

Question

I'm trying to override a StreamReader's ReadLine method, but having difficulty doing so due to inability to access some private variables. Is this possible, or should I just write my own StreamReader class?

Answer 1

Assuming you want your custom StreamReader to be usable anywhere that a TextReader can be used there are typically two options.

1. Inherit from StreamReader and override the functions that you want to have work differently. In your case this would be StreamReader.ReadLine.

2. Inherit from TextReader and implement the reader functionality completely to your requirements.

NB: For option 2 above, you can maintain an internal reference to a StreamReader instance and delegate all the functions to the internal instance, except for the piece of functionality that you want to replace. In my view, this is just an implementation detail of option 2 rather than a 3rd option.

Based on your question I assume you have tried option 1 and found that overriding StreamReader.ReadLine is rather difficult because you could not access the internals of the class. Well for StreamReader you are lucky and can achieve this without having access to the internal implementation of the StreamReader.

Here is a simple example:

Disclaimer: The ReadLine() implementation is for demonstration purposes and is not intended to be a robust or complete implementation.

class CustomStreamReader : StreamReader
{
  public CustomStreamReader(Stream stream)
    : base(stream)
  {
  }

  public override string ReadLine()
  {
    int c;

    c = Read();
    if (c == -1)
    {
      return null;
    }

    StringBuilder sb = new StringBuilder();
    do
    {
      char ch = (char)c;
      if (ch == ',')
      {
        return sb.ToString();
      }
      else
      {
        sb.Append(ch);
      }
    } while ((c = Read()) != -1);
    return sb.ToString();
  }
}

You will notice that I simply used the StreamReader.Read() method to read the characters from the stream. While definitely less performant that working directly with the internal buffers, the Read() method does use the internal buffering so should still yield pretty good performance, but that should be tested to confirm.

For fun, here is a example of option 2. I used the encapsulated StreamReader to reduce the actual code, this is not tested at all..

class EncapsulatedReader : TextReader
{
  private StreamReader _reader;

  public EncapsulatedReader(Stream stream)
  {
    _reader = new StreamReader(stream);      
  }

  public Stream BaseStream
  {
    get
    {
      return _reader.BaseStream;
    }
  }

  public override string ReadLine()
  {
    int c;

    c = Read();
    if (c == -1)
    {
      return null;
    }
    StringBuilder sb = new StringBuilder();
    do
    {
      char ch = (char)c;
      if (ch == ',')
      {
        return sb.ToString();
      }
      else
      {
        sb.Append(ch);
      }
    } while ((c = Read()) != -1);
    return sb.ToString();
  }

  protected override void Dispose(bool disposing)
  {
    if (disposing)
    {
      _reader.Close();
    }
    base.Dispose(disposing);
  }

  public override int Peek()
  {
    return _reader.Peek();
  }

  public override int Read()
  {
    return _reader.Read();
  }

  public override int Read(char[] buffer, int index, int count)
  {
    return _reader.Read(buffer, index, count);
  }

  public override int ReadBlock(char[] buffer, int index, int count)
  {
    return _reader.ReadBlock(buffer, index, count);
  }

  public override string ReadToEnd()
  {
    return _reader.ReadToEnd();
  }

  public override void Close()
  {
    _reader.Close();
    base.Close();
  }
}

Answer 2

Try This, I wrote this because I have some very large '|' delimited files that have \r\n inside of some of the columns and I needed to use \r\n as the end of the line delimiter. I was trying to import some files using SSIS packages but because of some corrupted data in the files I was unable to. The File was over 5 GB so it was too large to open and manually fix. I found the answer through looking through lots of Forums to understand how streams work and ended up coming up with a solution that reads each character in a file and spits out the line based on the definitions I added into it. this is for use in a Command Line Application, complete with help :). I hope this helps some other people out, I haven't found a solution quite like it anywhere else, although the ideas were inspired by this forum and others. This will not fix the files it only splits them... please be aware that this is still a work in progress :).

    class Program
    {
        static long _fileposition = 0;

        static void Main(string[] args)
        {

            // Check information passed in
            if (args.Any())
            {
                if (args[0] == "/?")
                {
                    var message = "Splits a file into smaller pieces";
                    message += "\n";
                    message += "\n";
                    message += "SplitFile [sourceFileName] [destinationFileName] [RowBatchAmount] [FirstRowHasHeader]";
                    message += "\n";
                    message += "\n";
                    message += "     [sourceFileName]  (STRING) required";
                    message += "\n";
                    message += "     [destinationFileName]  (STRING) will default to the same location as the sourceFileName";
                    message += "\n";
                    message += "     [RowBatchAmount]   (INT) will create files that have this many rows";
                    message += "\n";
                    message += "     [FirstRowHasHeader]    (True/False) Will Add Header Row to each new file";
                    Console.WriteLine(message);
                }
                else
                {
                    string sourceFileName = args[0];
                    string destFileLocation = args.Count() >= 2 ? args[1] : sourceFileName.Substring(0, sourceFileName.LastIndexOf("\\"));
                    int RowCount = args.Count() >= 3 ? int.Parse(args[2]) : 500000;
                    bool FirstRowHasHeader = true;
                    FirstRowHasHeader = args.Count() != 4 || bool.Parse(args[3]);

                    // Create Directory If Needed
                    if (!Directory.Exists(destFileLocation))
                    {
                        Directory.CreateDirectory(destFileLocation);
                    }

                    string line = "";
                    int linecount = 0;
                    int FileNum = 1;
                    string newFileName = Path.Combine(destFileLocation, Path.GetFileNameWithoutExtension(sourceFileName));
                    newFileName += FileNum + Path.GetExtension(sourceFileName);

                    // Always add Header Line
                    string HeaderLine = GetLine(sourceFileName, _fileposition);
                    int HeaderCount = HeaderLine.Split('|').Count();

                    do
                    {
                        // Add Header Line
                        if ((linecount == 0 & FirstRowHasHeader) | (_fileposition == 1 & !FirstRowHasHeader))
                        {
                            using (FileStream NewFile = new FileStream(newFileName, FileMode.Append))
                            {
                                System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
                                Byte[] bytes = encoding.GetBytes(HeaderLine);
                                int length = encoding.GetByteCount(HeaderLine);
                                NewFile.Write(bytes, 0, length);
                            }
                        }

                        //Evaluate Line
                        line = GetLine(sourceFileName, _fileposition, HeaderCount);

                        if (line == null) continue;

                        // Create File if it doesn't exist and write to it
                        using (FileStream NewFile = new FileStream(newFileName, FileMode.Append))
                        {
                            System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
                            Byte[] bytes = encoding.GetBytes(line);
                            int length = encoding.GetByteCount(line);
                            NewFile.Write(bytes, 0, length);
                        }

                        //Add to the line count
                        linecount++;

                        //Create new FileName if needed
                        if (linecount == RowCount)
                        {
                            FileNum++;
                            // Create a new sub File, and read into it
                            newFileName = Path.Combine(destFileLocation, Path.GetFileNameWithoutExtension(sourceFileName));
                            newFileName += FileNum + Path.GetExtension(sourceFileName);
                            linecount = 0;
                        }
                    } while (line != null);
                }
            }
            else
            {
                Console.WriteLine("You must provide sourcefile!");
                Console.WriteLine("use /? for help");
            }
        }

        static string GetLine(string sourceFileName, long position, int NumberOfColumns = 0)
        {
            byte[] buffer = new byte[65536];
            var builder = new StringBuilder();
            var finishedline = false;

            using (Stream source = File.OpenRead(sourceFileName))
            {
                source.Position = position;
                var crlf = "\r\n";
                var lf = "\n";
                var length = source.Length;

                while (source.Position = 0 & finishedline == false &  _fileposition = NumberOfColumns) | NumberOfColumns == 0)
                                            {
                                                // Remove all Control Line Feeds before the end of the line.
                                                builder = builder.Replace(crlf, lf);

                                                // Add Final Control Line Feed
                                                var x = (char)NewLine.Read();
                                                builder.Append(x);
                                                finishedline = true;
                                                _fileposition++;
                                                continue;
                                            }
                                        }

                                        break;
                                    }
                                default:
                                    builder.Append(c);
                                    break;
                            }
                        }
                    }
                    break;
                }
            }

            return (builder.ToString() == "" ? null: builder.ToString());
        }
    }

References: http://social.msdn.microsoft.com/forums/en-US/csharpgeneral/thread/b0d4cba1-471a-4260-94c1-fddd4244fa23/

this one helped me the most: https://stackoverflow.com/a/668003/1582188

Answer 3

this class can help you

public class MyStreamReader : System.IO.StreamReader
{
    public MyStreamReader(string path)
        : base(path)
    {

    }
    public override string ReadLine()
    {
        string result = string.Empty;
        int b = base.Read();
        while ((b != (int)',') && (b > 0))
        {
            result += this.CurrentEncoding.GetString(new byte[] { (byte)b });
            b = base.Read();
        }
        return result;
    }
}