CODEWITHSUNDEEP
c#filestreamlarge-files
Aeon2058
Aeon2058

Reputation: 547

Difficulty reading large file into byte array

I have a very large BMP file that I have to read in all at once because I need to reverse the bytes when writing it to a temp file. This BMP is 1.28GB, and I'm getting the "Out of memory" error. I can't read it completely (using ReadAllBytes) or using a buffer into a binary array because I can't initialize an array of that size. I also can't read it into a List (which I could then Reverse()) using a buffer because halfway through it runs out of memory.

So basically the question is, how do I read a very large file backwards (ie, starting at LastByte and ending at FirstByte) and then write that to disk?

Bonus: when writing the reversed file to disk, do not write the last 54 bytes.

Upvotes: 0

Views: 3770

Answers (3)

Roy Dictus
Roy Dictus

Reputation: 33139

With a StreamReader object, you can Seek (place the "cursor") to any particular byte, so you can use that to go over the entire file's contents in reverse.

Example:

const int bufferSize = 1024;
string fileName = 'yourfile.txt';

StreamReader myStream = new StreamReader(fileName);
myStream.BaseStream.Seek(bufferSize, SeekOrigin.End);

char[] bytes = new char[bufferSize];
while(myStream.BaseStream.Position > 0)
{
    bytes.Initialize();
    myStream.BaseStream.Seek(bufferSize, SeekOrigin.Current);
    int bytesRead = myStream.Read(bytes, 0, bufferSize);
}

Upvotes: 4

Memoizer
Memoizer

Reputation: 2279

You can use Memory Mapped Files.

http://msdn.microsoft.com/en-us/library/vstudio/dd997372%28v=vs.100%29.aspx

Also, you can use FileStream and positioning on necessary position by stream.Seek(xxx, SeekOrigin.Begin) (relative position) or Position property (absolute position).

Upvotes: 2

Tigran
Tigran

Reputation: 62246

You can not normally handle so big files in .NET, due the implied memory limit for CLR applications and collections inside them neither for 32 nor for 64 platform.

For this you can use Memory Mapped File, to read a file directly from the disk, without loading it into the memory. One time memory mapping created move the reading pointer to end of the file and read backwards.

Hope this helps.

Upvotes: 4

Related Questions