CODEWITHSUNDEEP
c#streambyte
nina
nina

Reputation: 95

Convert Stream to byte[] c# for large files of 2GB

Trying to convert Stream object to byte[] and using the below method for the same:

public static byte[] ReadFully(System.IO.Stream input)
{
            byte[] buffer = new byte[16*1024];
            using (System.IO.MemoryStream ms = new System.IO.MemoryStream())
            {
                int read;
                while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
                {
                    ms.Write(buffer, 0, read);
                }
                return ms.ToArray();
            }
}

However the input parameter "input" is for large file that is of 2 GB and hence the code does not get enter into while loop and hence does not convert it to byte array.

For smaller files it is working fine

Upvotes: 1

Views: 2894

Answers (2)

Sungjin Jung
Sungjin Jung

Reputation: 11

Object limited below 32bit. (that is why all index using int)

how about use list contains byte array to deal entire data?

    public List<byte[]> ReadBytesList(string fileName)
    {
        List<byte[]> rawDataBytes= new List<byte[]>();
        byte[] buff;
        FileStream fs = new FileStream(fileName,
                                       FileMode.Open,
                                       FileAccess.Read);
        BinaryReader br = new BinaryReader(fs);
        long numBytes = new FileInfo(fileName).Length;
        int arrayCount= (int)(numBytes / 2100000000); //2147483648 is max
        int arrayRest = (int)(numBytes % 2100000000);
        if(arrayCount>0)
        {
            for (int i = 0; i < arrayCount; i++)
            {
                buff = br.ReadBytes(2100000000);
                rawDataBytes.Add(buff);
            }
            buff = br.ReadBytes(arrayRest);
            rawDataBytes.Add(buff);
        }
        else
        {
            buff = br.ReadBytes(arrayRest);
            rawDataBytes.Add(buff);
        }
        return rawDataBytes;
    }

Upvotes: 0

mrousavy
mrousavy

Reputation: 311

That's what a Stream is for. You don't load the whole content into a byte[], you read a small buffer from the Stream into memory and handle it, then dispose and read the next buffer.

If you still need to use a byte[]:

It seems like your app can't handle more than 2^32 Bytes Memory, meaning it's 32bit. Try changing it to 64bit (in Project Properties go to Build and disable Prefer 32 bit)

Upvotes: 1

Related Questions