CODEWITHSUNDEEP
c#.netfilestream
escapecharacter
escapecharacter

Reputation: 965

Odd behaviour: copying more bufferSize with .NET Stream.CopyTo(Stream destination, int bufferSize)

I have two RIFF files, and I am copying chunks from one file to the other. The RIFF format I'm using is:

I use the following code to copy from a source to destination file. Chunks are referred to as frames.

public void CopySegmentTo(VPCaptureSession captureSession, int StartFrame, int EndFrame) {
        //captures several frames from this session onto the end of another one.
        //binary-styles

        Console.WriteLine(" captureSession.captureStream.Position: " + captureSession.captureStream.Position);
        long lastCaptureDestPosition = 0;

        captureSession.captureStream.Seek(0, SeekOrigin.End);
        int chunkLength; long timestamp; char[] charTag;

        for (int f = StartFrame; f <= EndFrame; f++)
        {
            Console.WriteLine("-----");

            this.captureStream.Seek(FrameTimes.ElementAt(f).Item3, SeekOrigin.Begin);

            Console.WriteLine("FrameIndex in this.captureStream: " + FrameTimes.ElementAt(f).Item3);

            BinaryReader reader = new BinaryReader(this.captureStream);
            byte[] tag = reader.ReadBytes(Tags.TAG_SIZE);
            charTag = TagChunk.encode.GetChars(tag);
            chunkLength = reader.ReadInt32();

            BinaryWriter writer = new BinaryWriter(captureSession.captureStream);
            writer.Write(tag);
            writer.Write(chunkLength);
            this.captureStream.CopyTo(captureSession.captureStream, chunkLength);

            long CaptureDestDelta = captureSession.captureStream.Position - lastCaptureDestPosition;

            Console.WriteLine("Loading " + f + " between " + StartFrame + "&" + EndFrame 
                + ". chunkLength: " + chunkLength
                + " this.captureStream.Position: " + this.captureStream.Position
                + " captureSession.captureStream.Position: " + captureSession.captureStream.Position
                + " CaptureDestDelta: " + CaptureDestDelta
                + " Delta Factor: " + CaptureDestDelta/chunkLength);

            lastCaptureDestPosition = captureSession.captureStream.Position;

        }

    }

The copying is very slow, and the resulting file seems to be enormous. Here is an example of the Console output:

captureSession.captureStream.Position: 0
-----
FrameIndex in this.captureStream: 0
Loading 0 between 0&5. chunkLength: 266736 this.captureStream.Position: 121225207 
captureSession.captureStream.Position: 121225207 CaptureDestDelta: 121225207 
Delta Factor: 454
-----
FrameIndex in this.captureStream: 266744
Loading 1 between 0&5. chunkLength: 311398 this.captureStream.Position: 121225207 
captureSession.captureStream.Position: 242183670 CaptureDestDelta: 120958463 
Delta Factor: 388
-----
FrameIndex in this.captureStream: 578150
Loading 2 between 0&5. chunkLength: 356578 this.captureStream.Position: 121225207 
captureSession.captureStream.Position: 362830727 CaptureDestDelta: 120647057 
Delta Factor: 338
-----
FrameIndex in this.captureStream: 934736
Loading 3 between 0&5. chunkLength: 430445 this.captureStream.Position: 121225207 
captureSession.captureStream.Position: 483121198 CaptureDestDelta: 120290471 
Delta Factor: 279
-----
FrameIndex in this.captureStream: 1365189
Loading 4 between 0&5. chunkLength: 437468 this.captureStream.Position: 121225207 
captureSession.captureStream.Position: 602981216 CaptureDestDelta: 119860018 
Delta Factor: 273
-----
FrameIndex in this.captureStream: 1802665
Loading 5 between 0&5. chunkLength: 439870 this.captureStream.Position: 121225207 
captureSession.captureStream.Position: 722403758 CaptureDestDelta: 119422542 
Delta Factor: 271

There is definitely something weird going on. Here's the major things I see:

Let me know if there's anything more I could be logging. I'm really stuck here.

Upvotes: 0

Views: 585

Answers (2)

Jim Mischel
Jim Mischel

Reputation: 134035

You have several problems, the primary one being that the Stream.Copy method you're calling copies the entire stream (from the current position). The second parameter is the buffer size, not the number of bytes to copy.

In addition, you're creating a new BinaryReader and a new BinaryWriter for every iteration of the loop. Both of these classes allocate managed resources and should be disposed when you're done with them. Unfortunately, disposing also closes the underlying stream. But leaving those objects around is just begging for a resource starvation problem at some point, especially if you start copying large parts of the file.

I would suggest that you ditch the BinaryReader and BinaryWriter and go directly to the streams.

You could do this by using Stream.Read to read the data, the only difficulty being that reading the int would require that you read 4 bytes and then call BitConverter.ToInt32 to convert to an integer. You could also use that method to copy data from the source (i.e. read into a big byte[] buffer), and then write to the output file using Stream.Write.

If you're using .NET 4.5, you have another option. You can open a BinaryReader on the input and a BinaryWriter on the output before you start your loop. .NET 4.5 introduced a new constructor that lets you say whether you want to leave the underlying stream open after disposing the reader/writer. In your code, it would look something like this:

using (var reader = new BinaryReader(this.CaptureStream, Encoding.Default, true))
{
    using (var writer = new BinaryWriter(captureSession.captureStream, Encoding.Default, true))
    {
        for (int f = StartFrame; f <= EndFrame; f++)
        {
        }
    }
}

You can still do seeks on the input stream, and the reader will do the right thing. Although you might want to write reader.BaseStream.Seek(...).

You can then use the BinaryReader.ReadInt32 to read as in your existing code, and BinaryReader.Read(byte[], int, int) and BinaryWriter.Write(byte[], int32, int32) to copy the block data.

(I didn't realize until a moment ago that .NET 4.5 finally addressed that problem with BinaryReader and BinaryWriter closing the streams.)

Upvotes: 2

escapecharacter
escapecharacter

Reputation: 965

I misunderstood the meaning of bufferSize in Stream.CopyTo(Stream destination, int bufferSize). I thought it was the total amount of bytes to be copied, not simply the buffer to be used while copying. There is no feature in the Stream object to copy only a specific number of bytes. I fixed the problem by replacing

this.captureStream.CopyTo(captureSession.captureStream, chunkLength);

with

writer.Write(reader.ReadBytes(chunkLength));

Upvotes: 0

Related Questions