How can I stream a BLOB to VARBINARY(MAX) on INSERT

Question

I've found lots of information on how to select a BLOB as a stream using CommandBehavior.SequentialAccess.

I'd like to stream the BLOB on an insert too (to avoid caching the BLOB as a byte array in memory), and I can't find any examples. I've found some documentation mentioning a .WRITE (expression,@Offset, @Length) syntax in the UPDATE T-SQL statement which is compatible with VARBINARY(MAX). So, I'm thinking of writing a class which can take a Stream and chunk it into the database using successive UPDATE (.WRITE) statements. Is this the right way to do this, or is there a better way?

Links to UPDATE.WRITE:

http://msdn.microsoft.com/en-us/library/ms178158(SQL.100).aspx

http://msdn.microsoft.com/en-us/library/ms177523(v=SQL.100).aspx

Links to selecting BLOBs using CommandBehavior.SequentialAccess:

http://msdn.microsoft.com/en-us/library/87z0hy49.aspx

Memory effective way to read BLOB data in C#/SQL 2005

Getting binary data using SqlDataReader

How to make streams from BLOBs available in plain old C# objects when using SqlDataReader?

Streaming VARBINARY data from SQL Server in C#

Here's a POC of using the .Write syntax:

DDL:

create database BlobTest
go
use blobtest
go

create table Blob
(
    Id bigint not null primary key identity(1,1),
    Data varbinary(max) not null default(0x)
)

C#:

using System.Data;
using System.Data.SqlClient;
using System.IO;
using System.Linq;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main()
        {
            string pathToBigFile = "C:\\bigfile.big";
            int optimumBufferSizeForSql = 8040; //See https://stackoverflow.com/questions/5629991/how-can-i-generate-an-insert-script-for-a-table-with-a-varbinarymax-field

            long newBlobId = InitialiseNewBlobInSqlServer();

            using (Stream stream = new FileStream(  pathToBigFile, 
                                                    FileMode.Open, 
                                                    FileAccess.Read, 
                                                    FileShare.ReadWrite))
            {
                byte[] buffer = new byte[optimumBufferSizeForSql];

                while(true)
                {
                    int numberBytesRead = stream.Read(buffer, 0, optimumBufferSizeForSql);

                    if (numberBytesRead == 0)
                    {
                        //Done
                        break;
                    }

                    WriteBufferToSqlServer(
                        numberBytesRead == optimumBufferSizeForSql ? buffer : buffer.Take(numberBytesRead).ToArray(),
                        newBlobId);
                }
            }
        }

        static long InitialiseNewBlobInSqlServer()
        {
            using (SqlConnection conn = new SqlConnection("Data Source=localhost; Initial Catalog=BlobTest; Integrated Security=SSPI;"))
            using (SqlCommand command = new SqlCommand())
            {
                command.Connection = conn;
                command.CommandType = CommandType.Text;
                command.CommandText = "Insert into blob (Data) values (0x); select convert(bigint,Scope_identity());";

                conn.Open();
                return (long) command.ExecuteScalar();
            }
        }

        static void WriteBufferToSqlServer(byte[] data, long blobId)
        {
            using (SqlConnection conn = new SqlConnection("Data Source=localhost; Initial Catalog=BlobTest; Integrated Security=SSPI;"))
            using (SqlCommand command = new SqlCommand())
            {
                command.Connection = conn;
                command.CommandType = CommandType.Text;
                command.Parameters.AddWithValue("@id", blobId);
                command.Parameters.AddWithValue("@data", data);
                command.CommandText = "Update Blob set Data.Write(@data, null, null) where Id = @id;";

                conn.Open();
                command.ExecuteNonQuery();
            }
        }
    }
}

Answer 1

You should be using RBS interface of SQL Server for working with blobs.