CODEWITHSUNDEEP
c#sql-servermongodbmongodb-.net-driver
balexandre
balexandre

Reputation: 75073

Using SqlBulkCopy with MongoDB

From a previous question, I'm trying to do a SqlBulkCopy from a MongoDB database, and I'm getting an error and can't find what column type I should have:

The given value of type ObjectId from the data source cannot be converted to type nvarchar of the specified target column.

enter image description here

Where my DataTable Column DataType is MongoDB.Bson.ObjectId.

What should be the type in Microsoft Sql Server to host this value?

My current code:

string connectionString = GetDestinationConnectionString();
var mongoCollection = GetSourceConnectionString();

var queryFind = Query.And(Query.NotExists("sync"), Query.NotIn("challenge_guid", new List<MongoDB.Bson.BsonValue>() { "87558b59-73ee-4f10-add4-b9031551e395" }));
var sourceData = mongoCollection.Find(queryFind).ToList();

DataTable dt = CollectionHelper.ConvertTo<MongoAnswerDoc>(sourceData);

using (SqlConnection destinationConnection =
            new SqlConnection(connectionString))
{
    destinationConnection.Open();

    // Set up the bulk copy object. 
    // Note that the column positions in the source
    // data reader match the column positions in 
    // the destination table so there is no need to
    // map columns.
    using (SqlBulkCopy bulkCopy =
                new SqlBulkCopy(destinationConnection))
    {
        bulkCopy.DestinationTableName = "JK_RawChallengeAnswers";

        try
        {
            // Write from the source to the destination.
            bulkCopy.WriteToServer(dt);
        }
        catch (Exception ex)
        {
            txtMsg.Text = ex.Message;
        }
        finally
        {
            // Dispose of the DataTable.
            dt.Dispose();
            // close connection
            destinationConnection.Close();
        }
    }
}

Upvotes: 0

Views: 879

Answers (1)

Remus Rusanu
Remus Rusanu

Reputation: 294237

From the Mongo spec:

ObjectId is a 12-byte BSON type, constructed using:

  • a 4-byte timestamp,
  • a 3-byte machine identifier,
  • a 2-byte process id, and
  • a 3-byte counter.

So you would need a BINARY(12) type column to map it in SQL.

Anyway, your code will run out of memory on any transfer of significance, using an intermediate DataTable in-memory copy is not the way to go. EnableStreaming and use an IDataReader to iterate over the source just-in-time.

Upvotes: 1

Related Questions