CODEWITHSUNDEEP
sqlvb.netlinqbulkinsert
GertDeWilde
GertDeWilde

Reputation: 351

Bulk Insert with Linq to Sql (vb.net)

I'm trying to do a bulk insert into my sql table, using Linq to Sql. I'm still in the learning fase of linq, so please be gentle.

The code is not the problem, since everything is importing great, but the process is just terrible slow: 4 seconds for each record, while I have around 15k records.

The main core of the problem (I think) is that before I want to add a record, I first have to check if this record already exist, to be sure I don't have duplicates. So this means that I do a search into my already existing database of 15k records, and send a true/false response if it has found the record.

        Dim expr = From spare In db.tblSpareParts Where spare.SparePartDeleted = False Select spare
        For Each part In expr
            If part.SparePartYnumber = ynumber Then
                Return True
            End If
        Next
        Return False

The main code, where we loop through an access document.

        For Each Me.dr In dt.Rows
            If dr.Item(0).ToString <> "" Then
                blnfoundit = db.getynumberinfo(dr.Item(0).ToString)
                If blnfoundit Then
                    db.setSparePart("Toevoegen", dr.Item(0).ToString, dr.Item(1).ToString, "", "", "Mat Ref: " & dr.Item(2).ToString & " - Vendor : " & dr.Item(5).ToString, 0, 0, CInt(IIf(dr.Item(12).ToString = "", 0, dr.Item(12).ToString)))
                End If
            End If
        Next

To the end a piece of the code to place everything in SQL:

                Dim sqlimport As New tblSparePart
                With sqlimport
                    .SparePartCurrentStock = currentstock
                    .SparePartDeleted = False
                    .SparePartDescription = description
                    .SparePartLastModified = DateTime.Now()
                    .SparePartLocation = location
                    .SparePartMinimumStock = minimumstock
                    .SparePartPrice = price
                    .SparePartRemarks = remarks
                    .SparePartType = type
                    .SparePartUserName = General.username
                    .SparePartYnumber = ynumber
                End With
                db.tblSpareParts.InsertOnSubmit(sqlimport)
                db.SubmitChanges()

Any ideas on how to speed up this process?

Btw, I did search the web for similar things, and found something about SqlBulkCopy, but I have no idea on how to use it, and if it would be useful. I have a Tags Table. How to Bulk Insert using LINQ?

Thanks in advance for the responses,

Gert

Upvotes: 1

Views: 984

Answers (1)

pseudocoder
pseudocoder

Reputation: 4392

If this works at all like LINQ to Entities, the slowness here is actually from creating the change tracking objects for each record.

In that case, you can use DbContext.Configuration.AutoDetectChangesEnabled = False to disable change tracking. Change detection is deferred until the loop is complete and the entire context is processed.

See this article for more information on the EF side: DbContext AutoDetectChangesEnabled set to false detecting changes

As for your problem with LINQ to SQL, you might look into the DataContext.ObjectTrackingEnabled property as outlined in this article: http://msdn.microsoft.com/en-us/library/system.data.linq.datacontext.objecttrackingenabled(v=vs.90).aspx

Perhaps setting this property to false before your loop and back to true afterwards (but before SubmitChanges) might help performance.

Upvotes: 1

Related Questions