Change PowerShell script from Invoke to Bulk Copy

Question

I have used the below code for a little while, mainly as a test to understand it a bit more. What I would really like to do now is improve it's speed by using bulk copy.

The code gets database and table information from a series of instances which are held in a text file on a server, it then adds the collected data via invoke into a table.

The issue is it sends the data for every database, and every table one at a time, if at all possible.

I have looked at a number of sites/blogs that give information on this, but they all seem to be not quite what I need.

$Stopwatch = [System.Diagnostics.Stopwatch]::StartNew()
$today = Get-Date
$srvlist = @(Get-Content "c:\scripts\tablegrowth.TXT")
foreach ($server in $srvlist) {
    $srv = New-Object "Microsoft.SqlServer.Management.SMO.Server" $server
    $dbs = $srv.Databases
    foreach ($db in $dbs) {
        if ($db.IsAccessible) {
            $name1 = $db.name
            $size1 = $db.size
            $dbspace1 = $db.DataSpaceUsage/1KB
            $dbindexsp1 = $db.IndexSpaceUsage/1KB
            $dbspaceavail1 = $db.SpaceAvailable/1KB
            #Write-Host "dbname=" $name1 $size1 $dbspace1 $dbindexsp1 $dbspaceavail1 $db.IsAccessible
            switch ($name1) {
                'master' {}
                'msdb' {}
                'model'  {}
                'SSISDB'  {}
                'SSRSData'  {}
                'Northwind' {}
                'tempdb' {}
                default {
                    Invoke-Sqlcmd -ServerInstance "******" -Database "DBAMonitoring" -Query "INSERT INTO is_sql_databases VALUES ('$server','$name1','$today',$dbspace1,$dbspaceavail1,$dbindexsp1) "
                    foreach ( $tbl in $db.tables) {
                        $tname1    = $tbl.Name
                        $tindexsp1 = $tbl.IndexSpaceUsed/1KB
                        $trows1    = $tbl.RowCount
                        $tspace1   = $tbl.DataSpaceUsed/1KB
                        #Write-Host "tbl name=" $tname1 $tindexsp1 $trows $tspace1
                        if ($trows1 -gt 500) {
                            Invoke-Sqlcmd -ServerInstance "****" -Database "DBAMonitoring" -    Query "INSERT INTO is_sql_tables VALUES ('$server','$name1','$tname1','$today',$trows1,$tspace1,$tindexsp1) "
                        } ####end RowCount > 0
                    } #####end Row loop
                } ###### end Default
            } #####end switch
        }  #####end IsAccessible
    }  #####end database loop
} ###end server loop

$Stopwatch.Stop()
$Stopwatch.Elapsed.TotalSeconds
$Stopwatch.Elapsed.TotalMinutes

Answer 1

One way is to load the rows into a DataTable and use .NET SqlBulkCopy directly in the script to bulk insert. Just change the DataTable column names and data types in the below example to match those of your actual table.

$dt = New-Object System.Data.DataTable;
[void]($dt.Columns.Add("server_name", [System.Type]::GetType("System.String")).MaxLength = 128)
[void]($dt.Columns.Add("name1", [System.Type]::GetType("System.String")).MaxLength = 128)
[void]($dt.Columns.Add("tname1", [System.Type]::GetType("System.String")).MaxLength = 128)
[void]($dt.Columns.Add("today", [System.Type]::GetType("System.DateTime")))
[void]($dt.Columns.Add("trows1", [System.Type]::GetType("System.Int64")))
[void]($dt.Columns.Add("tspace1", [System.Type]::GetType("System.Int64")))
[void]($dt.Columns.Add("tindexsp1", [System.Type]::GetType("System.Int64")))

$Stopwatch = [System.Diagnostics.Stopwatch]::StartNew()
$today = Get-Date
$srvlist = @(Get-Content "c:\scripts\tablegrowth.TXT")
foreach ($server in $srvlist) {
    $srv = New-Object "Microsoft.SqlServer.Management.SMO.Server" $server
    $dbs = $srv.Databases
    foreach ($db in $dbs) {
        if ($db.IsAccessible) {
            $name1 = $db.name
            $size1 = $db.size
            $dbspace1 = $db.DataSpaceUsage/1KB
            $dbindexsp1 = $db.IndexSpaceUsage/1KB
            $dbspaceavail1 = $db.SpaceAvailable/1KB
            #Write-Host "dbname=" $name1 $size1 $dbspace1 $dbindexsp1 $dbspaceavail1 $db.IsAccessible
            switch ($name1) {
                'master' {}
                'msdb' {}
                'model'  {}
                'SSISDB'  {}
                'SSRSData'  {}
                'Northwind' {}
                'tempdb' {}
                default {
                    # Invoke-Sqlcmd -ServerInstance "******" -Database "DBAMonitoring" -Query "INSERT INTO is_sql_databases VALUES ('$server','$name1','$today',$dbspace1,$dbspaceavail1,$dbindexsp1) "
                    foreach ( $tbl in $db.tables) {
                        $tname1    = $tbl.Name
                        $tindexsp1 = $tbl.IndexSpaceUsed/1KB
                        $trows1    = $tbl.RowCount
                        $tspace1   = $tbl.DataSpaceUsed/1KB
                        #Write-Host "tbl name=" $tname1 $tindexsp1 $trows $tspace1
                        if ($trows1 -gt 500) {
                            # Invoke-Sqlcmd -ServerInstance "." -Database "tempdb" -Query "INSERT INTO is_sql_tables VALUES ('$server','$name1','$tname1','$today',$trows1,$tspace1,$tindexsp1) "
                            $row = $dt.NewRow()
                            $dt.Rows.Add($row)
                            $row["server_name"] = $server
                            $row["name1"] = $name1
                            $row["tname1"] = $tname1
                            $row["today"] = $today
                            $row["trows1"] = $trows1
                            $row["tindexsp1"] = $tindexsp1
                        } ####end RowCount > 0
                    } #####end Row loop
                } ###### end Default
            } #####end switch
        }  #####end IsAccessible
    }  #####end database loop
} ###end server loop

$bcp = New-Object System.Data.SqlClient.SqlBulkCopy("Data Source=******;Integrated Security=SSPI;Initial Catalog=DBAMonitoring ")
$bcp.DestinationTableName = "dbo.is_sql_tables"
$bcp.WriteToServer($dt);

$Stopwatch.Stop()
$Stopwatch.Elapsed.TotalSeconds
$Stopwatch.Elapsed.TotalMinutes