vb.net managed memory and heaps are stable but process memory usage causes crash

Question

I have an app that reads a csv file with an image name, calls a sub that opens the image, scans to find a yellow pixel in a shape, and resolves the upper left corner, annotates the csv data and writes the record. Managed Heap and Native heap memory is stable, but process memory grows 350MB per image. For now I have my analysts chop the csv file into 25 image sets. But this is risky with 20-ish analysts.

I dispose the images, tried GCCollect(), large heapCompaction - nothing seems to help. I have read most every post and nothing seems to ring. I have included code - and tried to strip out displays and junk but left some of the scan comparisons.

I am running .net 4.6.1, Win 10/64, 16GB memory

Sub Process_CSV_Detail() . . . read csv file sequentially, set up file names, display for user, than call working process for each image - the memory leak seems to occur as each new image is accessed

    Call BackgroundProcess2()

End Sub

Public Sub BackgroundProcess2()

        GreenTest = 245
        RedTest = 245
        BlueTest = 70

    Try
        loadedImage = Image.FromFile(InputImageName)
    Catch ex As Exception
         . . . .Never gets here but some code 
    Finally
    End Try

    HeightVal = loadedImage.Height
    WidthVal = loadedImage.Width

    Dim MasterImage As New Bitmap(WidthVal, HeightVal, FormatVal)

    MasterImage = loadedImage

. . . Now looking for that pesky pixel

    For ycounter = 1 To HeightVal - 1 Step PixelStride
        For xcounter = 1 To WidthVal - 1 Step PixelStride

            Dim PixelColor As Color = MasterImage.GetPixel(xcounter, ycounter)
            PixelIsYellow = False


                If PixelColor.R > RedTest Then
                    If PixelColor.G > GreenTest Then
                        If PixelColor.B < BlueTest Then
                            PixelIsYellow = True
                            YellowPixelCount = YellowPixelCount + 1
                            MasterImage.SetPixel(xcounter, ycounter, Color.FromArgb(&HFFFFFF00))

                            xPixelIsYellow = True
                            yPixelIsYellow = True

                        End If
                    End If
                End If




            If PixelIsYellow = True Then
                'Now  find the upper left corner
                LeftXLoc = xcounter
                LeftYLoc = ycounter
                'Move to left until no more yellow, then back 1 step to 
                      'locate left pixel location
                Try
                    For xtestcounter = LeftXLoc To 1 Step -1
                        Dim PixelColorx As Color = MasterImage.GetPixel(xtestcounter, LeftYLoc)
                        If PixelColorx.R < RedTest Then
                            xPixelIsYellow = False
                            Exit Try
                        End If
                        If QA_Mode = False Then
                            If PixelColorx.G < GreenTest Then
                                xPixelIsYellow = False
                                Exit Try
                            End If
                        End If
                        If QA_Mode = True Then
                            If PixelColorx.G < GreenTest Then
                                xPixelIsYellow = False
                                Exit Try
                            End If
                        End If


                        If PixelColorx.B > 70 Then
                            xPixelIsYellow = False
                            Exit Try
                        End If

                    Next
                Catch ex As Exception
                Finally

                End Try
                LeftXLoc = xtestcounter + 1
                'Move up until no more yellow, then back 1 step to locate left pixel location
                Try
                    For ytestcounter = LeftYLoc To 1 Step -1
                        Dim PixelColory As Color = MasterImage.GetPixel(LeftXLoc, ytestcounter)
                        If PixelColory.R < RedTest Then
                            yPixelIsYellow = False
                            Exit Try
                        End If

                        If PixelColory.G < GreenTest Then
                            yPixelIsYellow = False
                            Exit Try
                        End If
                        If PixelColory.B > BlueTest Then
                            xPixelIsYellow = False
                            Exit Try
                        End If

                    Next
                Catch ex As Exception
                    MsgBox("Error in locating upper left pixel")
                Finally
                End Try
                LeftYLoc = ytestcounter + 1
                OutputLine = CurrentDataLine & "," & xcounter & "," & ycounter & "," & LeftXLoc & "," & LeftYLoc
                PrintLine(TargetFileNumber, OutputLine)
            End If
        Next
    Next
    loadedImage.Dispose()
    MasterImage.Dispose()
    ' - I have tried these but no effect 
    'GC.Collect()
    'Runtime.GCSettings.LargeObjectHeapCompactionMode = Runtime.GCLargeObjectHeapCompactionMode.CompactOnce
    'Finalize()
    End Sub

I expect that someone will have a silver bullet that allows process memory to be stable - Ive tried ANTS but no joy.

Answer 1

These two lines are (at least part of) the problem:

Dim MasterImage As New Bitmap(WidthVal, HeightVal, FormatVal)

MasterImage = loadedImage

You create a new bitmap with the specified dimensions and pixel format, then you immediately replace the MasterImage variable's reference to the new bitmap with that of your loadedImage. Now the new bitmap doesn't have any references, is not disposed and therefore lives on in memory until you close the process. And instead, MasterImage now refers to the exact same bitmap as loadedImage.

Thus, when your code disposes of the bitmaps, it is actually trying to dispose of the same bitmap twice:

loadedImage.Dispose()
MasterImage.Dispose() 'This is the exact same bitmap as loadedImage, which is already disposed of.

Image data in GDI+ is unmanaged memory, which is why the managed memory graph remains stable. Unmanaged memory is simply speaking any piece of memory that is not managed by a Garbage Collectior (GC), which is why calling any of its methods doesn't help. It must be freed manually by the programmer (in this case by calling Dispose()).

The solution is to just not create that new bitmap at all since you never actually use it. Remove the MasterImage variable altogether and operate on loadedImage directly.