CODEWITHSUNDEEP
c#listmemorymemory-management
Axxelsian
Axxelsian

Reputation: 807

List.clear Memory Problems

I have a program that processes a bunch of csv files with many columns so that the user can pull out specific columns and write them to another csv file that's much smaller, and only contains what they require. Since there can be up to 8 million rows, I've created a queue system using a list, where it adds pieces(50000 lines at a time) to the list buffer so that the writer can write that piece, and then use listbuffer[i].clear(); to remove it, so that i am using much less memory, and also limit the buffer to only contain 10 items, keeping total memory use as low as i require it.

My problem is that thelistbuffer[i].clear(); is not letting go of the memory it previously used until the entire program is finished and I've reinitialized the buffer so that it is blank for the next use.

**listbuffer is aList<List<List<string>>>

Just so this makes a little more sense...

listbuffer[chunk of lines][row #][column #]

Is there a way to tell the list to let go of the memory that list[i] is using?

Thanks!

Upvotes: 5

Views: 12973

Answers (3)

Oswaldo Junior
Oswaldo Junior

Reputation: 59

You must clear the list, and after that invoke the GC to remove it from memory. As follows with extensive method:

public static void ClearMemory<T>(this List<T> lista)
    {
        int identificador = GC.GetGeneration(lista);
        lista.Clear();
        GC.Collect(identificador, GCCollectionMode.Forced);
    }

Upvotes: -2

Vitor - DotNerd
Vitor - DotNerd

Reputation: 41

maybe it's too late, but I wrote this extension to clear memory from large lists.

It worked for me and I hope it helps.

Extension:

public static void LiberarLista<T>(this List<T> lista)
{
     lista.Clear();            
     int identificador = GC.GetGeneration(lista);
     GC.Collect(identificador, GCCollectionMode.Forced);
}

Example:

List<YourObject> hugeList= LetsRockBigData();

// Your code logic here 
// Finish... Let's clear the huge list 

hugeList.LiberarLista();

I apologize for not being expert in English. Only one DotNerd from Brazil and that's all folks...

Upvotes: 3

Hans Passant
Hans Passant

Reputation: 941307

Yes, using List<T>.Clear() clears the references to all objects in the list and sets the Count property to 0. It does not release the reference to the underlying array that stores the object references. You can set the Capacity property to 0 after clearing it to release the reference to that array.

If you actually have OOM problems then doing this is not exactly a cure, it can create more address space fragmentation trouble. Check this answer for hints on how to avoid creating too much garbage in the Large Object Heap.

Upvotes: 9

Related Questions