CODEWITHSUNDEEP
c#asp.net.net
A X
A X

Reputation: 1056

C# HashSet generating System.OutOfMemoryException sporadically but reproducibly

So we have two projects - a core project which we can call MainProject that has a function called LoadData() that loads 33,751,107 string text records into a HashSet one at a time, and a Unit Test project which we can call TestProject which runs a unit test against LoadData (it references MainProject).

If we run the unit test in TestProject, it passes and loads all 33,751,107 records successfully with no errors.

However, if we run the LoadData() function during startup of MainProject, it crashes with this exception:

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.

Interestingly, it crashes after successfully loading 23,986,243 records and then attempting to load the next batch.

This is very strange. How can the LoadData function crash in MainProject (which is an ASP.NET 4.6.1 web project) and NOT crash in TestProject (which is .NET Framework 4.7.2 unit test project, using MSTest framework) which simply runs the same LoadData function?

I even tried upgrading MainProject to ASP.NET 4.7.2 (which had no effect, still crashes). There is nothing special in web.config for MainProject, or in app.config for TestProject. I tried adding this to web.config:

<gcAllowVeryLargeObjects enabled="true"/>

This did not change or fix the problem. The behavior is not random, or intermittent - it happens reproducibly every time. I'm out of ideas.

Does anyone have any idea why this would happen?

Any help would be greatly appreciated.

Update: I tried creating a SuperHashSet class that creates a collection of multiple HashSets each with a max item count of 10,000,000 items. This dies with the same error, at exactly the same point, even though it is using ~3+ HashSets instead of one big one. Any ideas?

Upvotes: 0

Views: 216

Answers (1)

A X
A X

Reputation: 1056

Thanks to @GiddyaNaya for the answer - the issue was that the project was not building as x64. Once going through all the configuration settings, I found that deep down 2 levels it was still set to Any CPU. Thanks again for your help on this.

Upvotes: 1

Related Questions