SystemOutOfMemoryException while using the OutlookServicesClient

Question

I recently started using the new Outlook/Office 365 Rest API and got some great help regarding the authentication (new v2.0 app model), scopes, using the OutlookServicesClient and writing better (linq-)queries.

Unfortunately, some users reported some memory-issues, which resulted in the following System.OutOfMemoryException:

    Exception of type 'System.OutOfMemoryException' was thrown. :    at Microsoft.OData.Core.Json.JsonReader.ReadInput()
       at Microsoft.OData.Core.Json.JsonReader.ParseStringPrimitiveValue(Boolean& hasLeadingBackslash)
       at Microsoft.OData.Core.Json.JsonReader.ParseValue()
       at Microsoft.OData.Core.Json.JsonReader.Read()
       at Microsoft.OData.Core.Json.BufferingJsonReader.ReadInternal()
       at Microsoft.OData.Core.Json.BufferingJsonReader.ReadNextAndCheckForInStreamError()
       at Microsoft.OData.Core.Json.BufferingJsonReader.ReadInternal()
       at Microsoft.OData.Core.Json.BufferingJsonReader.Read()
       at Microsoft.OData.Core.JsonLight.ODataJsonLightDeserializer.ParseProperty(DuplicatePropertyNamesChecker duplicatePropertyNamesChecker, Func`2 readPropertyAnnotationValue, String& parsedPropertyName)
       at Microsoft.OData.Core.JsonLight.ODataJsonLightDeserializer.ProcessProperty(DuplicatePropertyNamesChecker duplicatePropertyNamesChecker, Func`2 readPropertyAnnotationValue, Action`2 handleProperty)
       at Microsoft.OData.Core.JsonLight.ODataJsonLightPropertyAndValueDeserializer.ReadComplexValue(IEdmComplexTypeReference complexValueTypeReference, String payloadTypeName, SerializationTypeNameAnnotation serializationTypeNameAnnotation, DuplicatePropertyNamesChecker duplicatePropertyNamesChecker)
       at Microsoft.OData.Core.JsonLight.ODataJsonLightPropertyAndValueDeserializer.ReadNonEntityValueImplementation(String payloadTypeName, IEdmTypeReference expectedTypeReference, DuplicatePropertyNamesChecker duplicatePropertyNamesChecker, CollectionWithoutExpectedTypeValidator collectionValidator, Boolean validateNullValue, Boolean isTopLevelPropertyValue, Boolean insideComplexValue, String propertyName, Nullable`1 isDynamicProperty)
       at Microsoft.OData.Core.JsonLight.ODataJsonLightEntryAndFeedDeserializer.ReadEntryDataProperty(IODataJsonLightReaderEntryState entryState, IEdmProperty edmProperty, String propertyTypeName)
       at Microsoft.OData.Core.JsonLight.ODataJsonLightEntryAndFeedDeserializer.ReadEntryPropertyWithValue(IODataJsonLightReaderEntryState entryState, String propertyName)
       at Microsoft.OData.Core.JsonLight.ODataJsonLightEntryAndFeedDeserializer.<>c__DisplayClass2.<ReadEntryContent>b__0(PropertyParsingResult propertyParsingResult, String propertyName)
       at Microsoft.OData.Core.JsonLight.ODataJsonLightDeserializer.ProcessProperty(DuplicatePropertyNamesChecker duplicatePropertyNamesChecker, Func`2 readPropertyAnnotationValue, Action`2 handleProperty)
       at Microsoft.OData.Core.JsonLight.ODataJsonLightEntryAndFeedDeserializer.ReadEntryContent(IODataJsonLightReaderEntryState entryState)
       at Microsoft.OData.Core.JsonLight.ODataJsonLightReader.ReadEntryStart(DuplicatePropertyNamesChecker duplicatePropertyNamesChecker, SelectedPropertiesNode selectedProperties)
       at Microsoft.OData.Core.JsonLight.ODataJsonLightReader.ReadAtEntryEndImplementationSynchronously()
       at Microsoft.OData.Core.JsonLight.ODataJsonLightReader.ReadAtEntryEndImplementation()
       at Microsoft.OData.Core.ODataReaderCore.ReadImplementation()
       at Microsoft.OData.Core.ODataReaderCore.ReadSynchronously()
       at Microsoft.OData.Core.ODataReaderCore.InterceptException[T](Func`1 action)
       at Microsoft.OData.Core.ODataReaderCore.Read()
       at Microsoft.OData.Client.Materialization.ODataReaderWrapper.Read()
       at Microsoft.OData.Client.Materialization.FeedAndEntryMaterializerAdapter.TryRead()
       at Microsoft.OData.Client.Materialization.FeedAndEntryMaterializerAdapter.TryStartReadFeedOrEntry()
       at Microsoft.OData.Client.Materialization.FeedAndEntryMaterializerAdapter.TryReadEntry(MaterializerEntry& entry)
       at Microsoft.OData.Client.Materialization.FeedAndEntryMaterializerAdapter.<LazyReadEntries>d__0.MoveNext()
       at Microsoft.OData.Client.Materialization.FeedAndEntryMaterializerAdapter.Read()
       at Microsoft.OData.Client.Materialization.ODataReaderEntityMaterializer.ReadNextFeedOrEntry()
       at Microsoft.OData.Client.Materialization.ODataEntityMaterializer.ReadImplementation()
       at Microsoft.OData.Client.MaterializeAtom.MoveNextInternal()
       at Microsoft.OData.Client.MaterializeAtom.MoveNext()
       at System.Linq.Enumerable.<CastIterator>d__94`1.MoveNext()
       at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
       at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
       at Microsoft.OData.ProxyExtensions.PagedCollection`2..ctor(DataServiceContextWrapper context, QueryOperationResponse`1 qor)
       at Microsoft.OData.ProxyExtensions.DataServiceContextWrapper.<>c__DisplayClass38`2.<ExecuteAsync>b__36(IAsyncResult r)
       at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.OData.ProxyExtensions.DataServiceContextWrapper.<ExecuteAsync>d__3a`2.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
       at Shared.Data.Office365.Office365Api.<GetEmailsReceived>d__23.MoveNext()

My query (to get the number of emails received for a date) looks like this:

var dtStart = date.Date.ToUniversalTime();
var dtEnd = date.Date.AddDays(1).ToUniversalTime();

var mailResults = await _client.Me.Messages
    .OrderByDescending(m => m.ReceivedDateTime)
    .Where(m => m.ReceivedDateTime.Value >= dtStart && m.ReceivedDateTime.Value <= dtEnd)
    //todo: filter if not in Junk Email and Deleted Folder (maybe with ParentFolderId)
    .Take(500) // todo: add paging to (and maybe make more efficient?)
    .Select(m => new DisplayEmail(m)).ExecuteAsync();

// todo: add paging to (and maybe make more efficient?)
foreach (var message in mailResults.CurrentPage)
{
    emails.Add(message);
}

I didn't find a way to dispose the sources and also don't think it's necessary, as there are just local variables. The result from the emails-list is then stored in the database. Memory usage for my tool was stable before I added this feature, and is now growing at around 10Mb per hour (depending on how many emails someone has).

Any suggestions are very much appreciated!

Answer 1

You might try a few things here:

• Reduce your page size. You're doing 500 at a time here (.Take(500)), you could try reducing that.
• Only request the properties you're interested in. If you modify your .Select to reference specific properties of your m variable, the API should use a $select query parameter in the API request to limit what comes back. See this tutorial for an example.