Why does a WCF service execute query all before persisting a new entity?

Question

I'm working on a bigger project which uses WCF data service and OData to communicate between the frontend and the backend and we have performance problems when creating entities. Digging deeper into the problem uncovered that when the client tries to save a new entity then a query is executed in the service which returns all entities and it is discarded and then the data is inserted.

I was thinking that the problem is related how our application was created. It uses a custom provider, but the strange thing is that the problem occurs even with the simplest test project.

I have used the following code posted below and I set breakpoints inside Provider.cs. When I invoke it with HttpRequester Firefox plugin to send a POST request (insert) according to OData then

1. IQueryable gets called
2. CreateResource is called
3. SetValue is called several times
4. SaveChanges is called

I have the question why the IQueryable gets called and how it should be prevented? I was not able to figure that out.

In our real life scenario and not in this test app the IQueryable returns several thousand records, or even more, and can be time consuming. This impacts the performance of insert operations.

Request to insert a record

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xml:base="http://localhost:50366/MyDataService.svc/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom">
<id>http://localhost:50366/MyDataService.svc/Employees(0)</id>
<title type="text"></title>
<updated>2012-07-31T18:03:45Z</updated>
<author>
  <name />
</author>
<link rel="edit" title="Employee" href="Employees(0)" />
<category term="Test.Dto.Employee" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
<content type="application/xml">
  <m:properties>
    <d:EmployeeID m:type="Edm.Int32">2</d:EmployeeID>
    <d:LastName>Test</d:LastName>
    <d:FirstName>Data</d:FirstName>
  </m:properties>
</content>
</entry>

MyDataService.svc.cs:

using System;
using System.Collections.Generic;
using System.Data.Services;
using System.Data.Services.Common;
using System.Linq;
using System.ServiceModel.Web;
using System.Web;

namespace Test
{
    public class MyDataService : DataService<Provider>
    {
        // This method is called only once to initialize service-wide policies.
        public static void InitializeService(DataServiceConfiguration config)
        {
            config.SetEntitySetAccessRule("*", EntitySetRights.All);
            config.SetServiceOperationAccessRule("*", ServiceOperationRights.All);
            config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
        }
    }
}

Employee.cs

using System.Data.Services.Common;

namespace Test.Dto
{

    [DataServiceKey("EmployeeID")]
    public class Employee
    {
        public virtual int EmployeeID { set; get; }
        public virtual string LastName { set; get; }
        public virtual string FirstName { set; get; }
    }

}

Provider.cs

using System.Linq;
using Test.Dto;
using System.Collections.Generic;
using System.Data.Services;
namespace Test
{

    public class Provider : IUpdatable
    {
        static IList<Employee> _employees = new List<Employee>() {
            new Employee {
                EmployeeID = 1,
                FirstName = "No",
                LastName = "Name"
            }
        };

        IList<Employee> _updates = new List<Employee>();

        public IQueryable<Employee> Employees 
        { 
            get 
            {
                return Provider._employees.AsQueryable();
            } 
        }

        public void AddReferenceToCollection(object targetResource, string propertyName, object resourceToBeAdded)
        {
            throw new System.NotImplementedException();
        }

        public void ClearChanges()
        {
            _updates.Clear();
        }

        public object CreateResource(string containerName, string fullTypeName)
        {
            if (Equals(fullTypeName, typeof(Employee).FullName))
            {
                var entity = new Employee();
                _updates.Add(entity);
                return entity;
            }
            else
            {
                throw new System.NotImplementedException();
            }                              
        }

        public void DeleteResource(object targetResource)
        {
            throw new System.NotImplementedException();
        }

        public object GetResource(IQueryable query, string fullTypeName)
        {
            throw new System.NotImplementedException();
        }

        public object GetValue(object targetResource, string propertyName)
        {
            throw new System.NotImplementedException();
        }

        public void RemoveReferenceFromCollection(object targetResource, string propertyName, object resourceToBeRemoved)
        {
            throw new System.NotImplementedException();
        }

        public object ResetResource(object resource)
        {
            throw new System.NotImplementedException();
        }

        public object ResolveResource(object resource)
        {
            return resource;
        }

        public void SaveChanges()
        {
            foreach (var item in _updates)
            {
                _employees.Add(item);                
            }
        }

        public void SetReference(object targetResource, string propertyName, object propertyValue)
        {
            throw new System.NotImplementedException();
        }

        public void SetValue(object targetResource, string propertyName, object propertyValue)
        {
            targetResource.GetType().GetProperty(propertyName).SetValue(targetResource, propertyValue, null);
        }
    }

}

Stack trace when breaking in IQueryable

>    WebApplication5.DLL!Test.Provider.Employees.get() Line 24    C#
    [Lightweight Function]    
    System.Data.Services.dll!System.Data.Services.Providers.ReflectionServiceProvider.GetResourceContainerInstance(System.Data.Services.Providers.ResourceSet resourceContainer) + 0x1ec bytes    
    System.Data.Services.dll!System.Data.Services.Providers.BaseServiceProvider.GetQueryRootForResourceSet(System.Data.Services.Providers.ResourceSet container) + 0xb bytes    
    System.Data.Services.dll!System.Data.Services.RequestUriProcessor.CreateFirstSegment(System.Data.Services.IDataService service, string identifier, bool checkRights, string queryPortion, bool isLastSegment, out bool crossReferencingUrl) + 0x40e bytes    
    System.Data.Services.dll!System.Data.Services.RequestUriProcessor.CreateSegments(string[] segments, System.Data.Services.IDataService service) + 0x103 bytes    
    System.Data.Services.dll!System.Data.Services.RequestUriProcessor.ProcessRequestUri(System.Uri absoluteRequestUri, System.Data.Services.IDataService service) + 0x3b bytes    
    System.Data.Services.dll!System.Data.Services.DataService<Test.Provider>.ProcessIncomingRequestUri() + 0xe2 bytes    
    System.Data.Services.dll!System.Data.Services.DataService<Test.Provider>.HandleRequest() + 0xc0 bytes    
    System.Data.Services.dll!System.Data.Services.DataService<Test.Provider>.ProcessRequestForMessage(System.IO.Stream messageBody) + 0x65 bytes    
    [Lightweight Function]    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(object instance, object[] inputs, out object[] outputs) + 0x33f bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(ref System.ServiceModel.Dispatcher.MessageRpc rpc) + 0x137 bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(ref System.ServiceModel.Dispatcher.MessageRpc rpc) + 0x5e bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage41(ref System.ServiceModel.Dispatcher.MessageRpc rpc) + 0x6c bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(ref System.ServiceModel.Dispatcher.MessageRpc rpc) + 0x89 bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(ref System.ServiceModel.Dispatcher.MessageRpc rpc) + 0x59 bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(ref System.ServiceModel.Dispatcher.MessageRpc rpc) + 0x3b bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(ref System.ServiceModel.Dispatcher.MessageRpc rpc) + 0x4e bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(ref System.ServiceModel.Dispatcher.MessageRpc rpc) + 0x125 bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(ref System.ServiceModel.Dispatcher.MessageRpc rpc) + 0x34 bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.MessageRpc.Process(bool isOperationContextSet) + 0xff bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(System.ServiceModel.Channels.RequestContext request, bool cleanThread, System.ServiceModel.OperationContext currentOperationContext) + 0x44b bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(System.ServiceModel.Channels.RequestContext request, System.ServiceModel.OperationContext currentOperationContext) + 0x127 bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(System.IAsyncResult result) + 0x43 bytes    
    System.ServiceModel.dll!System.ServiceModel.Dispatcher.ChannelHandler.OnAsyncReceiveComplete(System.IAsyncResult result) + 0x44 bytes    
    System.Runtime.DurableInstancing.dll!System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult result) + 0x32 bytes    
    System.Runtime.DurableInstancing.dll!System.Runtime.AsyncResult.Complete(bool completedSynchronously) + 0xfd bytes    
    System.Runtime.DurableInstancing.dll!System.Runtime.InputQueue<System.ServiceModel.Channels.RequestContext>.AsyncQueueReader.Set(System.Runtime.InputQueue<System.ServiceModel.Channels.RequestContext>.Item item) + 0x44 bytes    
    System.Runtime.DurableInstancing.dll!System.Runtime.InputQueue<System.ServiceModel.Channels.RequestContext>.EnqueueAndDispatch(System.Runtime.InputQueue<System.ServiceModel.Channels.RequestContext>.Item item, bool canDispatchOnThisThread) + 0x1aa bytes    
    System.Runtime.DurableInstancing.dll!System.Runtime.InputQueue<System.ServiceModel.Channels.RequestContext>.EnqueueAndDispatch(System.ServiceModel.Channels.RequestContext item, System.Action dequeuedCallback, bool canDispatchOnThisThread) + 0x5e bytes    
    System.ServiceModel.dll!System.ServiceModel.Channels.SingletonChannelAcceptor<System.ServiceModel.Channels.IReplyChannel,System.ServiceModel.Channels.ReplyChannel,System.ServiceModel.Channels.RequestContext>.Enqueue(System.ServiceModel.Channels.RequestContext item, System.Action dequeuedCallback, bool canDispatchOnThisThread) + 0x6b bytes    
    System.ServiceModel.dll!System.ServiceModel.Channels.HttpChannelListener.HttpContextReceived(System.ServiceModel.Channels.HttpRequestContext context, System.Action callback) + 0x1b4 bytes    
    System.ServiceModel.Activation.dll!System.ServiceModel.Activation.HostedHttpTransportManager.HttpContextReceived(System.ServiceModel.Activation.HostedHttpRequestAsyncResult result) + 0xd6 bytes    
    System.ServiceModel.Activation.dll!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest() + 0x232 bytes    
    System.ServiceModel.Activation.dll!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest() + 0x27 bytes    
    System.ServiceModel.Activation.dll!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequest(object state) + 0x49 bytes    
    System.ServiceModel.Activation.dll!System.ServiceModel.AspNetPartialTrustHelpers.PartialTrustInvoke(System.Threading.ContextCallback callback, object state) + 0x35 bytes    
    System.ServiceModel.Activation.dll!System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequestWithFlow(object state) + 0x7a bytes    
    System.Runtime.DurableInstancing.dll!System.Runtime.IOThreadScheduler.ScheduledOverlapped.IOCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* nativeOverlapped) + 0x78 bytes    
    System.Runtime.DurableInstancing.dll!System.Runtime.Fx.IOCompletionThunk.UnhandledExceptionFrame(uint error, uint bytesRead, System.Threading.NativeOverlapped* nativeOverlapped) + 0x39 bytes    
    mscorlib.dll!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint errorCode, uint numBytes, System.Threading.NativeOverlapped* pOVERLAP) + 0x74 bytes    
    [Native to Managed Transition]    
    [Appdomain Transition]    
    [Native to Managed Transition]    

Answer 1

There are a couple of issues here:

1. We have a bug in WCF DS. It was tough to consider fixing this while we were in the .NET Framework because of the potential for breaking changes. Now that we are out of the Framework and we have a public ask, we should fix this.
2. We don't enumerate the IQueryable, which means that the QueryProvider will never actually execute the query. As long as the getter is just returning an IQueryable and not doing any work, we shouldn't see any more than a nominal performance hit from this.

All of that said, it is something we can and should fix so that we don't leave people wondering about it. We'll try to get a fix out in 5.0.2 or 5.1.0 if we have the bandwidth to do so.