MSTest tests succeed if run in isolation, fail if run with other tests

Question

I've bumped into this really odd situation where, if I run all my tests together, certain tests will fail (about 7 of them). But if I run just the tests inside the class (they are all part of the same class), then they pass. The test project is a Windows Phone 8.1 MSTest, and I tried running it both with Resharper test runner and MSTest test runner and they both show the same problem. Here's my TestInitialize code:

[TestInitialize]
public void Init()
{
     ResolveDependencies();
     var adsApiService = ServiceLocator
         .Current
         .GetInstance<IApiService<ListAdsReply, PublicAdsEndPoint>>();            
     var navigationService = new NavigationServiceMock();
     var mainPageTrackingService = ServiceLocator
         .Current
         .GetInstance<IMainPageTrackingService>();
     var adInsertionTrackingService = ServiceLocator
         .Current
         .GetInstance<IAdInsertionTrackingService>();
     var connectionService = ServiceLocator
         .Current
         .GetInstance<IConnectionService>();
     _windowsApiService = new WindowsApiServiceMock();
     var contactAboutTrackingService = ServiceLocator
         .Current
         .GetInstance<IContactAboutTrackingService>();
     var filtersTrackingService = ServiceLocator
         .Current
         .GetInstance<IFiltersTrackingService>();
            var filtersService = ServiceLocator
         .Current
         .GetInstance<IFiltersService>();
     var messageHelperMock = new MessageHelperMock();
     _mainPageViewModel = new MainPageViewModel(adsApiService, navigationService, mainPageTrackingService, adInsertionTrackingService, connectionService, _windowsApiService, contactAboutTrackingService,filtersTrackingService, filtersService, messageHelperMock);
}

The ResolveDependencies method doesn't do anything special besides registering dependencies in a Unity container, registering it with ServiceLocator.SetLocatorProvider and doing some Automapper configurations. No async code there.

[TestMethod]
public async Task GivenParameterIsProvidedThenFetchDataShouldReturnValidData()
{
    _mainPageViewModel
        .SearchParams
        .Add(new KeyValuePair<string, string>("lim", "5"));
    var searchParams = _mainPageViewModel.SearchParams;
    await _mainPageViewModel.FetchData(searchParams);

    var list = _mainPageViewModel.AdsList;
    Assert.IsNotNull(list);
}

This is one of the failing tests.

public async Task<ListAdsReplyViewModel> FetchData(List<KeyValuePair<string, string>> parameters)
{
    _cancellationTokenSource = new CancellationTokenSource();

    _cancellationTokenSource
        .CancelAfter(Constants.TimeToCancelAsyncOperations);
    AddSearchKeywordToSearchParams();
    var result = await _listAdsReplyApiService
        .GetWithParametersAsync(_cancellationTokenSource, parameters);
    var vm = new ListAdsReplyViewModel
    {
        Ads = new List<AdInfoViewModel>()
    };
    foreach (var listAd in result.ListAds)
    {
        var listAdDto = Mapper.Map<ListAdDto>(listAd);
        var adInfo = new AdInfoViewModel(_navigationService, _mainPageTrackingService)
        {
            ListAdDto = listAdDto
        };
            vm.Ads.Add(adInfo);
     }
     vm.NextPage = result.NextPage;
     vm.ConfigEtag = result.ConfigEtag;
     vm.Sorting = result.Sorting;
     TotalAds = result.ListAdsCounterMap.AllAds;
     return vm;
}

private void AddSearchKeywordToSearchParams()
{
     if (!string.IsNullOrEmpty(SearchKeyWord))
     {
         var searchKeyword = SearchParams
             .FirstOrDefault(x => x.Key == "q");
         if (!searchKeyword.Equals(null))  
             SearchParams.Remove(searchKeyword);
         SearchParams.Add(new KeyValuePair<string, string>("q", SearchKeyWord));
     }
}

And this are the methods under test. The problem seems to happen when foreach((var listAd in result.ListAds) is called. It's like GetWithParametersAsync(_cancellationTokenSource, parameters); is not being awaited, because I get the following error for the failing tests:

Test method App.Tests.Integration.App.Shared.ViewModels.MainPageViewModelTests.GivenParameterIsProvidedThenFetchDataShouldReturnValidData threw exception: 
AutoMapper.AutoMapperMappingException: 

Mapping types:
ListAd -> ListAdDto
Core.Api.Models.PublicAds.ListAd -> Core.Dto.ListAdDto

Destination path:
ListAdDto

Source value:
Core.Api.Models.PublicAds.ListAd ---> System.NullReferenceException: Object reference not set to an instance of an object.

    at Core.Bootstrap.AutoMapperConfiguration.<>c__DisplayClass0_0.<Configure>b__12(ListAd src, ListAdDto dest)
   at AutoMapper.Internal.MappingExpression`2.<>c__DisplayClass57_0.<AfterMap>b__0(Object src, Object dest)
   at AutoMapper.TypeMap.<get_AfterMap>b__40_0(Object src, Object dest)
   at AutoMapper.Mappers.TypeMapObjectMapperRegistry.PropertyMapMappingStrategy.Map(ResolutionContext context, IMappingEngineRunner mapper)
   at AutoMapper.Mappers.TypeMapMapper.Map(ResolutionContext context, IMappingEngineRunner mapper)
   at AutoMapper.MappingEngine.AutoMapper.IMappingEngineRunner.Map(ResolutionContext context)
 --- End of inner exception stack trace ---
    at App.ViewModels.MainPageViewModel.<FetchData>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at App.Tests.Integration.App.Shared.ViewModels.MainPageViewModelTests.<GivenParameterIsProvidedThenFetchDataShouldReturnValidData>d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

Any thoughts?

Answer 1

As this was accepted in the comments, I've reposted as an Answer;

"Have you taken into account that MSTest will run your tests multi-threaded - and that by executing a single test you are forcing a single-thread ? Be very careful that all your tests are UNIT tests and have no dependency on each other, in the sequence of execution, or the possiblity that shared resources might have already been instantiated (or being in the process of instantiation) by another tests running in parallel. "