EF 6 not detecting existing subclass when creating a new parent class

Question

I have a Topic class:

 public class Topic : BaseEntity
    {        
        public string Name { get; set; }
        public bool Active { get; set; }
        public IList<Content>? Contents { get; set; }
    }

And a Content class:

public class Content : BaseEntity
    {
        public string Name { get; set; }
        public string URL { get; set; }
        public string StartingVersion { get; set; }
        public string EndingVersion { get; set; }
        public string Summary { get; set; }
        public bool Active { get; set; }
        public IList<Topic> Topics { get; set; }
}

The BaseEntity looks like this:

   public class BaseEntity
    {
        [Key]
        [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
        public int Id { get; set; }
        public int CreatedBy { get; set; }
        public DateTime CreatedDate { get; set; }
        public int ModifiedBy { get; set; }
        public DateTime ModifiedDate { get; set; }
    }

My DataContext looks like this:

   public DataContext(DbContextOptions<DataContext> options) : base(options) { }

        private DbSet<Topic> Topics { get; set; }
        private DbSet<Content> Contents { get; set; }
       
        protected override void OnModelCreating(ModelBuilder builder)
        {
           base.OnModelCreating(builder);
        }

And I'm trying to use a generic Repository. The saveEntity looks like this:

 public async Task<T> SetEntity<T>(T entity) where T : BaseEntity
        {
            using (var scope = _serviceProvider.CreateScope())
            {
                entity.CreatedDate = DateTime.Now;
                var _dbContext = scope.ServiceProvider.GetRequiredService<DataContext>();
                _dbContext.Add(entity);
                await _dbContext.SaveChangesAsync();
                return entity;
            }
        }

And the Content Service method that does the creation of Contents looks like this:

    public async Task<ContentDTO> AddContentAsync(ContentDTO content)
        {
            _modelHelper.ModelValidation(content);
            await Checks(content, false);
            foreach (var item in content.Topics)
            {
                Expression<Func<Topic, bool>> expTopic = i => i.Id == item.Id && i.Active == true;
                var topic = await _dataRepository.GetEntityAsync(expTopic);
                if (topic == null)
                {
                    throw new KeyNotFoundException($"Topic with ID {item.Id} not found");
                }
            }

            Content toSaveContent = Mapping.Mapper.Map<Content>(content);
            toSaveContent.Active = true;
            Content newContent = await _dataRepository.SetEntity(toSaveContent);
            return Mapping.Mapper.Map<ContentDTO>(newContent);
        }

My problem is that when I try to create a new Content EF fails to detect that the Topics included in the body of the Content are existing ones and tries to add them as new in the DB. Obviously, this raises a SQL exception saying I can't define the Id of the Topic.

What I'm missing??

Thank you for your help

EDIT: Also tried to retrieve the Topics from context, but didn't work either:

public async Task<ContentDTO> AddContentAsync(ContentDTO content)
        {
            Expression<Func<Content, bool>> exp = i => i.URL == content.URL && i.Active == true;
            if (_dataRepository.GetEntities(exp).Any())
            {
                throw new DuplicateWaitObjectException("Object already exist");
            }

            CheckObjectives(content.Objectives);

            Content toSaveContent = Mapping.Mapper.Map<Content>(content);
            _modelHelper.ModelValidation(toSaveContent);
            toSaveContent.Active = true;

            toSaveContent.Topics = new List<Topic>();
            foreach (var item in content.Topics)
            {
                Expression<Func<Topic, bool>> expTopic = i => i.Id == item.Id && i.Active == true;
                var topic = await _dataRepository.GetEntity(expTopic);
                if(topic == null)
                {
                    throw new KeyNotFoundException($"Topic with ID {item.Id} not found");
                }
                toSaveContent.Topics.Add(topic);
            }

            Content newContent = await _dataRepository.SetEntity(toSaveContent);

            return Mapping.Mapper.Map<ContentDTO>(newContent);
        }

EDIT2: You are right, Guru Stron, I'll extract the GetEntity from the foreach and just take them all before. This is my GetEntity method in the generic repository:

public async Task<T> GetEntity<T>(Expression<Func<T, bool>> predicate) where T : BaseEntity
        {
            using (var scope = _serviceProvider.CreateScope())
            {
                var _dbContext = scope.ServiceProvider.GetRequiredService<DataContext>();
                return _dbContext.Set<T>().Where(predicate).FirstOrDefault();
            }
        }

EDIT3: I'm sorry for the long delay. I'm not sure if this is a context issue. When I try to save a Content with the following JSON:

{
  "name": "Style",
  "url": "https://player.vimeo.com/video/41513143?h=6215248d63",
  "startingVersion": "3.10.1",
  "endingVersion": "3.10.1",
  "summary": "This is a very nice content",
  "topics": [
    {
      "id": 2,
      "name": "NewTopic"
    }
  ],
  "objectives": [
    {
      "id": 1,
      "index": 1,
      "description": "This is a nice Objective"
    }
  ]
}

I can see in the saving method of the repository that the Topic with ID 2 indeed exists:

It looks like the object Topic with Id 2 exists in the context but EF can't find it??

EDIT4: Edited for clarity

EDIT 5: Tried to add the DataContext as Scoped in the ServiceCollection, and inject it in the Repository:

  public static IServiceCollection AddDependencyInjectionConfiguration(this IServiceCollection services)
        {
            services.AddScoped<IDataRepository, DataRepository>();
            services.AddScoped<DataContext>();
[...]
}

Used DI in the Repository and removed Scopes for using the DataContext:

[...]
     public DataRepository(IServiceProvider serviceProvider, IHttpContextAccessor contextAccesor, DataContext dataContext)
        {
            _serviceProvider = serviceProvider;
            _httpContextAccessor = contextAccesor;
            _dbContext = dataContext;
        }
[...]
    public async Task<T> SetEntity<T>(T entity) where T : BaseEntity
        {

                entity.CreatedDate = DateTime.UtcNow;
                entity.CreatedBy = _currentUserId;
                _dbContext.Add(entity);                
                await _dbContext.SaveChangesAsync();
                return entity;
        }
[...]

And removed the Topic search in the service method to avoid the exception of "object already use in reading operation"

     public async Task<ContentDTO> AddContentAsync(ContentDTO content)
        {
            _modelHelper.ModelValidation(content);
            await Checks(content, false);
            Content toSaveContent = Mapping.Mapper.Map<Content>(content);
            toSaveContent.Active = true;
            Content newContent = await _dataRepository.SetEntity(toSaveContent);
            return Mapping.Mapper.Map<ContentDTO>(newContent);
        }

But the result is still the same... EF is trying to save the Topic...

EDIT 6:

I tried to update Topics before saving the Content, but it is still trying to save the same Topic:

  public async Task<ContentDTO> AddContentAsync(ContentDTO content)
        {
            await Checks(content, false);
            Content toSaveContent = Mapping.Mapper.Map<Content>(content);
            _modelHelper.ModelValidation(content);
            toSaveContent.Active = true;

            foreach (var item in content.Topics)
            {
                Topic? topic = await _dataRepository.GetEntityAsync<Topic>(x => x.Id == item.Id);
                if (topic == null)
                {
                    throw new KeyNotFoundException($"Topic with ID {item.Id} not found");
                }
                if (topic.Contents == null) {
                    topic.Contents = new List<Content>() { toSaveContent };
                }
                else {
                    topic.Contents.Add(toSaveContent);
                }

                await _dataRepository.UpdateEntityAsync(topic, topic.Id);
            }

            Content newContent = await _dataRepository.SetEntity(toSaveContent);
            return Mapping.Mapper.Map<ContentDTO>(newContent);
        }

EDIT 7: As @rjs123431 suggested I cleared the Topics list of the Content object to save and stored the reference to the Content in the Topics and updated the objects.

   public async Task<ContentDTO> AddContentAsync(ContentDTO content)
        {
            await Checks(content, false);
            _modelHelper.ModelValidation(content);

            Content toSaveContent = Mapping.Mapper.Map<Content>(content);
            toSaveContent.Active = true;
            toSaveContent.Topics = new List<Topic>();

            List<Topic> topicsToSave = new List<Topic>();
            foreach (var item in content.Topics)
            {
                Expression<Func<Topic, bool>> expTopic = i => i.Id == item.Id && i.Active == true;
                var topic = await _dataRepository.GetEntityAsync(expTopic);
                if (topic == null)
                {
                    throw new KeyNotFoundException($"Topic with ID {item.Id} not found");
                }else
                {
                    if (topic.Contents == null)
                        topic.Contents = new List<Content>() { toSaveContent };
                    else
                        topic.Contents.Add(toSaveContent);
                    topicsToSave.Add(topic);
                }
            }
            await _dataRepository.UpdateEntitiesAsync(topicsToSave);

            Content newContent = await _dataRepository.SetEntity(toSaveContent);
            return Mapping.Mapper.Map<ContentDTO>(newContent);
        }

But with this code, the Content is saved, but in the ContentTopic table nothing is saved, therefore I lose the reference to the Topics.

Answer 1

Since you have a many-to-many relationship, and want to link your content to topics that are already save, you should not add topic to your content.

            foreach (var item in content.Topics)
            {
                Expression<Func<Topic, bool>> expTopic = i => i.Id == item.Id && i.Active == true;
                var topic = await _dataRepository.GetEntity(expTopic);
                if(topic == null)
                {
                    throw new KeyNotFoundException($"Topic with ID {item.Id} not found");
                }
                //toSaveContent.Topics.Add(topic);   // no need for this line
            }

            Content newContent = await _dataRepository.SetEntity(toSaveContent);

Instead, after you save your content, loop through the topics and add the newly saved content to it and update the topic so content will be linked to that topic and vice versa.

Something like this:

            foreach (var topic in content.Topics)
            {
                var topicEntity = await _topicRepository.GetAllIncluding(x => x.Contents)
                    .FirstOrDefaultAsync(x => x.Id == topic.Id);

                if (topicEntity != null)
                {
                    topicEntity.Contents.Add(content);

                    await _topicRepository.UpdateAsync(topicEntity);
                }
            }

Update 2:

You can even get the topic and add the content to it without having to save the content first. Content should have an empty topics of course.

Answer 2

EF Core uses concept of change tracking to manage data changes.

You should not create scope inside you generic repository (assuming you have default scoped context registration) - each scope will have it's own database context with it's own tracking, so the context which performs saving will have no idea about related entities and consider them as new ones (as you observe).

Usual approach is to have the outside control to control over scope, for example in ASP.NET Core the framework will create a scope on per request level and usually the dbcontext is shared on per request/scope basis.

So you need to remove the manual scope handling in the repository and use constructor injection so the repository shares the change tracking information between get and save queries, otherwise you will need to write some cumbersome code which will find and attach existing related entities in all the navigation properties of the saved entity.