CODEWITHSUNDEEP
javaspring-bootspring-data-jpaout-of-memorylazy-loading
Thach Huynh
Thach Huynh

Reputation: 1293

Using Spring data JPA EntityGraph with LAZY load mode for NamedAttributeNode field

I am facing with 2 problems: N + 1 query and Out Of Memory (OOM).

I solved OOM by paging and lazy loading:

@OneToMany(fetch = FetchType.LAZY)
@JoinColumn(name = "department_id")
private Set<Employee> employees;

But when I use lazy loading, N + 1 query happened. So I try to use EntityGraph as https://www.baeldung.com/spring-data-jpa-named-entity-graphs. But as my researches and local test, EntityGraph always do eager loading for NamedAttributeNode field - association field, which I want to be lazy loading - do not load all data at first:

@Entity
@Table(name = "department")
@NamedEntityGraph(name = "Department",
        attributeNodes = {
                @NamedAttributeNode("employees")
        }
)
public class Department implements Serializable {
    @OneToMany(fetch = FetchType.LAZY)
    @JoinColumn(name = "department_id")
    private Set<Employee> employees;
}

So are there any way to get them both ? Use EntityGraph to avoid N + 1 and lazy loading to avoid OOM ?

UPDATE: Can EntityGraph works fine with Pageable effectively ? I mean do not load all data in JOIN query.

Upvotes: 0

Views: 6064

Answers (3)

Alain Cruz
Alain Cruz

Reputation: 5097

A simple solution would be to fetch the entities and then initialize the needed joins.

Page<Department> depPages = this.repository.findAll(PageRequest.of(page, maxRows, sort));
for (Department d : depPages.getContent()) Hibernate.initialize(d.getEmployees());

Important: Must be run inside a transaction, otherwise you will get a can't initialize exception.

Upvotes: 0

Alexis Toulotte
Alexis Toulotte

Reputation: 1

A simple solution is:

  • Fetch first your records with pagination but without @EntityGraph.
  • Fetch your records with @EntityGraph from previous returned ids.

=> No N+1 complexity, only 2 SQL requests and no native queries, etc.

Example:

On your main repository, add @EntityGraph annotation with associations you want to fetch:

public interface CarRepository extends JpaRepository<Car, String>, JpaSpecificationExecutor<Car>, CarCustomRepository {

    @Override
    @EntityGraph(attributePaths = { "foo", "bar", "bar.baz" })
    List<Car> findAllById(Iterable<String> ids);

}

Create a custom repository with a findAllWithEntityGraph method:

public interface CarCustomRepository {

    Page<Car> findAllWithEntityGraph(Specification<Car> specification, Pageable pageable);

}

The implementation of the custom repository. It first fetch entities without entity graph, then it re-fetch them with entity graph in order to load associations. Don't forget to re-sort entities in to preserve order:

public class CarCustomRepositoryImpl implements CarCustomRepository {

    @Autowired
    @Lazy
    private CarRepository carRepository;

    @Override
    public Page<Car> findAllWithEntityGraph(Specification<Car> specification, Pageable pageable) {
        Page<Car> page = carRepository.findAll(specification, pageable);
        List<String> ids = page.getContent().stream().map(Car::getId).collect(Collectors.toList());
        List<Car> cars = carRepository.findAllById(ids).stream()
                .sorted(Comparator.comparing(car -> ids.indexOf(car.getId())))
                .collect(Collectors.toList());
        return new PageImpl<>(cars, pageable, page.getTotalElements());
    }

}

Then, you just have to invoke CarRepository#findAllWithEntityGraph method in order to fetch records with pagination and no N+1 complexity.

The question is: why hibernate does not have this behavior by default?

Upvotes: 0

Ruslan
Ruslan

Reputation: 6290

Using EntityGraph all your NamedAttributeNode associations will be loaded in 1 query with Join clause. Enable sql log to see how many queries hibernate does for loading entities in different scenarios

logging.level.org.hibernate.SQL=DEBUG

You will see that using @OneToMany(fetch = FetchType.EAGER) without EntityGraph it loads employees in separate select queries (N + 1), but using EntityGraph it performs only 1 select ... join

Also don't forget to specify entity graph name in repository like:

@EntityGraph(value = "Department")
List<Department> findAll();

UPDATE: Spring DATA Pagination doesn't work on database side. It will fetch all data and then filtered in memory. That's how it works.. There are some workarounds, check this links:

How can I avoid the Warning "firstResult/maxResults specified with collection fetch; applying in memory!" when using Hibernate?

Avoiding "HHH000104: firstResult/maxResults specified with collection fetch; applying in memory!" using Spring Data

VladMihalcea Blog The best way to fix the Hibernate HHH000104

As for me the solution could be creating custom repository and using EntityManager to construct query manually.

Upvotes: 2

Related Questions