CODEWITHSUNDEEP
javahibernatespring-bootjpaspring-data-jpa
Ondrej Sotolar
Ondrej Sotolar

Reputation: 1442

Optimize JPA performance on inserting many entities with embedded primary key

I naively implemented a web service, that consumes a list of Json objects and stores them in a SQL database using springframework.data.jpa (JPA & Hibernate). However, the solution has a low performance and the profiler gave me a hint that the main problem lies in creating the entities from the Json objects one by one.

The code below is simplified, but basically: for each Json object in the incoming list there are two entities created: DataEntity and IdentityEntity. The former holds the data of interest and uses the latter as FK, which has a compound PK of a time and a person.

I'd like to speed up the storage process. I've determined with a profiler, that there are too many flush operations that are being done after insert of each new entity. Since I need to insert thousands of records at a given time, this causes the performance issue. Can I perhaps do the insert in one transaction or what are other ways of optimizing it?

Data class (I have many similar classes):

@Entity
public class DataEntity {
    @EmbeddedId
    private IdentityEntity identity;
    private Double data;
}

Embeddable entity:

@Embeddable
public class IdentityEntity implements Serializable {
    @NonNull
    private Long personId;
    @NonNull
    private Long datetimeId;
}

JPA repository:

@Repository
public interface DataRepository extends JpaRepository<DataEntity, IdentityEntity> {}

Simplified controller:

public class DataController{
    @Autowired
    private DataRepository dataRepository;
    @Autowired
    private DatetimeRepository datetimeRepository;

    @PostMapping("/upload")
    public void upload(...List<DataJson> items) {
        PersonEntity person = getPerson(...);          // fast enough
        for (DataJson i : items) {                 // begin transaction here?
            saveNewEntity(i, person.getId());
        }
    }

    private void saveNewEntity(DataJson json, Long personId) {
        TimeEntity savedDatetime = datetimeRepository.save(new TimeEntity(json.getDatetime()));
        IdentityEntity mi = IdentityEntity(personId, savedDatetime.getId());

        DataEntity entry = new DataEntity(mi, json.getData());
        dataRepository.save(entry);
    }
}

Edit: After further digging into the profiler, I've discovered that another time-consuming operation might be the transaction management itself. Although I haven't implemented or configured any transaction behavior, I suspect that the Spring Boot has something default configured for the Hibernate ORM. I'm beginning to think that a transaction is now being created in every iteration of the loop, being the 1st performance issue and also causing the 2nd issue, where at the end of the transaction everything is flushed and written into the DB.

Upvotes: 1

Views: 1239

Answers (1)

Selindek
Selindek

Reputation: 3423

Yep. All the methods in the SimpleJpaRepository are annotated with @Transactional.

Simply add a @Transactional annotation to your upload method.

... or

First create all of the objects and save them in one go using the save(Iterable<S> entities) method.

Upvotes: 2

Related Questions