Reputation: 1442
I naively implemented a web service, that consumes a list of Json objects and stores them in a SQL database using
springframework.data.jpa
(JPA & Hibernate). However, the solution has a low performance and the profiler gave me a hint that the main problem lies in creating the entities from the Json objects one by one.
The code below is simplified, but basically: for each Json object in the incoming list there are two entities created: DataEntity and IdentityEntity. The former holds the data of interest and uses the latter as FK, which has a compound PK of a time and a person.
I'd like to speed up the storage process. I've determined with a profiler, that there are too many flush operations that are being done after insert of each new entity. Since I need to insert thousands of records at a given time, this causes the performance issue. Can I perhaps do the insert in one transaction or what are other ways of optimizing it?
Data class (I have many similar classes):
@Entity
public class DataEntity {
@EmbeddedId
private IdentityEntity identity;
private Double data;
}
Embeddable entity:
@Embeddable
public class IdentityEntity implements Serializable {
@NonNull
private Long personId;
@NonNull
private Long datetimeId;
}
JPA repository:
@Repository
public interface DataRepository extends JpaRepository<DataEntity, IdentityEntity> {}
Simplified controller:
public class DataController{
@Autowired
private DataRepository dataRepository;
@Autowired
private DatetimeRepository datetimeRepository;
@PostMapping("/upload")
public void upload(...List<DataJson> items) {
PersonEntity person = getPerson(...); // fast enough
for (DataJson i : items) { // begin transaction here?
saveNewEntity(i, person.getId());
}
}
private void saveNewEntity(DataJson json, Long personId) {
TimeEntity savedDatetime = datetimeRepository.save(new TimeEntity(json.getDatetime()));
IdentityEntity mi = IdentityEntity(personId, savedDatetime.getId());
DataEntity entry = new DataEntity(mi, json.getData());
dataRepository.save(entry);
}
}
Edit: After further digging into the profiler, I've discovered that another time-consuming operation might be the transaction management itself. Although I haven't implemented or configured any transaction behavior, I suspect that the Spring Boot has something default configured for the Hibernate ORM. I'm beginning to think that a transaction is now being created in every iteration of the loop, being the 1st performance issue and also causing the 2nd issue, where at the end of the transaction everything is flushed and written into the DB.
Upvotes: 1
Views: 1239
Reputation: 3423
Yep. All the methods in the SimpleJpaRepository
are annotated with @Transactional
.
Simply add a @Transactional
annotation to your upload method.
... or
First create all of the objects and save them in one go using the save(Iterable<S> entities)
method.
Upvotes: 2