CODEWITHSUNDEEP
javaspringhibernatespring-batch
membersound
membersound

Reputation: 86637

How to bulk insert with Spring-Batch?

I'm trying to do bulk/batch inserts using spring-batch.

public ItemWriter<MyEntity> jpaItemWriter() {
    LocalSessionFactoryBuilder builder = new LocalSessionFactoryBuilder(ds);
    builder.addAnnotatedClasses(MyEntity.class);
    builder.setProperty("hibernate.show_sql", "true");
    builder.setProperty("hibernate.batch_size", "20");
    builder.setProperty("hibernate.order_updates", "true");
    builder.setProperty("hibernate.order_inserts", "true");
    HibernateItemWriter<MyEntity> writer = new HibernateItemWriter<>();
    writer.setSessionFactory(builder.buildSessionFactory());
    return writer;
}

Result: I'm getting only single insert statements, not bulk inserts! I can see it from the logs both of hibernate + on postgresql level. Why is the bulk insert not working?

Update:

@Entity
public class MyEntity {
    @Id
    String shortname;
    String fullname;
}

Upvotes: 1

Views: 4672

Answers (1)

outdev
outdev

Reputation: 5492

Spring has nothing to do with the SQL statements batching, its all managed by Hibernate.
I see you have batching enabled and configured properly, but that's not enough to make it work...you also need to use the right session type. in hibernate there are two session types: stateful session and stateless session.
The stateful session, which is obtained with 

sessionFactory.openSession();

and also is used by default if using @Transactional, never uses batching (even if configured) and sends all SQL statements, at once, at transaction commit. However, you can simulate batching by calling flush() from time to time, and SQL statements will be sent to the db on every flush().

The stateless session, which is obtained with

sessionFactory.openStatelessSession();

respects the batching configuration, so just switch to stateless session, and batching will works as expected. Hibernate will log every session.insert(), but will not sent the SQL insert statement to the database, instead the SQL insert statements are sent as batches of configured size. So its best to "tail -f" the database log. The main idea of having two session types is that, the stateful session uses cache, and every saved entity ends up in the 1st level cache, and therefore if you save 100k entities you will get OOM. The solution is to use stateless session which doesn't interact with any level cache. You can read more about the stateless session.

Upvotes: 3

Related Questions