CODEWITHSUNDEEP
springhibernatein-memory-database
Urbanleg
Urbanleg

Reputation: 6542

In memory database, with hibernate and periodically persisting to an actual db

I would like to use an in memory db with hibernate, so my queries are super quick. But moreover i would like to periodically persist that in memory state into a real mysql db. Ofcourse the in memory database should load its initial content on startup from that mysql db. Are there any good frameworks/practices for that purpose? (Im using spring) any tutorials or pointers will help.

Upvotes: 0

Views: 538

Answers (2)

Angular University
Angular University

Reputation: 43097

Actually the simplest would be to use some core Hibernate features for that, use the hibernate Session itself and combine it with the second level cache.

Declare the entities you want to cache as @Cacheable:

@Entity 
@Cacheable
@Cache(usage = CacheConcurrencyStrategy.NON_STRICT_READ_WRITE)
public class SomeReferenceData { ... }

Then implement the periodically flushing like this, supposing you are using JPA:

  • open an EntityManager
  • load the entities you want to cache using that entity manager and no other
  • Keep the entity manager opened until the next periodic flush, Hibernate is keeping track what instances of SomeReferenceData where modified in-memory via it's dirty checking mechanism, but no modification queries are being issued.
  • Reads on the database are being prevented via the second level cache
  • When the moment comes to flush the session, just begin a transaction and commit immediately.
  • Hibernate will update modified entities in the database, update the second level cache and resume execution
  • eventually close the entity manager and replace it with a new one, if you want to reload from the database eveything
  • otherwise keep the same entity manager open

code example:

Try this code to see the overall idea:

public class PeriodicDBSynchronizeTest {

    @Test
    public void testSynch() {

        // create the entity manager, and keep it
        EntityManagerFactory factory = Persistence.createEntityManagerFactory("testModel");

        EntityManager em = factory.createEntityManager();

        // kept in memory due to @Cacheable
        SomeReferenceData ref1 = em.find(SomeReferenceData.class, 1L);
        SomeReferenceData ref2 = em.find(SomeReferenceData.class, 2L);
        SomeReferenceData ref3 = em.find(SomeReferenceData.class, 3L);  
        ....  

        // modification are tracked but not committed
        ref1.setCode("005");

        // these two lines will flush the modifications into the database
        em.getTransaction().begin();
        em.getTransaction().commit();

       // continue using the ref data, and tracking modifications until the next request
       ...
    }
}

Upvotes: 0

Moo-Juice
Moo-Juice

Reputation: 38820

I'll be honest with you, most decent databases can be considered in-memory to an extent given that they cache data and try not to hit the hard-disk as often as they can. In my experience the best in-memory databases are either caches, or alagamations of other data sources that are already persisted in some other form, and then are updated in a live fashion for time-critical information, or refreshed periodically for non-time-critical information.

Loading data from a cold start in to memory is potentially a lengthy process, but subsequent queries are going to be super-quick.

If you are trying to cache what's already persisted you can look at memcache, but in essence in memory databases always rely on a more persistent source, be it MySQL, SQLServer, Cassandra, MongoDB, you name it.

So it's a little unclear what you're trying to achieve, suffice to say it is possible to bring data in from persistent databases and have a massive in memory cache, but you need to design around how stale certain data can get, and how often you need to hit the real source for up-to-the-second results.

Upvotes: 1

Related Questions