Hibernate L2C with Infinispan and JBoss EAP 7.0.6 is not working

Question

I've an application with Hibernate 5.0.9.Final deployed to JBoss EAP 7.0.6 (with Infinispan 8.0.1.Final as default). A couple of entities must be cached, they are annotated with JPA's @Cacheable and Hibernate's @Cache for supplying per-entity caching strategies.

The codebase contains a simple test that checks against Infinispan running locally alongside the testscase with direct access to the datasource, that is, there's no JBoss EAP in between, no Arquillian tests. The test runs successfully and I can see entities being fetched from the cache when available.

Now deploying the application to JBoss EAP and running results in a negative experience. I can see in the logs that Infinispan caches entities. I can also see on the JBoss Web Console the cache statistics. When a query that should result in cache hit counts is executed it turns out no hit counts occur and the query hits the datasource instead.

UPDATE: Both hibernate and infinispan statistics have been enabled using the appropriate properties in persistence.xml; and so are L2C and query cache. The region factory is set to JndiInfinispanRegionFactory, as explained at http://infinispan.org/docs/8.0.x/user_guide/user_guide.html. Also, all executed queries are JPQL queries. As far as I call tell Infinispan and JBoss AS/7 use their default configuration.

UPDATE 2: JBoss server is actually EAP 7.0.6, not AS/7 as previously stated.

We can't upgrade to a newer version of JBoss EAP at the moment (no Wildfly for now I'm afraid).

As an aside, attempts to run EhCache locally are successful but running inside JBoss EAP result in CNFE due to module problems.

persistence.xml

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
            xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
   <persistence-unit name="DefaultUnit" transaction-type="JTA">
       <provider>org.hibernate.jpa.HibernatePersistenceProvider</provider>
       <jta-data-source>java:/MyDataSource</jta-data-source>
       <class>...</class>
       <exclude-unlisted-classes>true</exclude-unlisted-classes>
       <shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>
       <properties>
           <property name="hibernate.dialect" value="org.hibernate.dialect.DB2Dialect" />
           <property name="hibernate.show_sql" value="true" />
           <property name="hibernate.connection.datasource" value="java:/MyDataSource"/>
           <property name="hibernate.generate_statistics" value="true" />
           <property name="hibernate.cache.infinispan.statistics" value="true"/>
           <property name="hibernate.transaction.manager_lookup_class" value="org.hibernate.transaction.JBossTransactionManagerLookup"/>
           <property name="hibernate.cache.use_second_level_cache" value="true" />
           <property name="hibernate.cache.use_query_cache" value="true" />
           <property name="hibernate.cache.region.factory_class" value="org.hibernate.cache.infinispan.JndiInfinispanRegionFactory"/>
           <property name="hibernate.cache.infinispan.cachemanager" value="java:jboss/infinispan/container/hibernate"/>
       </properties>
   </persistence-unit>
</persistence>

standalone.xml This block defines the hibernate container

<cache-container name="hibernate" default-cache="local-query" module="org.hibernate.infinispan">
    <local-cache name="entity">
        <transaction mode="FULL_XA"/>
        <eviction strategy="LRU" max-entries="10000"/>
        <expiration max-idle="100000"/>
    </local-cache>
    <local-cache name="local-query">
        <eviction strategy="LRU" max-entries="10000"/>
        <expiration max-idle="100000"/>
    </local-cache>
    <local-cache name="timestamps"/>
</cache-container>

Answer 1

When a query that should result in cache hit counts is executed it turns out no hit counts occur and the query hits the datasource instead.

When you say query, do you mean a JPQL query or Criteria API?

The entity query is meant for direct fetching by entity identifier. For queries, you'd need the Query Cache as well, but it's not clear if that's what you need.

Also, the hit count can be 0 if statistics are not enabled:

• Either, hibernate.generate_statistics which are related to Hibernate Core
• Or, hibernate.cache.infinispan.statistics, for Infinispan

One way to figure out what happens is to debug the DefaultLoadEventListener#doLoad method:

entity = loadFromSecondLevelCache( event, persister, keyToLoad );

And see why it can't be fetched from Infinispan.

Update:

Also, all executed queries are JPQL queries.

Did you enable the Query Cache?

Note that the Query Cache uses a single region so all your query results that use a certain table space are wiped out once you add/remove/modify any single entry belonging to that table space.

If you rely on the entity 2nd-level cache without using the Query Cache while trying to speed up JPQL, then it's going to work even less efficiently than without the 2nd-level cache. That's because the JPQL will execute the SQL query anyway, so you will just discard the result and go to the Cache to fetch entities by their ids. That can actually turn really bad if the entities are not found in the cache, meaning that a plethora of secondary queries will be issued.

The 2nd level cache makes a lot of sense if you try to offload the Master node while executing read-write transactions. For read-only queries, just use database replication and spread your load onto multiple Replicas whose Buffer Pools are configured to store the entire working set in RAM.

Answer 2

The persistence.xml configuration is incorrect. When deploying to EAP, this is all that's needed:

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
        xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
   <persistence-unit name="DefaultUnit" transaction-type="JTA">
       <provider>org.hibernate.jpa.HibernatePersistenceProvider</provider>
       <jta-data-source>java:/MyDataSource</jta-data-source>
       <class>...</class>
       <exclude-unlisted-classes>true</exclude-unlisted-classes>
       <shared-cache-mode>ENABLE_SELECTIVE</shared-cache-mode>
       <properties>
           <property name="hibernate.dialect" value="org.hibernate.dialect.DB2Dialect" />
           <property name="hibernate.show_sql" value="true" />
           <property name="hibernate.connection.datasource" value="java:/MyDataSource"/>
           <property name="hibernate.generate_statistics" value="true" />
           <property name="hibernate.cache.infinispan.statistics" value="true"/>
           <property name="hibernate.cache.use_second_level_cache" value="true" />
           <property name="hibernate.cache.use_query_cache" value="true" />
       </properties>
   </persistence-unit>
</persistence>

By defining your own region factory, you're risking not using the actual Infinispan instance that's managed by EAP that you're checking stats on.

Check the EAP 7 documentation for more info.

Answer 3

I would strongly suggest to avoid this combination:

• JBoss AS7 is ancient to the point of not being fully compatible with Java 8
• Infinispan 8 requires Java 8

Since you state that you can't upgrade to WildFly (11 is coming!) you're behind by almost 4 major versions: that's a lot of missing bugfixes and improvements, but also a significant difference in generations of libraries you're combining.

I would suggest to either:

• Stick with Ehcache
• Stick with the older version of Hibernate ORM included in your app server
• Update to WildFly 11 :-)
• get a supported JBoss EAP 7.x

I apologize for the commercial reference to EAP but it might be a good solution depending on why you can't upgrade to WildFly, as it's essentially combining later versions (especially compatibility fixed being backported) with a strict API backwards compatibility policy.

About Ehcache getting you into trouble with modules, I'd be happy to help with that. The module system is great and I'm confident this might be easy to solve, but that's probably better handled as a different question / forum.