Infinispan - Very slow for loading data with indexing. Can it be made faster?

Question

• I am trying out Infinispan cache to store java objects in local cache only mode.
• I want to query on both keys as well as some fields. So I am using query/indexing module of Infinispan.
• Lookup performance is very good on indexed fields. However loading all the items in cache is taking a huge amount of time compared to w/o indexing.
• e.g. for around 50k Objects, with Indexing Infinispan took 10 minutes to load the items in cache. Without indexing, it took only 2 sec.
• I wonder if Infinispan is this slow after indexing or if I am doing something grossly wrong.

Configuration infinispanConfiguration = new ConfigurationBuilder()
      .indexing()
         .enable()
         .indexLocalOnly(true)
      .build();

DefaultCacheManager cacheManager = new DefaultCacheManager(infinispanConfiguration)

;

    @Indexed @ProvidedId
    public class Book {
       @Field String title;
       @Field String description;
       @Field String author;
       @Field int yearOfPublication ;
       String briefDescription;
       int edition;
       boolean isBestSeller;
    }

Answer 1

I had the same issue.

The insert of 25000 Books (from Infinispan user guide) took 361 seconds

ConfigurationBuilder config = new ConfigurationBuilder();
config.indexing().index(Index.LOCAL);

The insert of 25000 Books took 1.6 seconds

ConfigurationBuilder config = new ConfigurationBuilder().indexing().setProperty("default.directory_provider", "ram").setProperty("default.indexmanager", "near-real-time");
config.indexing().index(Index.LOCAL);

Answer 2

As you know Infinispan uses Hibernate Search & Apache Lucene when you are using indexing module.

Based on your configuration you are using the default values for Lucene indexing. There are lots of options you can specify to improve indexing performance.

You can refer to the following for more information

[1] https://docs.jboss.org/author/display/ISPN/Querying+Infinispan [2] http://docs.jboss.org/hibernate/stable/search/reference/en-US/html_single/#lucene-indexing-performance