CODEWITHSUNDEEP
lucenespring-data-jpamicroserviceshibernate-search
Naveen
Naveen

Reputation: 25

FileSystem lock while creating Indexer

I have used hibernate-search where I have annotated the domains with @Indexed, @Field and many more. My project is based on multiple microservices like

  1. Search Service - which upon starting read the data from DB and create the indexes. (code has been mentioned below):
@Transactional(readOnly = true)
public void initializeHibernateSearch() {
    try {
        FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(centityManager);
            fullTextEntityManager.createIndexer().startAndWait();
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}
  1. MicroService 1 - which updates the index when any domain is inserted or updated.

The issue I am facing is that when Search Service is started and creates the indexes, it takes the locks on the index files and never releases the locks. and when Microservice 1 tries to update the index upon insertion or update, it throws an exception as below

org.apache.lucene.store.LockObtainFailedException: Lock held by another program: /root/data/index/default/Event/write.lock at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:118) at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) at org.apache.lucene.index.IndexWriter.(IndexWriter.java:776) at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:126) at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:92) at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117) at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriterDelegate(AbstractWorkspaceImpl.java:203) at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:81) at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:46) at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.applyChangesets(SyncWorkProcessor.java:166) at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.run(SyncWorkProcessor.java:152) at java.lang.Thread.run(Thread.java:748)

Could you please let me know what is the right approach to use hibernate-search in microservices architecture.

Upvotes: 0

Views: 647

Answers (1)

Sanne
Sanne

Reputation: 6107

There are several options. My recommendation is the first one, as it best fits the architectural spirit of Micro services.

  1. Each micro service has an independent index: don't share the index directory.
  2. Use the master/slave architecture described in the docs, so to have a single service write to the index - the other services will have to delegate to the single writer. Indexes can be replicated over network based filesystem, or using Infinispan.
  3. Disable exclusive_index_use, a configuration property (also described in the docs). I'm listing this for completeness, but this is generally a bad idea; it's going to be much slower and it's going to be your responsibility that one service busy writing won't timeout another service needing to write.

Upvotes: 2

Related Questions