Re-opening a Lucene index

Question

I have two separate classes - MyIndexer and MySearcher. The indexer class works fine and produces an index which is browsable with Luke. I see the docs in it. I'm using Lucene 4.0.0 just so that I can view what the data is using Luke.

I have a JUnit test class with two test methods. The first one invokes the indexer and creates the index. Then it closes the reader of the indexer. Then the second test method executes and tries to search for the results.

I get the following error:

org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed
    at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:252)
    at org.apache.lucene.index.SegmentReader.fields(SegmentReader.java:147)
    at org.apache.lucene.index.TermContext.build(TermContext.java:90)
    at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:167)
    at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:647)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:292)
    at com.foo.MySearcher.findArtifacts(MySearcher.java:76) // Line pointer shown in comments below
    ... (more irrelevant stacks)

MyIndexer relevant excerpt:

public class MyIndexer
{

    private IndexWriter writer;

    private IndexConfiguration indexConfiguration;


    public MyIndexer()
    {
    }

    public void initialize()
            throws IOException
    {
        StandardAnalyzer analyzer = new StandardAnalyzer(indexConfiguration.getLuceneVersion());

        final File indexDir = new File(indexConfiguration.getIndexDirectory()).getCanonicalFile();
        final FSDirectory index = FSDirectory.open(indexDir);

        IndexWriterConfig config = new IndexWriterConfig(indexConfiguration.getLuceneVersion(), analyzer);

        writer = new IndexWriter(index, config);
    }

    public void addData()
            throws IOException
    {
        // ...
        // Add some data here
        // ...

        writer.addDocument(doc);
    }

    public void close()
            throws IOException
    {
        if (writer != null)
        {
            writer.commit();
            writer.close();
        }
    }

    public IndexWriter getWriter()
    {
        return writer;
    }

    public void setWriter(IndexWriter writer)
    {
        this.writer = writer;
    }

    public IndexConfiguration getIndexConfiguration()
    {
        return indexConfiguration;
    }

    public void setIndexConfiguration(IndexConfiguration indexConfiguration)
    {
        this.indexConfiguration = indexConfiguration;
    }

}

MySearcher:

public class MySearcher
{

    private IndexReader reader;

    private IndexSearcher searcher;

    private IndexConfiguration indexConfiguration;


    public MySearcher()
    {
    }

    public void initialize()
            throws IOException
    {
        final File indexDir = new File(indexConfiguration.getIndexDirectory()).getCanonicalFile();
        final FSDirectory index = FSDirectory.open(indexDir);

        reader = DirectoryReader.open(index);
        searcher = new IndexSearcher(reader);
    }

    // This method is currently void for the sake of testing:
    public void findData(...)
            throws IOException
    {
        int hitsPerPage = 10;

        BooleanQuery query = new BooleanQuery();
        // ...
        // Does some irrelevant query-related stuff
        // ...

        TopScoreDocCollector collector = TopScoreDocCollector.create(hitsPerPage, true);
        searcher.search(query, collector); // Place where the exception occurs.
        ScoreDoc[] hits = collector.topDocs().scoreDocs;

        System.out.println("Found " + hits.length + " hits.");
        for (int i = 0; i < hits.length; ++i)
        {
            int docId = hits[i].doc;
            Document document = searcher.doc(docId);

            System.out.println(/*... Should print out some data from each matching document's fields */);
        }

        reader.close();
    }

    public IndexReader getReader()
    {
        return reader;
    }

    public void setReader(IndexReader reader)
    {
        this.reader = reader;
    }

    public IndexSearcher getSearcher()
    {
        return searcher;
    }

    public void setSearcher(IndexSearcher searcher)
    {
        this.searcher = searcher;
    }

    public IndexConfiguration getIndexConfiguration()
    {
        return indexConfiguration;
    }

    public void setIndexConfiguration(IndexConfiguration indexConfiguration)
    {
        this.indexConfiguration = indexConfiguration;
    }

}

MyLuceneTest:

public class MyLuceneTest
{

    public static final String INDEX_DIR = "target/test-indexes/index-001";

    private static IndexConfiguration configuration;


    @Before
    public void setUp()
            throws Exception
    {
        configuration = new IndexConfiguration();
        configuration.setIndexDirectory(INDEX_DIR);
    }

    @Test
    public void testVerboseIndexing()
            throws IOException
    {
        MyIndexer indexer = new MyIndexer();
        indexer.setIndexConfiguration(configuration);
        indexer.initialize();

        // Adding some data here
        ...
        indexer.addData(...);
        ...

        indexer.close();

        // Obviously, we're not doing any assertions right now.
    }

    @Test
    public void testSearch()
            throws IOException
    {
        MySearcher searcher = new MySearcher();
        searcher.setIndexConfiguration(configuration);
        searcher.initialize();

        searcher.findData(...);

        searcher.findData(...);

        searcher.findData(...);

        // Obviously, we're not doing any assertions right now.
    }

}

So... What's the deal here...? How does one normally open an index once the indexer has stopped executing and the app is done? I'm obviously doing something stupid, as I'm sure you can later re-open the index.

Answer 1

You are closing reader in findData method.

The best practice for Lucene is to close IndexSearcher after you finish your search and close IndexWriter when you are closing application. For IndexReader it really depends how often you need to have refreshed index. You can search for Near-Real Time search or use IndexReader.openIfChanged.

Answer 2

You're problem is here:

searcher.initialize();
searcher.findData(...);
searcher.findData(...);
searcher.findData(...);

combined with the fact that you are calling reader.close() at the end of the method findData. I'd guess that the first call to findData does not cause the problem, but the second call does.

MySearcher appears to be designed to keep the same reader and searcher open across multiple searches (which is good). It doesn't really make sense to close the reader after a search, to me. You should simply remove that call to reader.close().