CODEWITHSUNDEEP
javaapachetestinglucene
Michał Rowicki
Michał Rowicki

Reputation: 1424

Simple test code for Lucene 4.3.0 doesn't work

I wrote simple code based on book "Lucene in action" which should show me how work different analyzers (with Lucene 4.3.0). But when I try to increment tokenStream, then I receive NullPointerException. Do you have any suggestions? Below I attach the code:

public static void main(String[] args) throws IOException {
    System.out.println("\n----");
    System.out.println("StandardAnalyzer");
    displayTokensWithFullDetails(new StandardAnalyzer(Version.LUCENE_43),
        "I'll email you at [email protected]");
}

public static void displayTokensWithFullDetails(Analyzer analyzer,
                                                  String text) throws IOException {

    TokenStream stream = analyzer.tokenStream("contents",
                                              new StringReader(text));

    //defining attributes

    while(stream.incrementToken()) {

      //print details of tokens

    }
    System.out.println();
  }

I can't start a loop. Do you have idea what am I doing wrong?

Here I add stacktrace:

----
StandardAnalyzer
Exception in thread "main" java.lang.NullPointerException
at           org.apache.lucene.analysis.standard.StandardTokenizerImpl.zzRefill(StandardTokenizerImpl.java:923)
at org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:1133)
at org.apache.lucene.analysis.standard.StandardTokenizer.incrementToken(StandardTokenizer.java:171)
at org.apache.lucene.analysis.standard.StandardFilter.incrementToken(StandardFilter.java:49)
at org.apache.lucene.analysis.core.LowerCaseFilter.incrementToken(LowerCaseFilter.java:54)
at org.apache.lucene.analysis.util.FilteringTokenFilter.incrementToken(FilteringTokenFilter.java:50)
at AnalyzerUtils.displayTokensWithFullDetails(AnalyzerUtils.java:115)
at AnalyzerUtils.main(AnalyzerUtils.java:169)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

Upvotes: 3

Views: 1528

Answers (1)

Michał Rowicki
Michał Rowicki

Reputation: 1424

Ok. I found solution. After initiation of TokenStream, StandardTokenizationImpl don't receive Reader object but receive null. When we type reset() on the stream StandardTokenizationImpl wiil receive finally our Stream. So my code should like as below:

public static void main(String[] args) throws IOException {
    System.out.println("\n----");
    System.out.println("StandardAnalyzer");
    displayTokensWithFullDetails(new StandardAnalyzer(Version.LUCENE_43),
        "I'll email you at [email protected]");
}

public static void displayTokensWithFullDetails(Analyzer analyzer,
                                                  String text) throws IOException {

    TokenStream stream = analyzer.tokenStream("contents",
                                              new StringReader(text));

    stream.reset(); //Add this line removes NullPointerException

    //defining attributes

    while(stream.incrementToken()) {

      //print details of tokens

    }
    System.out.println();
  }

Upvotes: 3

Related Questions