CODEWITHSUNDEEP
solrcluster-analysisnoclassdeffounderror
The Bndr
The Bndr

Reputation: 13394

Solr, clustering (carrot) and NoClassDefFoundError

i'm running Solr 3.4 and would like to use the clusteringComponent. Following this tutorial: http://wiki.apache.org/solr/ClusteringComponent in combination with default entries at the solrconfig.xml i have the following configuration @solrconfig.xml

  <searchComponent name="clustering" 
                   enable="${solr.clustering.enabled:true}"
                   class="org.apache.solr.handler.clustering.ClusteringComponent" >

    <!-- Declare an engine -->
    <lst name="engine">
      <str name="name">default</str>
      <str name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
      <str name="LingoClusteringAlgorithm.desiredClusterCountBase">20</str>
    </lst>

    <lst name="engine">
      <str name="name">stc</str>
      <str name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
    </lst>
  </searchComponent>


<requestHandler name="/cl" class="solr.SearchHandler" >
  <lst name="defaults">
    <str name="echoParams">explicit</str>

    <bool name="clustering">true</bool>
    <str name="clustering.engine">default</str>
    <bool name="clustering.results">true</bool>

    <!-- Fields to cluster on -->
    <str name="carrot.title">UEBSCHRIFT</str>
    <str name="carrot.snippet">TEXT</str>
  </lst>

So if i try to use the requestHandler http://server:8080/solr/mycore/cl?q=*:* i get the following Java exception:

java.lang.NoClassDefFoundError: com.carrotsearch.hppc.ObjectContainer
    at java.lang.J9VMInternals.verifyImpl(Native Method)
    at java.lang.J9VMInternals.verify(J9VMInternals.java:72)
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:134)
    at org.carrot2.text.preprocessing.pipeline.BasicPreprocessingPipeline.&lt;init&gt;(BasicPreprocessingPipeline.java:106)
    at org.carrot2.text.preprocessing.pipeline.CompletePreprocessingPipeline.&lt;init&gt;(CompletePreprocessingPipeline.java:32)
    at org.carrot2.clustering.lingo.LingoClusteringAlgorithm.&lt;init&gt;(LingoClusteringAlgorithm.java:129)
    at java.lang.J9VMInternals.newInstanceImpl(Native Method)
    at java.lang.Class.newInstance(Class.java:1325)
    at org.carrot2.util.pool.SoftUnboundedPool.borrowObject(SoftUnboundedPool.java:80)
    at org.carrot2.core.PoolingProcessingComponentManager.prepare(PoolingProcessingComponentManager.java:128)
    at org.carrot2.core.Controller.process(Controller.java:333)
    at org.carrot2.core.Controller.process(Controller.java:240)
    at org.apache.solr.handler.clustering.carrot2.CarrotClusteringEngine.cluster(CarrotClusteringEngine.java:136)
    at org.apache.solr.handler.clustering.ClusteringComponent.process(ClusteringComponent.java:91)
    at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:194)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1368)
    at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:356)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:252)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
    at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
    at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
    at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
    at java.lang.Thread.run(Thread.java:735)
Caused by: java.lang.ClassNotFoundException: com.carrotsearch.hppc.ObjectContainer
    at java.lang.Throwable.&lt;init&gt;(Throwable.java:80)
    at java.lang.ClassNotFoundException.&lt;init&gt;(ClassNotFoundException.java:76)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:419)
    at java.lang.ClassLoader.loadClass(ClassLoader.java)
    at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:923)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:609)
    ... 31 more

The point is, that I have no idea what this mean. I'm searching for hours without finding an solution.

By the way: I'm running on tomcat with the following options:

export CATALINA_OPTS="-Dsolr.clustering.enabled=true"

(is this still required in Solr 3.4?)

The catalina option is part of the java command, as you can see with ps -efa

/usr/lib64/jvm/java-1_6_0-ibm-1.6.0/jre//bin/java -Djava.util.logging.config.file=/opt/tomcat6/conf/logging.properties -Xms2048m -Xmx2048m -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dsolr.clustering.enabled=true -Djava.endorsed.dirs=/opt/tomcat6/endorsed -classpath /opt/tomcat6/bin/bootstrap.jar -Dcatalina.base=/opt/tomcat6 -Dcatalina.home=/opt/tomcat6 -Djava.io.tmpdir=/opt/tomcat6/temp org.apache.catalina.startup.Bootstrap start

Does anyone has an idea what i could do to solve this problem?

//Update: if i add hppc-0.3.4-jdk15.jar, i get the following error:

java.lang.NoClassDefFoundError: org.apache.mahout.math.matrix.DoubleMatrix2D
    at java.lang.J9VMInternals.verifyImpl(Native Method)
    at java.lang.J9VMInternals.verify(J9VMInternals.java:72)
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:134)
    at org.carrot2.clustering.lingo.LingoClusteringAlgorithm.&lt;init&gt;(LingoClusteringAlgorithm.java:134)
[...]
Caused by: java.lang.ClassNotFoundException: org.apache.mahout.math.matrix.DoubleMatrix2D
    at java.lang.Throwable.&lt;init&gt;(Throwable.java:80)
    at java.lang.ClassNotFoundException.&lt;init&gt;(ClassNotFoundException.java:76)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:419)
    at java.lang.ClassLoader.loadClass(ClassLoader.java)
    at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:923)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:609)
    ... 29 more

it looks like, i have to install an mahout archive, but i think, all packages for clustering are included in Solr 3.4?! It looks like, i'm on the wrong way?!

Upvotes: 0

Views: 2505

Answers (4)

jvea
jvea

Reputation: 59

With solR 4 copy this file in folder conf

http://svn.apache.org/repos/asf/labs/alike/trunk/demo/solrhome/collection1/conf/solrconfig.xml

Upvotes: 0

Dawid Weiss
Dawid Weiss

Reputation: 26

Why not use Solr's default packaging (this is officially supported)? It ships with Jetty and will save you the headaches connected with classpath because things are already configured.

Answering your question, you'll need all the JARs from Solr's default clustering extension folders; for 4.0 alpha this would be: contrib/clustering/lib/*.jar

carrot2-core-3.5.0.jar hppc-0.3.3.jar jackson-core-asl-1.7.4.jar jackson-mapper-asl-1.7.4.jar mahout-collections-0.3.jar mahout-math-0.3.jar simple-xml-2.4.1.jar

Upvotes: 0

Has QUIT--Anony-Mousse
Has QUIT--Anony-Mousse

Reputation: 77454

Did you add the Mahout Math package?

It seems to be a separate package.

NoClassDefFoundError: org.apache.mahout.math.matrix.DoubleMatrix2D
                      ^^^^^^^^^^^^^^^^^^^^^^

Upvotes: 0

Jayendra
Jayendra

Reputation: 52779

If you are using Solr with tomcat as a seperate instance, you would need to copy the jars so that they are available for Solr.

Quote from README.txt

NOTE: This Solr example server references certain Solr jars outside of this server directory for non-core modules with statements in solrconfig.xml. If you make a copy of this example server and wish to use the ExtractingRequestHandler (SolrCell), DataImportHandler (DIH), UIMA, the clustering component, or other modules in "contrib", you will need to copy the required jars into solr/lib or update the paths to the jars in your solrconfig.xml.

Check for the clustering and carrot jars in solrconfig.xml.
Probably you are missing hppc-0.3.4-jdk15.jar

Upvotes: 2

Related Questions