CODEWITHSUNDEEP
hadoopsolrbigdata
xxlali
xxlali

Reputation: 1046

Integration of Hadoop and Solr

According to my research, I can integrate hadoop and solr. I have downloaded and install both of them. But couldn't integrate them with each other. And also I couldn't find a proper tutorial for this purpose.

I use Ubuntu 14.04.02, Apache Hadoop 2.6.0 and Solr 5.2.1.

How can I integrate Hadoop and Solr on my machine?

Note: I installed hadoop as Single Node. Also I am very beginner about this concepts.

Upvotes: 1

Views: 902

Answers (2)

ganeshTripathi
ganeshTripathi

Reputation: 31

you can use Solr with hadoop in two ways

  1. document based
  2. using lily indexers With Hbase

so if you want to use a document that present in HDFS to be indexed by SOLR. you need to follow following steps: Step A.

solrctl --zk zookeeper_server:port/solr --solr solr-server:port/solr instancedir --generate <path of collection>/collection_name

edit /collection_name/conf/schema.xml with your attributes that present in Data to be indexed 

solrctl --zk zookeeper_server:port/solr --solr solr-server:port/solr instancedir --create <collection_name> <path of collection>/collection_name
solrctl --zk zookeeper_server:port/solr --solr solr-server:port/solr  collection --create <collection_name> -s <num_of_solr_shard> -r <num_of_solr_replication>

you can any number to , but

* <= number of solr nodes in cluster eg If you have 7 nodes , you can have 3,2 or 2,3 as per need.

so for your case it would be 1 & 1.

Step B. once collection is been created , Data can be indexed by following command

curl http://solr-server:port/solr/<collection_name>/update/csv --data-binary @<path_of_data_file_in_linux> -H 'Content-type:text/plain; charset=utf-8'

If you want to index Hbase Data follow Step A. to create Solr Collection, & Then use Lily Indexer(key value indexer) to create indexer on hbase , after that that data can be seen on SOLR as XML or JSON.

Upvotes: 1

Amar
Amar

Reputation: 3845

I would recommend you to read about Cloudera Search (http://www.cloudera.com/content/cloudera/en/documentation/cloudera-search/v1-latest/Cloudera-Search-User-Guide/csug_introducing.html)

This is basically an open source project by cloudera integrating Hadoop and Solr.

Upvotes: 0

Related Questions