CODEWITHSUNDEEP
mongodbsolr
Nanda Kumar
Nanda Kumar

Reputation: 2226

Solr with mongodb connection

Connect my mongodb database with solr cloud using mongo-connector? can any one tell me the steps to connect the mongo database to fetch data to the solr for indexing

Upvotes: -1

Views: 3134

Answers (1)

Nanda Kumar
Nanda Kumar

Reputation: 2226

Step 1: Installing the Mongo Connector

To install the mongo connector run

 Pip install mongo-connector

Step 2: Creating a Solr Core 

 ./bin/solr create -c <corename>-p 8983 -s 3 -rf 3

Step 3:Configuring to Solr

The fields in the mongodb documents to be indexed are specified in the schema.xml configuration file . open the schema.xml in a vi editor. As

vi/solr/solr-6.6.2/server/solr/configsets/data_driven_schema_configs/ 
conf/schema.xml

Step 4: Mongo Connector also stores the metadata associated with the each mongodb document it indexes in fields ns and _ts. Also add the ns and _ts fields to the schema.xml.

<schema>
<?xml version="1.0" encoding="UTF-8" ?>
<schema name="example" version="1.5">
<field name="time_stamp" type="string" indexed="true"  stored="true"  
multiValued="false" />
<field name="category" type="string" indexed="true"  stored="true"  
multiValued="false" />
<field name="type" type="string" indexed="true"  stored="true"  
multiValued="false" />
<field name="servername" type="string" indexed="true"  stored="true"  
multiValued="false" />
<field name="code" type="string" indexed="true"  stored="true"  
multiValued="false" />
<field name="msg" type="string" indexed="true"  stored="true"  
multiValued="false" />
<field name="_ts" type="long" indexed="true" stored="true" />
<field name="ns" type="string" indexed="true" stored="true"/>
 <field name="_version_" type="long" indexed="true" stored="true"/>

</schema>

Step 5:We also need to configure the org.apache.solr.handler.admin.LukeRequestHandler request handler in the solrconfig.xml. 

Open the solrconfig.xml in the vi editor.

 vi ./solr-5.3.1/server/solr/configsets/basic_configs/conf/solrconfig.xml

Specify the request handler for the Mongo Connector.

*<requestHandler name="/admin/luke" 
class="org.apache.solr.handler.admin.LukeRequestHandler" />*

Also configure the auto commit to true so that Solr auto commits the data from MongoDB after the configured time.

<autoCommit>
<maxTime>15000</maxTime>
<openSearcher>true</openSearcher>
</autoCommit>

Step 6:Solr needs to be restarted

Bin/solr restart -force

Starting MongoDB Server Mongo Connector requires a MongoDB replica set to be running for it to index MongoDB data in Solr. A replica set is a cluster of MongoDB servers that implements replication and automatic failover. A replica set could consist of only a single server and  the port is specified as 27017, the data directory for MongoDB is specified as /data/db and the replica set name is specified as rs0 with the –replSet option.

Sudo mongod --port 27017 --dbpath /data/db --replSet rs0

Step 7: Starting MongoDB Shell Start the Mongodb shell with the following command

Mongo

MongoDB shell gets started. We need to initiate the replica set. Run the following command to initiate the replica set.

 rs.initiate()

Step 8:Starting MongoDB Connector and Indexing MongoDB Database with Solr Run Mongo-connector command as below

mongo-connector --unique-key=id –n solr.wlslog -m localhost:27017 -t 
http://xx.xxx.xxx.xx:8983/solr/wlslog -d solr_doc_manager

In above statement solr.wlslog--> solr is a databasename wlslog is a collection name Solr/wlslog--> wlslog is a CORE name

For Future reference use the below link https://www.toadworld.com/platforms/nosql/b/weblog/archive/2017/02/03/indexing-mongodb-data-in-apache-solr

Upvotes: 3

Related Questions