CODEWITHSUNDEEP
solr
FAlonso
FAlonso

Reputation: 494

Solr - Efficient way to search across multiple cores?

I am building a user-facing search engine for movies, music and art where users perform free-text queries (like Google) and get the desired results. Right now, I have movies, music and art data indexed separately on different cores and they do not share a similar schema. For ease of maintenance, I would prefer having them in separate cores as it is now.

Till date, I have been performing my queries individually on each core, but I want to expand this capability to perform a single query that runs across multiple cores/indexes. Say I run a query by the name of the artist and the search engine returns me all the relevant movies, music and art work they have done. Things get tricky here.

Based on my research, I see that there are two options in this case.

  1. Create a fourth core, add shards attribute that points to my other three cores. Redirect all my queries to this core to return required results.
  2. Create a hybrid index merging all three schema's and perform queries on this index.

With the first option, the downside I see is that the keys need to be unique across my schema for this to work. I am going to have the key artistName across all my cores, this isn't going to help me.

I really prefer keeping my schema separately, so I do not want to delve into the second option as such. Is there a middle ground here? What would be considered best practice in this case?

Linking other SO questions that I referred here:

Best way to search multiple solr core
Solr Search Across Multiple Cores
Search multiple SOLR core's and return one result set

Upvotes: 1

Views: 397

Answers (1)

Abhijit Bashetti
Abhijit Bashetti

Reputation: 8678

I am of the opinion that you should not be doing search across multiple core.

Solr or Nosql databases are not meant for it. These database are preferred when we want to achieve faster response which is not possible with the RDBMS as it involves the joins.

The joins in the RDBMS slower's the performance of your query as the data grows in size.

To achieve the faster response we try to convert the data into flat document and stores it in the NoSQL database like MongoDB, Solr etc..

  1. You should covert your data into such a way that, it should be part of single document.

  2. If above option is not possible then create individual cores and retrieve the specific data from specific core with multiple calls.

  3. You can also check for creating parent child relation document in solr.

  4. Use solr cloud option with solr streaming expression.

Every option has its pros and cons. It all depends on your requirement and what you can compromise.

Upvotes: 1

Related Questions