Solr supporting varying facets for different types of products

Question

I am using Solr for indexing different types of products. The product types (category) have different facets. For example:

camera: megapixel, type (slr/..), body construction, ..
processors: no. of cores, socket type, speed, type (core i5/7)
food: type, origin, shelf-life, weight
tea: type (black/green/white/..), origin, weight, use milk?
serveware: type, material, color, weight
...

And they have common facets as well:

Brand, Price, Discount, Listing timeframe (like new), Availability, Category

I need to display the relevant facets and breadcrumbs when user clicks on any category, or brand page or a global search across all types of products. This is same as what we see on several ecommerce sites.

The query that I have is: Since the facet types are more or less unique across different types of products, do I put them in separate schemas? Is that the way to do it? The fundamental worry is that those fields will not have any data for other types of products. And are there any implementation principles here that makes retrieving the respective faces for a given product type easier?

I would like to have a design that is scalable to accommodate lots of items in each product type as we go forward, as well as that is easy to use and performance oriented, if possible. Right now I am having a single instance of Solr.

Answer 1

The only risk of underpopulated facets are when they misrepresent the search. I'm sure you've used a search site where the metadata you want to facet on is underpopulated so that when you apply the facet you also eliminate from your result set a number of records that should have been included. The thing to watch is that the facet values are populated consistently where they are appropriate. That means that your "tea" records don't need to have a number of cores listed, and it won't impact anything, but all of your "processor" records should, and (to whatever extent possible) they should be populated consistently. This means that if one processor lists its number of cores as "4", and another says "quadcore", these are two different values and a user applying either facet value will eliminate the other processor from their result. If a third quadcore processor is entirely missing the "number of cores" stat from the no_cores facet field (field name is arbitrary), then your facet could be become counterproductive.

So, we can throw all of these records into the same Solr, and as long as the facets are populated consistently where appropriate, it's not really necessary that they be populated for all records, especially when not applicable.

Applying facets dynamically

Most of what you need to know is in the faceting documentation of Solr. The important thing is to specify the appropriate arguments in your query to tell Solr which facets you want to use. (Until you actually facet on a field, it's not a facet but just a field that's both stored="true" and indexed="true".) For a very dynamic effect, you can specify all of these arguments as part of the query to Solr.

&facet=true

This may seem obvious, but you need to turn on faceting. This argument is convenient because it also allows you to turn off faceting with facet=false even if there are lots of other arguments in your query detailing how to facet. None of it does anything if faceting is off.

&facet.field=no_cores

You can include this field over and over again for as many fields as you're interested in faceting on.

&facet.limit=7
&f.no_cores.facet.limit=4

The first line here limits the number of values for returned by Solr for each facet field to 7. The 7 most frequent values for the facet (within the search results) will be returned, with their record counts. The second line overrides this limit for the no_cores field specifically.

&facet.sort=count

You can either list the facet field's values in order by how many appear in how many records (count), or in index order (index). Index order generally means alphabetically, but depends on how the field is indexed. This field is used together with facet.limit, so if the number of facet values returned is limited by facet.limit they will either be the most numerous values in the result set or the earliest in the index, depending on how this value is set.

&facet.mincount=1

There are very few circumstances that you will want to see facet values that appear zero times in your search results, and this can fix the problem if it pops up.

The end result is a very long query:

http://localhost/solr/collecion1/search?facet=true&facet.field=no_cores&
 facet.field=socket_type&facet.field=processor_type&facet.field=speed&
 facet.limit=7&f.no_cores.facet.limit=4&facet.mincount=1&defType=dismax&
 qf=name,+manufacturer,+no_cores,+description&
 fl=id,name,no_cores,description,price,shipment_mode&q="Intel"

This is definitely effective, and allows for the greatest amount of on-the-fly decision-making about how the search should work, but isn't very readable for debugging.

Applying facets less dynamically

So these features allow you to specify which fields you want to facet on, and do it dynamically. But, it can lead to a lot of very long and complex queries, especially if you have a lot of facets you use in each of several different search modes.

One option is to formalize each set of commonly used options in a request handler within your solrconfig.xml. This way, you apply the exact same arguments but instead of listing all of the arguments in each query, you just specify which request handler you want.

<requestHandler name="/processors" class="solr.SearchHandler">
 <lst name="defaults">
  <str name="defType">dismax</str>
  <str name="echoParams">explicit</str>
  <str name="fl">id,name,no_cores,description,price,shipment_mode</str>
  <str name="qf">name, manufacturer, no_cores, description</str>
  <str name="sort">score desc</str>
  <str name="rows">30</str>
  <str name="wt">xml</str>
  <str name="q.alt">*</str>
  <str name="facet.mincount">1</str>
  <str name="facet.field">no_cores</str>
  <str name="facet.field">socket_type</str>
  <str name="facet.field">processor_type</str>
  <str name="facet.field">speed</str>
  <str name="facet.limit">10</str>
  <str name="facet.sort">count</str>
 </lst>
 <lst name="appends">
  <str name="fq">category:processor</str>
 </lst>
</requestHandler>

If you set up a request hander in solrconfig.xml, all it does is serve as a shorthand for a set of query arguments. You can have as many request handlers as you want for a single solr index, and you can alter them without rebuilding the index (reload the Solr core or restart the server application (JBoss or Tomcat, e.g.), to put changes into effect).

There are a number of things going on with this request handler that I didn't get into, but it's all just a way of representing default Solr request arguments so that your live queries can be simpler. This way, you might make a query like:

http://localhost/solr/collection1/processors?q="Intel"

to return a result set with all of your processor-specific facets populated, and filtered so that only processor records are returned. (This is the category:processor filter, which assumes a field called category where all the processor records have a value processor. This is entirely optional and up to you.) You will probably want to retain the default search request handler that doesn't filter by record category, and which may not choose to apply any of the available (stored="true" and indexed="true") fields as active facets.