CODEWITHSUNDEEP
elasticsearchelastica
Alexandre Elshobokshy
Alexandre Elshobokshy

Reputation: 10922

Search in multiple indexes in elastica

I am looking for a way to search in more than one index at the same time using Elastica.

I have an index products, and an index user.

products contains {product_id, product_name, price} and user contains {product_id, user_name, date}. Knowing that the product_id in both of them is the same, in products each products_id is unique but in user they're not as a user can buy the same product multiple times.

Anyway, I want to automatically get the price of a product from the products index while searching through the user index.

I know that we can search over multiple indexes like so (correct me if I'm wrong) :

$search = new \Elastica\Search($client);
$search->addIndex('users')
        ->addType('user')
        ->addIndex('products')
        ->addType('product');

But the problem is, when I write an aggregation on the products_id for example and then create a new query with some filters :

$products_agg = new \Elastica\Aggregation\Terms('products_id');
$products_agg->setField('products_id')->setSize(0);

$query = new \Elastica\Query();
$query->addAggregation($products_agg);
$query->setQuery($bool);

$search->setQuery($query);

How does elastica know in which index to search? How can I link this products_id to the other index?

Upvotes: 4

Views: 2316

Answers (4)

gaurav9620
gaurav9620

Reputation: 1197

Basically you want to join two indexes based on a common field as in sql.

What you can do is model you data in the same index using join datatype

https://www.elastic.co/guide/en/elasticsearch/reference/master/parent-join.html

Index all documents in the same index ,

  • Make all product documents - parent.
  • Make all user documents as child

And the use parent-child aggregations and queries

https://www.elastic.co/guide/en/elasticsearch/reference/master/parent-join.html#_parent_join_queries_and_aggregations

NOTE: make sure of the performance implication of parent-child mapping https://www.elastic.co/guide/en/elasticsearch/reference/master/parent-join.html#_parent_join_and_performance

One more thing you can do is put all the information of the product with every user that buys it.

But this can unnecessarily waste you space and is not a good practice as per data rules are concerned.

But since this is a search engine and elasticsearch suggests that best is to normalise and duplicate data rather that using parent-child.

Upvotes: 1

Gasol
Gasol

Reputation: 2519

The Elastica library has support for Multi Search API, The multi search API allows to execute several search requests within the same API. The endpoint for it is _msearch.

The format of the requests is similar to the bulk API, The first line is header part that includes which index / indices to search on, The second line includes the typical search body requests.

{"index" : "products", "type": "products"}
{"query" : {"match_all" : {}}, "from" : 0, "size" : 10} // write your own query to get price
{"index" : "uesrs", "type" : "user"}
{"query" : {"match_all" : {}}} // query for user

Check test case in Multi/SearchTest.php to see how to use.

Upvotes: 1

ziad
ziad

Reputation: 404

You can't query two indices with different mappings. Best way to solve your problem is to just do two queries (application-side joins). First query you do the aggregations on the user and the second you get the prices.

Another option would be to add the price to the user index. Sometimes you have to sacrifice a little space for better usability.

Upvotes: 0

Mohamed Assem
Mohamed Assem

Reputation: 164

you can try the following: 1- naming indexes with specific name like the following

myFirstIndex-myProjectName
mySecIndex-myProjectName
myThirdIndex-myProjectName

and so on.

2- that's give me the ability using * in the field of indexes to search because it accepts wildcard so i can search across multiple fields like this using kibana Dev Tools

GET *-myProjectName/_search
{
  "_source": {
    "excludes": [ "*" ]
  },
  "query": { "match_all": {} },
}

this will search on each index includes -myProjectName.

Upvotes: 0

Related Questions